Testing

LSEG Tick History REST API with CURL
Zoya Farberov
Developer Advocate Developer Advocate

Introduction

In this article we will go over testing LSEG Tick History API extraction workflow with curl.  Using on demand TickHistoryRawExtractionRequest as an example, we will:

  1. Authenticate to obtain a valid token
  2. Submit on-demand extraction request
  3. Ascertain that the extraction request processing has been completed
  4. Obtain the extraction result

A specific version of curl is not required to do this type of testing.

Another on-demand request can be substituted in the workflow, in place of TickHistoryRawExtractionRequest, and tested with curl.

Authentication - Obtaining A Valid Token

    	
            curl -i -X POST -k -H "Content-Type: application/json" -H "Prefer: respond-async"  https://selectapi.datascope.refinitiv.com/RestApi/v1/Authentication/RequestToken -H 'X-Direct-Download: True' -x MYVALIDPROXY:MYPROXYPORT -d @creds.txt

with creds.txt file stored in the run directory and containing:

    	
            
{


 "Credentials": { 


      "Username": "MYVALIDDSSUSERID", 


      "Password": "MYVALIDDSSPASSWORD"


 }


}

The result from this call should contain a valid token, that is good for 24 hours of use with other requests, similar to

    	
            
HTTP/1.1 200 OK


Cache-Control: no-cache


Pragma: no-cache


Content-Type: application/json; charset=utf-8


Expires: -1


Server: Microsoft-IIS/10.0


BeginRequestTime: 22:21:25.9504696


BeginRequestDate: 2021-08-23


X-Request-Execution-Correlation-Id: CiD/0/AAAAAA.07ae7673cf8d0f48/RA


X-App-Id: Custom.RestApi


X-App-Version: 15.0.155.64


CPUUtilization: 14.20399


RequestsPerSec: 4.75508


W3WP-PrivateBytes: 7196


Date: Mon, 23 Aug 2021 22:21:25 GMT


Content-Length: 405


 


{"@odata.context":"https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#Edm.String","value":"_YuBtkhCDqU8h3S7amxPa4pDtdLKcuefiYq0ab9s23RPfXmi2c7By2ZZZA1VcnoerhtUfsSQnyBlmAN0sDyB-iXSDxm8rxefnv21c7x_tDncPB_Cnp22SZoLhiXyaqXCXmuZcuA1kX3zsyHTGQo_CM5cDzhWUdpbVeiLKpIsXfrsj47kfCTOem1w_RjXvW2G_fsdot7XDt_-krdXGF0L-mpkLtdPJPXWPoBkoDGwWAhygraUpslQy-QhUucEX5Ra8ov5f7Hhu1ZYq4LhKxHZ51aILypJy3pWefVHXqcp21Ts"}

Submit On-Demand Extraction Request

We use the valid token obtained in the previous step to submit TickHistoryRawExtractionRequest

 

    	
            curl -i -X POST -k -H "Content-Type: application/json" -H "Prefer: respond-async" "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRaw" -H "Authorization: Token MYVALIDTOKEN" -H 'Content-Type: application/json' -H 'X-Direct-Download: True' -x MYVALIDPROXY:MYPROXYPORT -d @requestBody.txt

with file requestBody.txt stored in the run folder and containing:

    	
            
{ 


  "ExtractionRequest": { 


    "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest", 


    "IdentifierList": { 


      "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  


      "InstrumentIdentifiers": [{


        "Identifier": "CARR.PA",


        "IdentifierType": "Ric"


      }]


    },


    "Condition": {


      "MessageTimeStampIn": "GmtUtc",


      "ReportDateRangeType": "Range",


      "QueryStartDate": "2016-09-29T12:00:00.000Z",


      "QueryEndDate": "2016-09-29T12:10:00.000Z",


      "ExtractBy": "Ric",


      "SortBy": "SingleByRic",


      "DomainCode": "MarketPrice",


      "DisplaySourceRIC": true


    }


  }


}

In result we look for two pieces for information, status and ExtractionID:

    	
            
HTTP/1.1 202 Accepted


Cache-Control: no-cache


Pragma: no-cache


Expires: -1


Location: https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x07ae64718b9d0f41')


Server: Microsoft-IIS/10.0


BeginRequestTime: 21:30:25.3627291


BeginRequestDate: 2021-08-23


Status: InProgress


Progress:


X-Request-Execution-Correlation-Id: CiD/19945/AAAAAA.07ae64718bad0f41/RA


X-App-Id: Custom.RestApi


X-App-Version: 15.0.155.64


CPUUtilization: 26.47529


RequestsPerSec: 0


W3WP-PrivateBytes: 7120


Date: Mon, 23 Aug 2021 21:30:55 GMT


Content-Length: 0

The status code of 202 means that request is valid and is accepted for processing, the status code of 200 means that request is valid, and request processing is completed.  Any other status means that request should be examined as it is not valid in some aspect of the other.

The extraction Id will be used in the next steps to retrieve the results, as well as examine the status of the request until it's processing is completed, resulting in status of 200.

 Checking Status of the Extraction Using Extraction Id

This step is required if the result from the previous step is not completed = 200 but is accepted = 202.

Next we use the extraction Id just obtained to check on the status of the request, waiting in between the checks, until the status that is returned is completed = 200  

    	
            curl -i -X GET -k -H "Content-Type: application/json" -H "Prefer: respond-async" "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x07ae64718b9d0f41')" -H "Authorization: Token MYVALIDTOKEN" -H 'Content-Type: application/json' -H 'X-Direct-Download: True' -x MYPROXY:MYPROXYPORT

the result being the status of the request:

    	
            
HTTP/1.1 200 OK


Cache-Control: no-cache


Pragma: no-cache


Content-Type: application/json; charset=utf-8


Expires: -1


Server: Microsoft-IIS/10.0


BeginRequestTime: 21:57:10.4224282


BeginRequestDate: 2021-08-23


X-Request-Execution-Correlation-Id: CiD/19945/AAAAAA.07ae85c312ed0f67/RA


X-App-Id: Custom.RestApi


X-App-Version: 15.0.155.64


CPUUtilization: 17.59716


RequestsPerSec: 7.619745


W3WP-PrivateBytes: 1569704


Date: Mon, 23 Aug 2021 21:57:10 GMT


Content-Length: 1414

Obtaining a Ready Extraction Result

Once the result is ready we can GET the result and store it into a file

    	
            curl -X GET "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/RawExtractionResults('0x07ae64718b9d0f41')/$value" -H "Authorization: Token MYVALIDTOKEN" -H 'Content-Type: application/gzip' -H 'X-Direct-Download: True' -x MYVALIDPROXY:MYPROXYPORT -v -N -o "tst.csv.gz"

We can next unzip the contents of file tst.csv.gz:

    	
            
#RIC,Domain,Date-Time,GMT Offset,Type,MsgClass/FID number,UpdateType/Action,FID Name,FID Value,FID Enum String,PE Code,Template Number,Key/Msg Sequence Number,Alias Underlying RIC,Number of FIDs


CARR.PA,Market Price,2016-09-29T12:00:00.942730204Z,+2,Raw,UPDATE,UNSPECIFIED,,,,8132,,2800,,13


,,,,FID,4981,,TRTN,0.89,


,,,,FID,4982,,TRTN_3MT,5.85,


,,,,FID,5615,,YR_TRTN,-5.23,


,,,,FID,5619,,YTD_TRTN,-7.95,


,,,,FID,9257,,ASY_VA_DT,2016-09-29,


,,,,FID,9959,,TRTN_1W,-0.59,


,,,,FID,9960,,TRTN_1M,4.97,


,,,,FID,9961,,TRTN_2Y,1.34,


,,,,FID,9962,,TRTN_3Y,-0.39,


,,,,FID,9963,,TRTN_4Y,60.61,


,,,,FID,9964,,TRTN_5Y,52.58,


,,,,FID,9965,,MTD_TRTN,2.29,


,,,,FID,9966,,QTD_TRTN,3.29,


CARR.PA,Market Price,2016-09-29T12:00:03.161984291Z,+2,Raw,UPDATE,QUOTE,,,,8132,,2816,,14


,,,,FID,3866,,AUC_BID,,


,,,,FID,3862,,AUC_BIDSIZ,,


,,,,FID,22,,BID,23.24,


,,,,FID,6579,,BID_COND_N,0,


,,,,FID,30,,BIDSIZE,2752,

Another, very popular approach, that is used to test LSEG Tick History and Datascope Select REST APIs is tool Postman.  Detailed info on testing with Postman complete with starter example deck can be found in REST API Tutorials mentioned in References section

References