DataScope Select - REST API

Close Menu
Expand All Collapse All
Introductory Tutorials Tutorials Introduction Programming without SDK Tutorial REST API Tutorials REST API Tutorials Introduction REST API Tutorial 1: Connecting to the DSS server REST API Tutorial 2: On Demand End of Day Extraction REST API Tutorial 3: On Demand intraday extraction, embargo REST API Tutorial 4: On Demand price history extraction REST API Tutorial 5: On Demand corporate actions extraction REST API Tutorial 6: On Demand ownership data extraction REST API Tutorial 7: On Demand T&C extraction REST API Tutorial 8: On Demand composite extraction REST API Tutorial 9: On Demand extraction: instrument list REST API Tutorial 10: GUI control calls: immediate extract REST API Tutorial 11: Search by Instrument REST API Tutorial 12: Search for an Equity REST API Tutorial 13: Search for a Future or Option REST API Tutorial 14: On Demand price history extraction raw .Net SDK Tutorials .Net SDK Tutorial 1: Connecting to the DSS server .Net SDK Tutorial 2: GUI control calls: List, report, sched .Net SDK Tutorial 3: GUI control calls: Validate, extraction .Net SDK Tutorial 4: GUI control calls: Embargo, note files .Net SDK Tutorial 5: On Demand: EoD extraction .Net SDK Tutorial 6: On Demand: EoD extraction, file I/O .Net SDK Tutorial 7: On Demand: large instrument lists .Net SDK Tutorial 8: On Demand: Terms & Conditions .Net SDK Tutorial 9: On Demand: Composite extraction .Net SDK Tutorial 10: Search by Instrument .Net SDK Tutorial 11: Search for Equity .Net SDK Tutorial 12: Search for Future or Option

REST API Tutorial 14: On Demand price history extraction raw

Last update November 2019
Environment Any
Language Any HTTP is supported
Compilers None
Prerequisites DSS login, internet access
Source code Below

Tutorial purpose

This tutorial explains how to:

  • Retrieve price history data from the DSS server, in compressed CSV format, using an on demand request.

The On Demand extractions in tutorials 2-8 use the ExtractWithNotes endpoint, whereas this tutorial uses the ExtractRaw endpoint, in a use case similar to that in tutorial 4.

Instead of delivering data in JSON format, this endpoint delivers data in a Gzipped CSV format. This is usefull for large data requests. For more information on these two endpoints, please refer to the Tutorials introduction.

Important note:

The legacy Timeseries Pricing extraction will be removed end February 2020, the more powerful Price History extraction replaces it. This tutorial was updated in May 2019 to reflect this. If you are using the Timeseries Pricing call, you must modify your code. As part of the migration process, it is important to understand that, in some instances, field content extracted from the Timeseries Pricing report will be available under different field names in the Price History report. It is also possible that some previously populated values in the Timeseries Pricing report will be blank in the Price History report. This is due to data mapping changes. For more information, please refer to the Product Change Notification PCN 10897.

 

Table of contents

 

Get available field list for price history - HTTP request

If you do not know what fields are available, you can request a list of them. For a price history request this was covered in Tutorial 4.

 

Get price history data - HTTP request

The On Demand extractions in tutorials 2-8 use the ExtractWithNotes endpoint, whereas this tutorial uses the ExtractRaw endpoint, in a use case similar to that in tutorial 4. Instead of delivering data in JSON format, this endpoint delivers data in a Gzipped CSV format.

Apart from the endpoint, this request has the same format as that in Tutorial 4. Please refer to that tutorial for the details.

Note: the tutorial 14 sample in the Postman collection retrieves data for 2 RICs from 1996 till end 2017, whereas tutorial 4 requests 3 RICs for a short period, but those are the only differences.

URL:              

https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRaw

This endpoint URL is different from those we used previously !

Method:          POST

Headers:

Prefer: respond-async
Content-Type: application/json
Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

Body:

{
  "ExtractionRequest": {
    "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.PriceHistoryExtractionRequest",
    "ContentFieldNames": [
      "File Code",
      "RIC",
      "Trade Date",
      "Last Trade Price",
      "Universal Close Price",
      "Alternate Close Price",
      "High Price",
      "Low Price",
      "Open Price",
      "Volume Weighted Average Price",
      "Turnover",
      "Volume",
      "Accumulated Volume Unscaled",
      "Bid Price",
      "Asset Type",
      "Quote ID",
      "Bid Yield",
      "Exchange Code",
      "Currency Code"          
    ],
    "IdentifierList": {
      "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  
      "InstrumentIdentifiers": [
      	{ "Identifier": "ALVG.DE", "IdentifierType": "Ric" },
      	{ "Identifier": "IBM.N", "IdentifierType": "Ric" }
      ]
    },
    "Condition": {
      "AdjustedPrices": true,
      "QueryStartDate": "1996-01-01T00:00:00.000Z",
      "QueryEndDate": "2017-12-31T23:59:59.999Z"
    }
  }
}

 

Get price history data - HTTP response

Note: an embargo can not occur as this is historical data, not intraday data.

On Demand extraction requests are executed as soon as possible. There is no guarantee on the delivery time, it depends on the amount of requested data, and the server load.

In the request we set a preference for an asynchronous response. We will get a response in 30 seconds (default wait time) or less.

The HTTP status of the response can have one of several values, here we shall detail the most likely ones:

  • 202 Accepted - this is the one we are most likely to receive. It means the request was accepted, but processing has not yet completed. The next step is to check the request status by polling it regularly until it returns a 200 OK.
  • 200 OK - this one may happen if the request was very small. It means the request processing has completed. We can skip the step where we check the request status, and proceed directly to the last step, which is to retrieve the data.
  • Other codes: follow this link for a full list with detailed explanations.

It is strongly recommended that you ensure your code handles all possible status codes.

When requests take more than 30 seconds, a 202 Accepted is returned as the first response. This is fairly common, which means that 202 Accepted will be the usual first response.

You can customize the wait time, but this is not recommended.

Let us now look at the two most common responses in detail.

Get price history data - 202 Accepted HTTP response

The request was accepted, but processing has not yet completed. This response is the most likely, especially if the request is for a large amount of data.

Status:                        202 Accepted

Relevant headers:

Location: https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x06df5f2780212ba5')

The location URL must be saved, we will use it in the next step, check request status. Note: the last part of the URL (0x06a4e311c910963c) is the job ID for this request.

Body:                          Response does not contain any data

 

Get price history data - 200 OK HTTP response

Instead of a 202 Accepted, we could receive a 200 OK. This means the request has completed.

Status:                        200 OK

Relevant headers:

Content-Type: application/json; charset=utf-8

Body:

{
  "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
  "JobId": "0x06df5f2780212ba5",
  "Notes": [
    "Extraction Services Version 13.2.41189 (85522a4c8b22), Built Nov 11 2019 22:27:03\r\nUser has overridden estimates broker entitlements.\r\nProcessing started at 19112019 17:43:49.\r\nUser ID: 33314\r\nExtraction ID: 2000000113046389\r\nSchedule: 0x06df5f2780212ba5 (ID = 0x0000000000000000)\r\nInput List (2 items):  (ID = 0x06df5f2780212ba5) Created: 19112019 17:43:49 Last Modified: 19112019 17:43:49\r\nReport Template (19 fields): _OnD_0x06df5f2780212ba5 (ID = 0x06df5f2780712ba5) Created: 19112019 17:43:46 Last Modified: 19112019 17:43:46\r\nSchedule dispatched via message queue (0x06df5f2780212ba5)\r\nSchedule Time: 19112019 17:43:47\r\nTimeseries Date Range: 01011996 to 31122017\r\nProcessing completed successfully at 19112019 17:43:53, taking 4.366 Secs.\r\nExtraction finished at 19112019 16:43:53 UTC, with servers: x01q16, ETS (0.3 secs), QSHC12 (0.1 secs), QSSHA1 (0.0 secs)\r\nUsage Summary for User 33314, Client 11122, Template Type Price History\r\nBase Usage\r\n    Instrument              Instrument           Terms      Price\r\n  Count Type                Subtype            Source     Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n    2 Equities                             N/A      N/A\r\n-------\r\n    2 Total instruments charged.\r\n    0 Instruments with no reported data.\r\n=======\r\n    2 Instruments in the input list.\r\nNo TRPS complex usage to report -- 2 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",
    "Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"
  ]
}

The JobId value must be saved, we will use it in the next step.

The Notes contain information on the request, IDs, timestamps, eventual errors, and extraction quota status. If the request completed successfully, it will also contain the message: Processing completed successfully.

As the request status has been returned directly (because it was a very quick extraction), the next step (check request status) is not required.

We skip it to go directly to retrieve the data, using the returned JobId.

 

Check request status - HTTP request

Skip this step if the previous step returned an HTTP status of 200 OK.

If the previous step returned an HTTP status of 202 Accepted, this step must be executed, and repeated in a polling loop until it returns an HTTP status of 200 OK.

URL:

This is the Location URL, taken from the 202 response header received in the previous step.

https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x06df5f2780212ba5')

Method:          GET

Headers:

This is the same as for the other steps:

Prefer: respond-async
Content-Type: application/json
Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

 

Check request status - HTTP response

If you receive an HTTP status 202 Accepted response (the same as in the previous step), it means the request has not yet completed. You must wait a bit and check the request status again. There is no sense in polling every second, it would just place useless burden on the network and servers. Polling should be done at reasonable intervals, like 30 seconds or more (the larger the request, the longer the recommended polling interval).

If you receive an HTTP status 200 OK response, the request has completed:

Status:                        200 OK

Relevant headers:

Content-Type: application/json; charset=utf-8

Body:

{
  "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#RawExtractionResults/$entity",
  "JobId": "0x06df5f2780212ba5",
  "Notes": [
    "Extraction Services Version 13.2.41189 (85522a4c8b22), Built Nov 11 2019 22:27:03\r\nUser has overridden estimates broker entitlements.\r\nProcessing started at 19112019 17:43:49.\r\nUser ID: 33314\r\nExtraction ID: 2000000113046389\r\nSchedule: 0x06df5f2780212ba5 (ID = 0x0000000000000000)\r\nInput List (2 items):  (ID = 0x06df5f2780212ba5) Created: 19112019 17:43:49 Last Modified: 19112019 17:43:49\r\nReport Template (19 fields): _OnD_0x06df5f2780212ba5 (ID = 0x06df5f2780712ba5) Created: 19112019 17:43:46 Last Modified: 19112019 17:43:46\r\nSchedule dispatched via message queue (0x06df5f2780212ba5)\r\nSchedule Time: 19112019 17:43:47\r\nTimeseries Date Range: 01011996 to 31122017\r\nProcessing completed successfully at 19112019 17:43:53, taking 4.366 Secs.\r\nExtraction finished at 19112019 16:43:53 UTC, with servers: x01q16, ETS (0.3 secs), QSHC12 (0.1 secs), QSSHA1 (0.0 secs)\r\nUsage Summary for User 33314, Client 11122, Template Type Price History\r\nBase Usage\r\n    Instrument              Instrument           Terms      Price\r\n  Count Type                Subtype            Source     Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n    2 Equities                             N/A      N/A\r\n-------\r\n    2 Total instruments charged.\r\n    0 Instruments with no reported data.\r\n=======\r\n    2 Instruments in the input list.\r\nNo TRPS complex usage to report -- 2 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",
    "Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"
  ]
}

The JobId value must be saved, we will use it in the next step.

The Notes contain information on the request, IDs, timestamps, eventual errors, and extraction quota status. If the request completed successfully, it will also contain the message: Processing completed successfully.

We can now retrieve the data, using the returned JobId.

Note: this 200 response is identical in format and content to the 200 OK response that could also have been returned directly when we requested the extraction.

 

Retrieve data - HTTP request

It is mandatory to have received a 200 OK response with a JobID from a previous step before proceeding with this last step.

URL:

Note the JobId value used as parameter in the URL:

https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/RawExtractionResults('0x06df5f2780212ba5')/$value

Method:          GET

Headers:

This is the same as for the other steps, except for the Content-Type. You must include “Accept-Encoding: gzip” for raw stream downloads,  to ensure the server sends the data in Gzipped CSV format.

Prefer: respond-async
Content-Type: Accept-Encoding: gzip
Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

 

Retrieve data - HTTP response

We should get a response of this type:

Status:                        200 OK

Relevant headers:

Content-Encoding: gzip
Content-Type: text/plain

Body:

The content is compressed plain text in CSV format. Depending on the nature of the data, the time range and number of instruments, the response can be quite long and contain thousands of lines.

Here is the beginning of the response content for our request (note that Postman will decompress it automatically):

File Code,RIC,Trade Date,Last Trade Price,Universal Close Price,Alternate Close Price,High Price,Low Price,Open Price,Volume Weighted Average Price,Turnover,Volume,Accumulated Volume Unscaled,Bid Price,Asset Type,Quote ID,Bid Yield,Exchange Code,Currency Code
62,ALVG.DE,1996/01/02,,130.954631,,131.068248,127.75064,127.75064,,,,294759.594399,130.954631,EQTY,0x0003dc004aeb7d03,,GER,EUR
62,ALVG.DE,1996/01/03,,131.931735,,132.613435,130.977354,131.113695,,,,343136.169053,131.931735,EQTY,0x0003dc004aeb7d03,,GER,EUR
62,ALVG.DE,1996/01/04,,131.481813,,132.340755,130.6615,131.749948,,,,160880.236638,131.431821,EQTY,0x0003dc004aeb7d03,,GER,EUR
62,ALVG.DE,1996/01/05,,130.022974,,130.795568,129.341274,130.613781,,,,268883.752143,129.841187,EQTY,0x0003dc004aeb7d03,,GER,EUR
62,ALVG.DE,1996/01/08,,131.409098,,131.795395,129.300372,130.613781,,,,307134.997218,131.113695,EQTY,0x0003dc004aeb7d03,,GER,EUR

Note: returned results will not necessarily include data for all dates you requested, it could be missing on some days, and this could vary depending on the instrument. Possible reasons:

  • Banking holiday.
  • Non volatile instrument that is not quoted on the day of the request.

 

Tutorial Group: 
REST API Tutorials