Thomson Reuters Tick History (TRTH) - REST API

Close Menu
Expand All Collapse All
Introductory Tutorials Tutorials Introduction Video introduction to the TRTH REST API Programming without SDK Tutorial REST API Tutorials REST API Tutorials Introduction REST API Tutorial 1: Connecting to the server REST API Tutorial 2: Retrieving VBD files REST API Tutorial 3: On Demand data extraction workflow REST API Tutorial 4: On Demand tick data extraction REST API Tutorial 5: On Demand market depth extraction REST API Tutorial 6: On Demand intraday bars extraction REST API Tutorial 7: On Demand End of Day extraction REST API Tutorial 8: On Demand raw data extraction REST API Tutorial 9: On Demand corporate actions extraction REST API Tutorial 10: On Demand Histo Reference extraction REST API Tutorial 11: On Demand extraction: instrument list REST API Tutorial 12: GUI control calls: immediate extract REST API Tutorial 13: Historical Search REST API Tutorial 14: Historical Criteria Search REST API Tutorial 15: Historical Chain Constituents Search REST API Tutorial 16: Search by Instrument REST API Tutorial 17: Search for an Equity REST API Tutorial 18: Search for a Future or Option .Net SDK Tutorials .Net SDK Tutorial 1: Connecting to the server .Net SDK Tutorial 2: Retrieving VBD files .Net SDK Tutorial 3: GUI control calls: List, report, sched .Net SDK Tutorial 4: GUI control calls: Validate, extraction .Net SDK Tutorial 5: On Demand extraction, file IO Google BigQuery Tutorials Introduction and environment set-up Data retrieval and tick filters Data analytics Query optimization

REST API Tutorial 12: GUI control calls: immediate extract

Last update February 2020
Environment Any
Language Any HTTP is supported
Compilers None
Prerequisites DSS login, internet access
Source code Below

Tutorial purpose

This tutorial goes through the basics of programmatic GUI control, following the entire workflow for a scheduled data extraction.

Instead of using an On Demand extraction, which is a simplified query, it performs a whole set of actions that could also be done manually in the web GUI. This means that the result of the actions described in this tutorial can also be seen in the web GUI, which gives you a good testing tool.

For an explanation on these two approaches, look here.

In this tutorial a Time and Sales (Tick) data extraction is performed. Other extractions will work in a similar way.

 

Table of contents

 

Basic operations: explanations

Before explaining the code, let us briefly explain what we are trying to achieve.

To retrieve data, 3 basic elements are required:

  • An instrument list. This is the list of financial instrument identifiers for which data will be requested. Using the API, an instrument list can be created, and instruments added to it.
  • report template. It defines the format of the requested data, and the list of fields. Depending on the requested data (pricing data, reference data, news, estimates, corporate actions, etc.), different report options and data fields are available. Using the API, a report template can be created, and its data fields selected.
  • An extraction schedule. It defines when the data will be retrieved, and triggers the actual data extraction. Its frequency could be once-off or recurring. Its timing could be immediate, at a fixed time, or triggered when data is available.

They are all stored on the DSS server, and you can have several of each.

They can be created, displayed, modified and deleted manually, using the web GUI, and that might be the best solution if they are invariable in time and only need to be created once. But regular changes to some or all of these 3 elements might be required, and the purpose of the API is to give us the tools to automate that. The most frequent use case is the need to change the instrument list, which could be on a daily basis.

The purpose of this tutorial is to demonstrate how to do these things programmatically, using the TRTH REST API.

If your use case only requires regular changes to an instrument list, you could decide to manually create an empty instrument list, a report template and an extraction schedule, and then create a program that automatically changes the contents of the instrument list. You could also do it all programmatically, which is easier.

 

Retrieve the available field list from the server

Get available field list - HTTP request

This step is exactly the same as what we did at the start of Tutorial 4, but for the sake of completeness we repeat it here. All the other steps are new.

This step is optional. If you do not know what content field names are available, you can request a list of those available

URL:

The available field set depends on the type of data you want to request. The data type must therefore be specified in the request, which is done by setting a report template type in the URL. As we want tick data we set a value of TickHistoryTimeAndSales.

https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/GetValidContentFieldTypes(ReportTemplateType=ThomsonReuters.Dss.Api.Extractions.ReportTemplates.ReportTemplateTypes'TickHistoryTimeAndSales')

Method:          GET

Headers:

Note: for all requests we need a user token. The token was retrieved in Tutorial 1.

Prefer: respond-async
Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

 

Get available field list - HTTP response

If the token is valid, this is the response we get:

Status:                        200 OK

Relevant headers:

Content-Type: application/json; charset=utf-8

Body:

There are more than 300 values in the response. Here is the beginning of the response:

{
  "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ContentFieldTypes",
  "value": [
    {
      "Code": "THT.Auction - Exchange Time",
      "Name": "Auction - Exchange Time",
      "Description": "Exchange supplied exchange time (Local or GMT depending on the exchange).",
      "FormatType": "Text",
      "FieldGroup": "Auction"
    },
    {
      "Code": "THT.Auction - Price",
      "Name": "Auction - Price",
      "Description": "Auction Price.",
      "FormatType": "Number",
      "FieldGroup": "Auction"
    },
    {
      "Code": "THT.Auction - Qualifiers",
      "Name": "Auction - Qualifiers",
      "Description": "Trade qualifiers or market condition indicator; See Qualifiers for more details.",
      "FormatType": "Text",
      "FieldGroup": "Auction"
    },
    {
      "Code": "THT.Auction - Sequence Number",
      "Name": "Auction - Sequence Number",
      "Description": "An exchange derived sequence number associated with the auction (applicable to US markets).",
      "FormatType": "Number",
      "FieldGroup": "Auction"
    },
    {
      "Code": "THT.Auction - Volume",
      "Name": "Auction - Volume",
      "Description": "Auction Volume.",
      "FormatType": "Number",
      "FieldGroup": "Auction"
    },

This goes on with all the other available fields. Here is the last part:

    {
      "Code": "THT.Trade - Volume",
      "Name": "Trade - Volume",
      "Description": "Last traded volume",
      "FormatType": "Number",
      "FieldGroup": "Trade"
    },
    {
      "Code": "THT.Trade - Weak Market",
      "Name": "Trade - Weak Market",
      "Description": "Market weakness",
      "FormatType": "Number",
      "FieldGroup": "Trade"
    },
    {
      "Code": "THT.Trade - Yield",
      "Name": "Trade - Yield",
      "Description": "An update to indicate Dividend Yield as adjusted by the last trade or closing price.",
      "FormatType": "Number",
      "FieldGroup": "Trade"
    }
  ]
}

The result contains the field code, name, a description, field type (number, text, date) and group.  Use this to choose the field names you want. In the next step we will make a request for data, using some data fields we chose.

 

Create an instrument list

Create an instrument list - HTTP request

In this step we create an empty instrument list. We will populate it in the next step, by appending instrument identifiers to the list.

    URL:               

    https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/InstrumentLists

    Method:          POST

    Headers:

    Prefer: respond-async
    Content-Type: application/json
    Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

    Body:

    The body of the request must mention:

    • The data type, which is an InstrumentList.
    • The name of the instrument list. This must be unique, an error will be generated if the name already exists.
    {
      "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.SubjectLists.InstrumentList",
      "Name": "myInstrumentListName"
    }

     

    Create an instrument list - HTTP response

    If the token is valid, and an instrument list with the same name does not yet exist, this is the response we get:

    Status:                        201 Created

    Relevant headers:

    Content-Type: application/json; charset=utf-8
    Location: https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/InstrumentLists('')

    Body:

    {
      "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#InstrumentLists/$entity",
      "ListId": "0x05a2b85bfb4b3036",
      "Name": "myInstrumentListName",
      "Count": 0,
      "Created": "2017-03-10T09:00:42.292Z",
      "Modified": "2017-03-10T09:00:42.292Z"
    }

    Your application must cache the value of ListId returned in the body content, it will be required in the next steps.

    In case of error, see the Error handling section below.

     

    Create an instrument list - check in the DSS web GUI

    Let us display the result of this operation in the web GUI: click on DATASCOPE SELECT top left, then on Instrument Lists:

    At this stage, the instrument list is created, but when we click on it we see it is empty:

    If there are many and you can’t find it, sort the entries by clicking on the Creation Date column header.

     

    Append instruments to an instrument list

    Append instruments to an instrument list - HTTP request

    Intrument identifiers are added to the instrument list by appending them. This can even be done several times.

    The request path refers to the instrument list Id (0x05a2b85bfb4b3036) which was returned when we created the instrument list in the previous step.

      URL:               

      https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/InstrumentLists('0x05a2b85bfb4b3036')/ThomsonReuters.Dss.Api.Extractions.InstrumentListAppendIdentifiers

      Method:          POST

      Headers:

      Prefer: respond-async
      Content-Type: application/json
      Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

      Body:

      The body of the request must mention:

      • The list of instrument identifiers, each one with its type (Cusip, Isin, Ric, etc., more than 30 are available). The user defined identifier is optional; it is just an additional tag that will be stored in the instrument list on the server. It could contain a customer internal identifier, asset class, industry sector, country, currency, etc. Instrument identifiers can be filtered or sorted by this tag in the web GUI.
      • In the example below we define one instrument using a CUSIP code (Committee on Uniform Securities Identification Procedures, an identification system for north American and Canadian stocks), and another one using a RIC (Reuters Instrument Code, a Thomson Reuters proprietary ID for all financial instruments). More information on them can be found by browsing in the API Reference Tree or searching in the help in the web GUI:



        We also include, for training purposes, an invalid instrument, to see the effects of instrument validation.
      • What to do if there is a duplicate instrument identifier.
      {
        "Identifiers": [
          {
            "Identifier": "CARR.PA",
            "IdentifierType": "Ric",
            "UserDefinedIdentifier": "EQUITYTEST"
          },
          {
            "Identifier": "438516AC0",
            "IdentifierType": "Cusip",
            "UserDefinedIdentifier": "BONDTEST"
          },
          {
            "Identifier": "JUNK.JUNK",
            "IdentifierType": "Ric",
            "UserDefinedIdentifier": "INVALID"
          }
        ],
        "KeepDuplicates": false
      }

       

      Append instruments to an instrument list - HTTP response

      If the token is valid, this is the response we get:

      Status:                        200 OK

      Relevant headers:

      Content-Type: application/json; charset=utf-8

      Body:

      {
        "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ThomsonReuters.Dss.Api.Extractions.SubjectLists.InstrumentsAppendIdentifiersResult",
        "ValidationResult": {
          "ValidInstrumentCount": 2,
          "OpenAccessSegments": [],
          "StandardSegments": [
            {
              "Code": "E",
              "Description": "Equity",
              "Count": 1
            },
            {
              "Code": "G",
              "Description": "GORP",
              "Count": 1
            }
          ],
          "ValidationDuplicates": [],
          "Messages": [
            {
              "Severity": "Info",
              "Message": "RIC, JUNK.JUNK (not found)"
            }
          ]
        },
        "AppendResult": {
          "AppendedInstrumentCount": 2,
          "AppendDuplicates": []
        }
      }

      The body contains several sections:

      • Instrument identifiers validation results, including the number of valid instruments, and some high level information on the segmentation by asset class of the instrument identifiers.
      • If there are issues, a messages section will list all invalid instruments.
      • Instrument identifiers append results, including the number of appended instruments, and details on eventual duplicates,

       

      Append instruments to an instrument list - check in the DSS web GUI

      The result of this operation can be seen in the web GUI. Now we find our 2 instruments in the instrument list:

       

      Create a report template

      Create a report template - HTTP request

      The report template type must be set in the path of the request. In this case it is a Time and Sales (tick) report template.

        URL:               

        https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/TickHistoryTimeAndSalesReportTemplates

        Method:          POST

        Headers:

        Prefer: respond-async
        Content-Type: application/json
        Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

        Body:

        The body of the request must mention several parameters. The main ones are:

        • The data type, which in this case is an TickHistoryTimeAndSalesReportTemplate.
        • Some parameters to customise the output file headers and trailers.
        • The name of the report template. This must be unique, an error will be generated if the name already exists.
        • The list of data content fields we want in the result. These were chosen among all the available ones, see Retrieve the available field list from the server above.
        • For more information on available report templates and corresponding fields, see here.
        {
          "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ReportTemplates.TickHistoryTimeAndSalesReportTemplate",
          "ShowColumnHeaders": false,
          "Name": "myTandSTemplateName",
          "Headers": [],
          "Trailers": [],
          "ContentFields": [
            { "FieldName": "Quote - Ask Price" },
            { "FieldName": "Quote - Ask Size" },
            { "FieldName": "Quote - Bid Price" },
            { "FieldName": "Quote - Bid Size" },
            { "FieldName": "Quote - Exchange Time" }
          ],
            "Condition": {
              "MessageTimeStampIn": "GmtUtc",
              "ApplyCorrectionsAndCancellations": false,
              "ReportDateRangeType": "Range",
              "QueryStartDate": "2016-09-29T15:15:00.000Z",
              "QueryEndDate": "2016-09-29T15:45:00.000Z"
            }
        }

         

        Create a report template - HTTP response

        If the token is valid, and a report template with the same name does not yet exist, this is the response we get:

        Status:                        201 Created

        Relevant headers:

        Content-Type: application/json; charset=utf-8
        Location: https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/TickHistoryTimeAndSalesReportTemplates('')

        Body:

        {
          "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#TickHistoryTimeAndSalesReportTemplates/$entity",
          "ReportTemplateId": "0x05a2b932465b3036",
          "ShowColumnHeaders": true,
          "CompressionType": "GZip",
          "CreateDate": "2017-03-10T09:15:20.037Z",
          "LastChangedDate": "2017-03-10T09:15:20.037Z",
          "Name": "myTandSTemplateName",
          "OutputFormat": "CommaSeparatedValues",
          "ReportFieldCount": 5,
          "Delimiter": "None",
          "DeliveryType": "None",
          "TemplateTypeCode": "THT",
          "Headers": [],
          "Trailers": [],
          "ContentFields": [
            {
              "FieldName": "Quote - Ask Price",
              "Justification": "Center",
              "WidthStyle": "VariableWidth",
              "Format": {
                "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ReportTemplates.ContentFieldNumberFormat",
                "DecimalPlaces": 9,
                "DecimalSeparator": "Period",
                "IntegerPlaces": 18,
                "UseLeadingZero": false,
                "NegativeSignPosition": "Before",
                "ThousandSeparator": "Comma",
                "UseThousandSeparator": true,
                "UseTrailingZero": false
              }
            },
            {
              "FieldName": "Quote - Ask Size",
              "Justification": "Center",
              "WidthStyle": "VariableWidth",
              "Format": {
                "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ReportTemplates.ContentFieldNumberFormat",
                "DecimalPlaces": 9,
                "DecimalSeparator": "Period",
                "IntegerPlaces": 18,
                "UseLeadingZero": false,
                "NegativeSignPosition": "Before",
                "ThousandSeparator": "Comma",
                "UseThousandSeparator": true,
                "UseTrailingZero": false
              }
            },
            {
              "FieldName": "Quote - Bid Price",
              "Justification": "Center",
              "WidthStyle": "VariableWidth",
              "Format": {
                "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ReportTemplates.ContentFieldNumberFormat",
                "DecimalPlaces": 9,
                "DecimalSeparator": "Period",
                "IntegerPlaces": 18,
                "UseLeadingZero": false,
                "NegativeSignPosition": "Before",
                "ThousandSeparator": "Comma",
                "UseThousandSeparator": true,
                "UseTrailingZero": false
              }
            },
            {
              "FieldName": "Quote - Bid Size",
              "Justification": "Center",
              "WidthStyle": "VariableWidth",
              "Format": {
                "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ReportTemplates.ContentFieldNumberFormat",
                "DecimalPlaces": 9,
                "DecimalSeparator": "Period",
                "IntegerPlaces": 18,
                "UseLeadingZero": false,
                "NegativeSignPosition": "Before",
                "ThousandSeparator": "Comma",
                "UseThousandSeparator": true,
                "UseTrailingZero": false
              }
            },
            {
              "FieldName": "Quote - Exchange Time",
              "Justification": "Center",
              "WidthStyle": "VariableWidth",
              "Format": {
                "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ReportTemplates.ContentFieldTextFormat",
                "Capitalization": "None"
              }
            }
          ],
          "Condition": {
            "SortBy": "SingleByRic",
            "MessageTimeStampIn": "GmtUtc",
            "ApplyCorrectionsAndCancellations": false,
            "ReportDateRangeType": "Range",
            "QueryStartDate": "2016-09-29T15:15:00.000Z",
            "QueryEndDate": "2016-09-29T15:45:00.000Z",
            "Preview": "None",
            "ExtractBy": "Ric",
            "DisplaySourceRIC": false
          }
        }

        Your application must cache the value of ReportTemplateId returned in the body content, it will be required in the next steps.

        In case of error, see the Error handling section below.

         

        Create a report template - check in the web GUI

        Let us display the result of this operation in the web GUI: click on DATASCOPE SELECT top left, then on Report Templates:

        We find our report template with 5 fields:

        If there are many and you can’t find it, sort the entries by clicking on the Creation Date column header.

         

        Schedule an immediate extraction

        Schedule an immediate extraction - HTTP request

        An extraction schedule defines when the data will be retrieved, and triggers the actual extraction. It must refer to an instrument list (for which the extraction will be done), and a report template (defining what data needs to be extracted).

        An extraction schedule can run once or recurring. Its timing can be immediate, at a fixed time, or triggered when data is available.

        It is possible to define several schedules.

        In this tutorial we define an extraction schedule for our previously created instrument list and report template, that will be run immediately, only once.

          URL:               

          https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/Schedules

          Method:          POST

          Headers:

          Prefer: respond-async
          Content-Type: application/json
          Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

          Body:

          The body of the request must mention several parameters. The main ones are:

          • The name of the extraction schedule. This must be unique, an error will be generated if the name already exists.
          • The recurrence. In this example we define it to be single and immediate.
          • The trigger. In this example we define an immediate trigger.
          • The instrument list id. Its value is that which was returned when we created the instrument list.
          • The report template id. Its value is that which was returned when we created the report template.
          {
            "Name": "myImmediateSchedule",
            "Recurrence": {
              "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.Schedules.SingleRecurrence",
              "IsImmediate": true
            },
            "Trigger": {
              "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.Schedules.ImmediateTrigger"
            },
            "ListId": "0x05a2b85bfb4b3036",
            "ReportTemplateId": "0x05a2b932465b3036"
          }

           

          Schedule an immediate extraction - HTTP response

          If the token is valid, and a schedule with the same name does not yet exist, this is the response we get:

          Status:                        201 Created

          Relevant headers:

          Content-Type: application/json; charset=utf-8
          Location: https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/Schedules('')

          Body:

          {
            "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#Schedules/$entity",
            "ScheduleId": "0x05a2b98d233b3036",
            "Name": "myImmediateSchedule",
            "TimeZone": "W. Europe Standard Time",
            "Recurrence": {
              "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.Schedules.SingleRecurrence",
              "ExtractionDateTime": "0001-01-01T00:00:00Z",
              "IsImmediate": true
            },
            "Trigger": {
              "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.Schedules.ImmediateTrigger",
              "LimitReportToTodaysData": false
            },
            "UserId": 33314,
            "CreateDate": "2017-03-10T09:21:32.227Z",
            "LastChangeDate": "2017-03-10T09:21:32.227Z",
            "ListId": "0x05a2b85bfb4b3036",
            "ReportTemplateId": "0x05a2b932465b3036"
          }

          Most of the returned content is a recap of the request, but with added details. As the extraction is scheduled to be immediate, the ExtractionDateTime and TimeZone will not be used. 

          Your application must cache the value of ScheduleId returned in the body content, it will be required later.

          In case of error, see the Error handling section below.

           

          Schedule an immediate extraction - check in the DSS web GUI

          Let us display the result of this operation in the web GUI: click on DATASCOPE SELECT top left, then on Schedules:

          We find our schedule, as defined. Immediately after sending the request, its status shows it has been Submitted and is Processing:

          Once the scheduled data extraction has completed, its last status changes to Completed:

          If there are many and you can’t find it, sort the entries by clicking on the Creation Date column header.

           

          Check the extraction status

          Extraction status life cycle

          The extraction will first be queued, its status will be pending. Once triggered, it will be processed. Once all the data is extracted, it will be complete.

          The following chart illustrates the extraction events as they progress in time from left to right, with the related values of the extraction Status and DetailedStatus:

          In the previous step we showed how to check the schedule and extraction status in the web GUI, which is fine for testing, but it is also possible to do this programmatically.

           

          Check the extraction status - HTTP request

          The schedule id we received in the response to our schedule creation request is used as a parameter set in the path of the request.

          URL:               

          https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/Schedules('0x05a2b98d233b3036')/LastExtraction

          Method:          GET

          Headers:

          Prefer: respond-async
          Content-Type: application/json
          Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

           

          Check the extraction status - HTTP response

          If the schedule has not yet completed, an HTTP status 204 is returned (the body of such a response is empty). We must wait, then check the extraction status again, until receiving an HTTP status 200:

          Status:                        200 OK

          Relevant headers:

          Content-Type: application/json; charset=utf-8

          Body:

          {
            "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ReportExtractions/$entity",
            "ReportExtractionId": "2000000000197524",
            "ScheduleId": "0x05a2b98d233b3036",
            "Status": "Completed",
            "DetailedStatus": "Done",
            "ExtractionDateUtc": "2017-03-10T09:21:32.570Z",
            "ScheduleName": "myImmediateSchedule",
            "IsTriggered": false
          }

          In the response we find the Status and DetailedStatus. Here we see the status is Completed.

          Your application must cache the value of ReportExtractionId returned in the body content, it will be required later.

           

          Retrieve the extraction report

          The extraction report contains the list of files generated on the server.

          Retrieve the extraction report - HTTP request

          The report extraction id we received in the response to our extraction status request is used as a parameter set in the path of the request.

          URL:               

          https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ReportExtractions('2000000000197524')/Files

          Method:          GET

          Headers:

          Prefer: respond-async
          Content-Type: application/json
          Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

           

          Retrieve the extraction report - HTTP response

          If the token is valid, this is the response we get:

          Status:                        200 OK

          Relevant headers:

          Content-Type: application/json; charset=utf-8

          Body:

          {
            "@odata.context": "https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ExtractedFiles",
            "value": [
              {
                "ExtractedFileId": "VjF8MHgwNWEyYjk5ODRiNmIyZjk2fA",
                "ReportExtractionId": "2000000000197524",
                "ScheduleId": "0x05a2b98d233b3036",
                "FileType": "Full",
                "ExtractedFileName": "33314.myImmediateSchedule.20170310.102132.2000000000197524.tm08n01.csv.gz",
                "LastWriteTimeUtc": "2017-03-10T09:22:18.000Z",
                "ContentsExists": true,
                "Size": 59134
              },
              {
                "ExtractedFileId": "VjF8MHgwNWEyYjk5NjZmY2IyZjc2fA",
                "ReportExtractionId": "2000000000197524",
                "ScheduleId": "0x05a2b98d233b3036",
                "FileType": "Note",
                "ExtractedFileName": "33314.myImmediateSchedule.20170310.102132.2000000000197524.tm08n01.csv.gz.notes.txt",
                "LastWriteTimeUtc": "2017-03-10T09:22:19.206Z",
                "ContentsExists": true,
                "Size": 1262,
                "ReceivedDateUtc": "2017-03-10T09:22:19.206Z"
              }
            ]
          }

          Note. in the result set, the order of the files can vary.

          We see 2 files have been generated on the DSS server:

          1. The Full extracted data itself.
          2. The extraction Note file, which contains details of the extraction.

          Each file has an ExtractedFileId. Your application must cache the values of these file ids, to be able to access the files contents.

           

          Retrieve the data from the server

          Once the status is Completed, and we have retrieved the ExtractedFileId, we can retrieve the data.

          Retrieve the data from the server - HTTP request

          The extracted file id we received for the Full data file in the response to our extraction report request is used as a parameter set in the path of the request.

          URL:               

          https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractedFiles('VjF8MHgwNWEyYjk5ODRiNmIyZjk2fA')/$value

          Method:          GET

          Headers:

          Prefer: respond-async
          Content-Type: application/json
          Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

           

          Retrieve the data from the server - HTTP response

          If the token is valid, this is the response we get:

          Status:                        200 OK

          Relevant headers:

          Content-Type: text/plain

          Note: contrary to most responses, this time the content is not in JSON format !

          Body:

          Here is the beginning of the data (there is much more):

          #RIC,Domain,Date-Time,GMT Offset,Type,Bid Price,Bid Size,Ask Price,Ask Size,Exch Time
          CARR.PA,Market Price,2016-09-29T13:15:00.097355899Z,+2,Quote,23.09,3085,23.095,100,13:15:00.048000000
          CARR.PA,Market Price,2016-09-29T13:15:00.097355899Z,+2,Quote,23.09,3120,23.095,100,13:15:00.049000000
          CARR.PA,Market Price,2016-09-29T13:15:01.033433922Z,+2,Quote,23.09,3370,23.095,100,13:15:00.977000000
          CARR.PA,Market Price,2016-09-29T13:15:01.033433922Z,+2,Quote,23.09,3370,23.1,713,13:15:00.977000000
          CARR.PA,Market Price,2016-09-29T13:15:01.033433922Z,+2,Quote,23.095,1700,23.1,713,13:15:00.977000000
          CARR.PA,Market Price,2016-09-29T13:15:01.033433922Z,+2,Quote,23.095,1700,23.1,300,13:15:00.978000000
          CARR.PA,Market Price,2016-09-29T13:15:01.033433922Z,+2,Quote,23.095,1900,23.1,300,13:15:00.978000000
          CARR.PA,Market Price,2016-09-29T13:15:01.033433922Z,+2,Quote,23.095,2170,23.1,300,13:15:00.978000000
          CARR.PA,Market Price,2016-09-29T13:15:01.037513480Z,+2,Quote,23.095,2346,23.1,300,13:15:00.979000000
          ...

          The content of the file is the data, in CSV format. After the default fields (RIC, data type, time stamp, record type, we find the 5 requested fields (bid and ask price and size, and the exchange time). Depending on the instruments, chosen fields and time of request, it can happen that no data is available for some fields.

           

          Retrieve the extraction notes from the server - HTTP request

          The example above illustrated how to retrieve the contents of the data file. Retrieving the contents of the other files can be done using their respective ExtractedFileId values. The extracted file id we received for the Note file in the response to our extraction report request is used as a parameter set in the path of the request.

          URL:               

          https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ExtractedFiles('VjF8MHgwNWEyYjk5NjZmY2IyZjc2fA')/$value

          Method:          GET

          Headers:

          Prefer: respond-async
          Content-Type: application/json
          Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

           

          Retrieve the extraction notes from the DSS server - HTTP response

          If the token is valid, this is the response we get:

          Status:                        200 OK

          Relevant headers:

          Content-Type: text/plain

          Body:

          Extraction Services Version 10.8.36127 (95fc12628942), Built Jan 13 2017 03:21:22
          User ID: 33314
          Extraction ID: 2000000000197524
          Schedule: myImmediateSchedule (ID = 0x05a2b98d233b3036)
          Input List (2 items): myInstrumentListName (ID = 0x05a2b85bfb4b3036) Created: 10032017 10:00:42 AM Last Modified: 10032017 10:08:20 AM
          Report Template (5 fields): myTandSTemplateName (ID = 0x05a2b932465b3036) Created: 10032017 10:15:20 AM Last Modified: 10032017 10:15:20 AM
          Schedule dispatched via message queue (0x05a2b98d3a9b3036), Data source identifier (B6835AEAF898422C96325D7AC3A62989)
          Schedule Time: 10032017 10:21:32 AM
          Processing started at 10032017 10:21:40 AM
          Processing completed successfully at 10032017 10:22:18 AM
          Extraction finished at 10032017 09:22:18 AM UTC, with servers: tm08n01, TRTH (32.042 secs)
          Instrument <RIC,CARR.PA> expanded to 1 RIC: CARR.PA.
          Instrument <CSP,438516AC0> expanded to 2 RIC instances: TESTFINN=TEST to US438516AC05=JPNY.
          Total instruments after instrument expansion = 3
          Quota Message: INFO: Tick History Cash Quota Count Before Extraction: 995; Instruments Extracted: 1; Tick History Cash Quota Count After Extraction: 995, 99.5% of Limit; Tick History Cash Quota Limit: 1000
          Manifest: #RIC,Domain,Start,End,Status,Count
          Manifest: CARR.PA,Market Price,2016-09-29T13:15:00.097355899Z,2016-09-29T13:44:59.549562770Z,Active,12791
          Manifest: TESTFINN=TEST,Market Price,,,Inactive,0
          Manifest: US438516AC05=JPNY,Market Price,,,Inactive,0

          The file contents give us important details about the extraction. It is recommended to save the extraction notes, and also to parse them to detect issues, warnings or errors.

           

          Clean up

          In the initial steps of this tutorial, we created 3 items on the DSS server: an instrument list, a report template and a schedule.

          Whenever such an item is not required any more, it can be deleted, programmatically.

          Deleting an item requires a DELETE method, and using the id of the item to be deleted inside the appropriate path. This is why, in the item creation steps above, we systematically saved the ids received them from the DSS server. If you did not save them, then you will have to delete them manually, using the web GUI.

          Deleting unused items is a good practice, to avoid cluttering the DSS server with lots of old useless items, and is also a pre-requisite if you want to re-use a name for an instrument list, report template or schedule.

          The choice of deleting an item or not depends on its future utility. Obviously, a recurring schedule that is expected to run on a regular basis will not be deleted !

          In this tutorial we shall delete all the items we created. Results can be checked by using the web GUI.

          Delete the schedule from the server - HTTP request

          Note: the schedule id is a parameter in the path.

          URL:               

          https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/Schedules('0x05a2b98d233b3036')

          Method:          DELETE

          Headers:

          Prefer: respond-async
          Content-Type: application/json
          Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

           

          Delete the schedule from the server - HTTP response

          If the token and schedule id are valid, this is the response we get:

          Status:                        204 No Content

          Body:                          Response does not contain any data.

           

          Delete the report template from the server - HTTP request

          URL:               

          https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/ReportTemplates('0x05a2b932465b3036')

          Method:          DELETE

          Headers:

          Prefer: respond-async
          Content-Type: application/json
          Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

           

          Delete the report template from the server - HTTP response

          If the token and report template id are valid, this is the response we get:

          Status:                        204 No Content

          Body:                          Response does not contain any data.

           

          Delete the instrument list from the server - HTTP request

          URL:               

          https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/InstrumentLists('0x05a2b85bfb4b3036')

          Method:          DELETE

          Headers:

          Prefer: respond-async
          Content-Type: application/json
          Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

           

          Delete the instrument list from the server - HTTP response

          If the token and instrument list id are valid, this is the response we get:

          Status:                        204 No Content

          Body:                          Response does not contain any data.

           

          Error handling

          We handled a few cases above, like instrument validation when appending instruments to a list, or checking the extraction notes to understand why some data is missing.

          A very common error, especially when learning, is attempting to run a piece of code several times, and running into item creation errors due to existing ones with the same name. The following describes the symptoms and remedies.

          Create an instrument list (or other item) error - HTTP response if duplicate

          If an instrument list with the same name already exists, an error is returned:

          Status:                        400 Bad Request

          Relevant headers:

          Content-Type: application/json; charset=utf-8
          X-Validation-Messages: [{"Id":"DuplicateInstrumentListName","ItemType":"Instrument List","ItemId":"0x05a2ba76016b3036","PropertyName":"Name","Severity":4,"Message":"An instrument list with the selected name myInstrumentListName already exists","DiagnosticMessage":null}]

          The message is quite clear as to the cause of the error.

          Body:

          {
            "error": {
              "message": "Validation Error:\r\n\r\nAn instrument list with the selected name myInstrumentListName already exists"
            }
          }

          Here too the message is quite clear as to the cause of the error.

           

          Other similar errors

          Similar errors will arise for a report template or extraction schedule that has the same name as an existing one.

           

          Error causes and handling

          Possible causes:

          The most probable cause is that you ran a piece of code that creates an item (instrument list, report template or schedule) twice. As one cannot create 2 items with the same name, an error is generated.

          It could also be possible that, by pure chance, you have an instrument list, report template and / or schedule name that has exactly the same name as one of those in this tutorial.

          To solve this:

          If you ran a piece of code twice, you might have forgotten to save the item's id. In that case you need to delete the item, and start again:

          1. Manually connect to the DSS web GUI, and, if they exist, manually delete any of the following items created (and not deleted) by this tutorial:
            • Instrument list myInstrumentListName
            • Report template myTandSTemplateName
            • Schedule myImmediateSchedule
          2. Run your code again.

          If you have an instrument list, report template and / or schedule name that has exactly the same name as one of those in this tutorial: 

          1. Change the offending name in the code of this tutorial.
          2. Run the code again.

           

          Tutorial Group: 
          REST API Tutorials