This article demonstrates access to EDP Research in Node.JS application. The application subscribes to Alert API to receive Alert when new Research is available to access, and then retrieve documents or text extracted from Research documents. The concept used by this application can also be applied to the EDP News subscription as it uses the same Alert delivery mechanism.

EDP Research Overview

EDP Research provides the buy-side with their entitled sell side research reports on a real-time basis. This system will use notification-retrieval type method for the buy-side firms to access entitled broker research reports on a real-time basis and in a programmatic fashion.

 

Alert delivery mechanism

The data from an EDP service can be delivered in variety of mechanisms, depending on content set. Alerts delivery is one of major content delivery types to receive asynchronous updates (alerts) to subscription criteria.

Below is the illustration for News Alert. User application starts by expressing an interest in alerts to source. Source then creates a FIFO queue in cloud and start pushing messages. User application polls the queue for available message, retrieve and remove message from the queue. Messages need to be decrypted to get content payload.

For more information about Alert delivery mechanism, please see this Alerts delivery mechanism in EDP.

Application diagram

Web application in this article uses ERTRESTController JavaScript to connect to services in Elektron Data Platform and Amazon SQS. The ERTRESTController is developed based on this Elektron RealTime (ERT) Javascript components. For EDP service connection, the ERTRESTController send request to EDP using Node JS server as a proxy. This is to avoid Cross-Origin Resource Sharing (CORS) issue, when using frontend JavaScript code to access responses for cross-origin requests. For Amazon SQS, the Web application uses AWS JavaScript SDK for AWS SQS access.

Application implementation workflow

The application is implemented based on general workflow

  1. Login to the Elektron Data Platform and get access token
  2. Delete existing Alert subscription
  3. Research Alerts subscription
  4. Get the credentials to a cloud queue
  5. Polling and retrieving messages
  6. Decrypting the message
  7. Retrieving Research Document

 

1) Login to the Elektron Data Platform and get access token

This step uses the existing implementation of the ERTController JavaScript component. The application calls get_access_token() method to login to Elektron Data Platform and get access token used by any requests to EDP services. The input parameters are EDP username, password and client ID. The ERTController also handles the process to refresh new token from Elektron Data Platform.

2) Delete existing Alert subscription

Currently, only one subscription is allowed at a time. The application will receive status: 422 - Number of user subscription exceed the limit, when number of user subscription is more than one.  To avoid this issue, the application will delete all subscriptions first with unsubscribeToResearch() method.

ERTRESTController.prototype.unsubscribeToResearch = function ()
{
    let request_data = {
        method: 'DELETE',
        uri: research_subscription_url,
        headers: {
            'Accept': 'application/json',
            'Authorization': 'Bearer ' + this.auth_obj.access_token
        },
        json: true,
        resolveWithFullResponse: true
    };

3) Research Alerts subscription

Next, the application sends a request for Research Alerts subscription by calling subscribeToResearch() method. The method creates Research Subscription request passing user’s UUID.

ERTRESTController.prototype.subscribeToResearch = function (uuid) 
{
    let request_data = {
        method: 'POST',
        uri: research_subscription_url,
        headers: {
            'Accept': 'application/json',
            'Authorization': 'Bearer ' + this.auth_obj.access_token
        },
        body: {
            "transport": {
                "transportType": "AWS-SQS"
            },
            "userID": uuid
        },
        json: true,
        resolveWithFullResponse: true
    };

 

4) Get the credentials to a cloud queue

Next, the application get Cloud Credential for a cloud queue in getCloudCredential() method. The application passes the endpoint to queue received from the Research Alert subscription on step 3.

ERTRESTController.prototype.getCloudCredential = function (endpoint) 
{
    let request_data = {
        method: 'GET',
        uri: cloud_credential_url,
        headers: {
            'Accept': 'application/json',
            'Authorization': 'Bearer ' + this.auth_obj.access_token
        },
        qs: {
            endpoint: endpoint
        },
        json: true,
        resolveWithFullResponse: true
    };

 

5) Polling and retrieving messages

At this step, the application has credential information to access the queue. It uses AWS SDK for JavaScript to access and poll message from the Amazon SQS queue. The function requires credential; AccessKeyID, SecretAccessKey and SessionToken received from the previous step. 

function pullingFromSQS() {
    // Configure AWS SDK with cloud credential
    AWS.config.update({
        "accessKeyId": cloudCredential.accessKeyId, "secretAccessKey": cloudCredential.secretKey, "region": "us-east-1", "sessionToken": cloudCredential.sessionToken
    });

    // Create an SQS service object
    var sqs = new AWS.SQS({ apiVersion: '2012-11-05' });

    // Configure queue's URL
    var queueURL = cloudCredential.queueURL;

    var params = {
        AttributeNames: [
            "SentTimestamp"
        ],
        MaxNumberOfMessages: 1,
        MessageAttributeNames: [
            "All"
        ],
        QueueUrl: queueURL,
        VisibilityTimeout: 20,
        WaitTimeSeconds: 3
    };

    console.log("Receiving message...");
    // Pull message from SQS queue
    sqs.receiveMessage(params, function (err, data) {

 

6) Decrypting the message

EDP Research/News Alert service encrypts messages using AES256 with GCM, and then base64 encoded before being pushed into the queue. Once the application receives messages, it needs to decrypt it using the cryptography Key received from the step 2).

According to the Alerts delivery mechanism in EDP article, the structure of encrypted message after base64 decoding is below.

With the following length:

  • AES Key Length = 256 bytes
  • GCM AAD Length = 16 bytes
  • GCM TAG Length = 16 bytes
  • GCM NONCE Length = 12 bytes

This application uses Web Cryptography API library to decrypt the message. Please note that the usage of Encrypted Message and Tag can depend on Library used for decryption. For example, this Web Cryptography API library expects application to pass Tag as a part of Encrypted Message.

The decrypted message contains metadata of the alert message in JSON format. The Research document information is in “payload” object. For more information about field in Research document, please see the Research API Data Point Description File .

Below is sample of data.


{  
   "envelopeVersion":"1.0",
   "ecpMessageID":"9ce18554-6e8d-34f5-b9af-e6d389643155",
   "sourceSeqNo":15580870905248656,
   "contentServicePermID":9000001,
   "distributionSeqNo":3,
   "sourceTimestamp":"2019-05-17T09:58:22.350Z",
   "receiveTimestamp":"2019-05-17T09:58:22.675Z",
   "distributionTimestamp":"2019-05-17T09:58:22.844Z",
   "payloadVersion":"1.4",
   "subscriptionID":"2a1c2e7f-e917-466a-9502-08e25bd7aa3a",
   "payload":{  
      "rptStylesResp":[  
         {  
            "uid":150000002,
            "value":"RPT_CO"
         }
      ],
      "fileName":"85235540.pdf",
      "authorRating":2,
      "companyName":"BOC International (China) Co., Ltd.",
      "fileExt":"pdf",
      "pages":4,
      "reg":[  
         ...
      ],
      "isLocked":false,
      "subj":[  
         ...
      ],
      "enhancedPDF":false,
      "secondaryFileType":"",
      "contrPermID":112604684128,
      "langDesc":{  
         "rfc1766":"zh",
         "lang":"zh"
      },
      "secondaryFileExt":"",
      "author":[  
         {  
            "c":"7KB1",
            "value":"Weiya Zhang",
            "permId":114604294655
         },
         {  
            "c":"7KAY",
            "value":"Zhouyu Deng",
            "permId":114604294658
         }
      ],
      ...
      "docPermID":112664278656,
      "transpRpt":false,
      "tocEnd":-1,
      "hasEarns":false,
      "estimateRating":2,
      "disciplinesResp":[  

      ],
      "cntry":[  
         {  
            "c":"CHN",
            "value":"China"
         }
      ],
      "ratingType":"overall",
      "localCode":"7cc6b08c-fed8-40aa-a1c9-3c5d5f86605c",
      "tkr":[  
         {  
            "organizationPermId":5000514619,
            "value":"002773.SZ",
            "bPrim":true
         }
      ],
      "docTyp":[  

      ],
      "headline":"康弘药业-朗沐 DME 适应症获批,其竞争优势明显-05/17/2019",
      "ind":[  
         {  
            "c":"5110",
            "value":"Chemicals",
            "setcode":"TRBC"
         },
         {  
            "c":"220",
            "value":"Chemicals",
            "setcode":"MX#001"
         },
         {  
            "c":"56201040",
            "value":"Pharmaceuticals (4)",
            "setcode":"TRBC"
         }
      ],
      "releaseDate":"2019-05-17T09:52:03.000Z",
      "docID":85235540,
      "hasFPP":true,
      ...

 

7) Retrieving Research Document

After receiving Research document information, the application then add the document information such as headline and docID into a table in the Web page. Each row in the table contains buttons to retrieve Research Document in JSON text or PDF file.

Once the button is clicked, the getDocumentUrl() method will be invoked to request CDN signed URL to retrieve Research document. Once the CDN signed URL is received, the Application will directly open the document from the CDN signed URL in a new tab of browser.

ERTRESTController.prototype.getDocumentUrl = function (uuId, docID, type)
{
    let baseUrl = research_document_url
    let fileFormat = "/" + type;

    let finalUrl = baseUrl + docID + fileFormat;

    let request_data = {
        method: 'GET',
        uri: finalUrl,
        headers: {
            'Accept': 'application/json',
            'Authorization': 'Bearer ' + this.auth_obj.access_token
        },
        qs: {
            uuid: uuId
        },
        json: true,
        resolveWithFullResponse: true
    };

 

Application Setup

The application source code is available at GitHub page. It requires the following JavaScript package for the JavaScript runtime environment Node.js. Please note that the NPM program is installed on your computer when you install Node.js.

  • express
  • request-promise
  • request

You can use the following command to download required package.

npm install

The EDP/Research access is required to run this example. Please contact your Refinitiv's Technical Account Manager or Technical Relationship Manager to help you to access EDP account and services. You will receive your Machine ID as a user name and a link to activate your machine account and set your password via the Welcome Email that you receive when you subscribe to ERT in Cloud.

 

How to run

  1. Configure EDP Username, password and UUID in the EDP_Research.html.
let credential = {
            'username': '<EDP Username>',  // Username
            'password': '<EDP Password>',  // Password
            'clientId': '<EDP Client>',    // AppKey
            'uuid': '<EDP User UUID>'      // User UUID
        };

 

  1. Run Node .js server
> node server.js

 

  1. Open the "http://localhost:8080/EDP_Research.html" URL in browser. The application will automatically login to EDP, subscribe Research Alert and poll alert messages from SQS queue.

Below is the sample page.

Diagnostic

Application normally logs more information such as body of response message in Web browser and Node JS console. You can use this information for diagnostic, when application encouters any issues.

 

Reference