Overview

Elektron Data Platform (EDP) gives you seamless and holistic access to all of Refinitiv content (whether real-time or non- real-time, analytics or alternative datasets), commingled with your own content, enriching, integrating and distributing the data through a single interface, delivered wherever you need it.

As part of the Elektron Data Platform, Elektron Real Time in Cloud (ERT in Cloud) gives you access to best in class Real Time market data delivered in the cloud. ERT in Cloud is a new delivery mechanism for EDP, using the AWS (Amazon Web Services) cloud. Once a connection to EDP is established using ERT in Cloud, data can be retrieved using Elektron WebSocket API.

This article is the first part of how to implement the ERT in Cloud Node.js application with TypeScript language articles. The application source codes are implemented in TypeScript language to authentcate with EDP gateway via HTTP REST API, then connects and consuming ERT in Cloud straming market data through WebSocket connection. The source codes will be compiled to readable JavaScript file and run in a console with Node.js runtime environment. This first part article focuses on how to implement a Node.js TypeScript application to authenticate and get Pricing service discovery from EDP REST API.

*Note: The application functionalities are based on ERT in Cloud market_price_edpgw_service_discovery.py Python example. Access to the Refinitiv Elektron Data Platform and Elektron Real Time in Cloud are required.

TypeScript Overview

TypeScript is an open-source programming language developed and maintained by Microsoft. The language lead architect is Anders Hejlsberg, principal designer and lead architect of C# and creator of Delphi and Turbo Pascal. TypeScript is a typed superset of JavaScript that compiles to readable, standards-based JavaScript. The syntax is designed for application-scale JavaScript by adding optional types, classes, modules, ECMAScript 2015 features and future proposals to JavaScript. TypeScript supports tools for large-scale JavaScript applications for any browser, for any host, on any OS. TypeScript is as a first-class programming language in Microsoft Visual Studio and Angular web application platform. It also supported by various application frameworks like ReactNodeJS and Express frameworkASP.Net CoreVue.js and more.

Application files

The project includes completed TypeScript source codes, NPM and Webpack. All dynamic dependencies for compiling and building Node.js JavaScript source file are defined in package.json file which can be installed via npm install command.

  • src/ folder: Application source code folder, all codes are written in TypeScript language
  • src/market_price_edpgw_service_discovery.ts: The main Node.js source code
  • src/edp_rest_controller.ts: The controller application file that handles EDP REST API
  • src/edprt_websocket_controller.ts: The controller application that handles ERT in Cloud WebSocket API
  • src/json_msg_interface.ts: The TypeScript interface file for all EDP and ERT in Cloud messages
  • dist/ folder: The folder that the compiled Node.js application file named market_price_edpgw_service_discovery.js will be generated and deployed
  • node_modules/ folder: NPM dependencies installation folder
  • LICENSE.md: License declaration file
  • README.md: readme file
  • package.json: Project's NPM dependencies file
  • tsconfig.json: Project's TypeSccript compiler configurations file
  • webpack.config.js: Project's Webpack compiler configurations file
  • assetinfo.json: Project ID file (for internal Refinitiv)

EDP and ERT in Cloud Application Implementation Process

The application requires the following steps to authentication and consume data from EDP and ERT in Cloud

  1. Send a HTTP POST request message with username and password information to EDP Authentication service to obtain access token, refresh token, expiration time and scope information
  2. Send a HTTP GET request message with access token to EDP Realtime Service Discovery service to get lists of ERT in Cloud WebSocket streaming server endpoints, ports and location information (VIPs)
  3. Connect to ERT in Cloud WebSocket endpoint from step (2)
  4. Once a WebSocket connection is established, sends a JSON Login request message with access token ERT in Cloud WebSocket server
  5. Once the Login request message is accepted by ERT in Cloud WebSocket server, the application can send requests and closes as required to main watch-list of subscribed instruments.
  6. Handle asynchronous responses and status changes when they occur, including “pings” and connection failures.
  7. Re-issue EDP Authentication service before token expiration to keep session open..
  8. Once the application receives a new access token, application must re-send a new JSON Login request message with a new access token and refresh attribute value false to ERT in Cloud WebSocket server
  9. All current item streams will remain opened

EDP REST API Application Code Walkthrough

All HTTP REST communication logic are implemented in edp_rest_controller.ts application class file. The edp_rest_controller.ts application use Node.js EventEmitter class to asynchronous communicate with market_price_edpgw_service_discovery.ts main application. The main application registers for specific edp_rest_controller's events to receive HTTP results from EDP HTTP REST services.

The edp_rest_controller.ts application uses NPM's Request-Promise module to make HTTP request asynchronously with EDP HTTP REST services.

All REST JSON messages structure are defined in json_msg_interface.ts file.

json_msg_interface.ts Code Walkthrough: Define JSON messages interface

Firstly, we define interfaces for all JSON messages in the json_msg_interface.ts file. The TypeScript's interfaces feature helps developers type-checking their variables and data in the implementation time to avoid data type error in a final JavaScript application. We use export keyword to let these interfaces can be used by other modules.

export interface IREST_Token {
    username: string;
    password: string;
    grant_type: string;
    takeExclusiveSignOnControl: boolean;
    scope: string;
    client_id: string;
}

export interface IREST_RefreshToken {
    username: string;
    refresh_token: string;
    grant_type: 'refresh_token';
    takeExclusiveSignOnControl: boolean;
}

Next, we define interfaces for Request-Promise module's JSON parameters.

export interface IREST_TokenPostData {
    method: string;
    uri: string;
    form: any;
    headers: any;
    json: boolean;
    resolveWithFullResponse: boolean;
    auth: any;
    simple: boolean;
}

export interface IREST_ServiceGetData {
    method: string;
    uri: string;
    qs: IDataAuthen;
    headers: any;
    json: boolean;
    resolveWithFullResponse: boolean;
}

Finally, we define an interface for EDP authentication object which will be used for requesting further EDP and ERT in Cloud services information.

export interface IDataAuthen {
    access_token: string;
    refresh_token: string;
    expires_in: string;
    transport: string;
    dataformat: string;
    position: string;
    appId: string;
}

edp_rest_controller.ts Code Walkthrough: Initiate the class

Firstly, we import all required interfaces and Node.js modules in the edp_rest_controller.ts file.

import * as EventEmitter from "events";
import * as rp from "request-promise";

import { IREST_Token } from "./json_msg_interface";
import { IREST_TokenPostData } from "./json_msg_interface";
import { IDataAuthen } from "./json_msg_interface";
import { IREST_ServiceGetData } from "./json_msg_interface";
import { IREST_RefreshToken } from "./json_msg_interface";

 

Next, we create a RESTConnectionController class in the edp_rest_controller.ts file. This class handles all HTTP operations between the main application class (market_price_edpgw_service_discovery.ts file) and EDP HTTP REST services. We use export keyword to let this class can be used by other modules/classes. The main application needs to pass username, password, client_id and scope information to RESTConnectionController class via a constructor function. We also create the emitter variable as an EventEmitter object for communicating with the main class.

export { RESTConnectionController };

class RESTConnectionController {

    private username: string;
    private password: string;
    private client_id: string;
    private emitter: EventEmitter;
    private scope: string;

    constructor(username: string, password: string, client_id: string, scope: string, emitter: EventEmitter) {
        this.username = username;
        this.password = password;
        this.client_id = client_id;
        this.scope = scope;
        this.emitter = emitter;
    }

}

Next, we implement a sendHttpRequest() function to asynchronous sends and receives HTTP request/response messages with EDP HTTP REST services via Request-Promise module. If a request success (receives HTTP Status 200 from EDP gateway), it returns HTTP response body to a caller. If a request fails, it returns HTTP error response to caller instead. The function returns result as a Promise object for both sucess and fail cases.

// Send HTTP message using Node.js's request-promise library
private sendHttpRequest(post_message: any) {

    // console.log('sendHttpRequest()');
    return rp(post_message)
        .then((response) => { // Get HTTP response 200
            // console.log(`response.statusCode = ${response.statusCode}`);
            return response.body;
        }).catch((error) => { // Get HTTP Error response
            throw error; nvm
        });
}

edp_rest_controller.ts application: Implmenting Authentication Process

Next, we implement a getToken() function to request EDP Authentication for both newly authentication and re-authentication cases. This function requires EDP Authentication service URL as mandatory parameter. A refresh token parameter is an optional parameter for re-authentication case only. We create a "data" JSON object to keep authentication JSON POST message for later use.

class RESTConnectionController {
    
    // Get Authentication Token and Re-Refresh Token based on Machine ID and Password
    public getToken = (url: string, refresh_token?: string) => {

        // Authentication Post body
        const data: IREST_Token = {
            username: this.username,
            password: this.password,
            grant_type: 'password',
            takeExclusiveSignOnControl: true,
            scope: 'trapi',
            client_id: this.client_id,
        };

    }
}

Then we create Request-Promise's options object to specify a HTTP request information (URL, method, outgoing data, etc) for EDP Authentication service. The service requires the following operations and parameters:

  • HTTP method: POST
  • HTTP headers:
    • Accept: 'application/json'
    • Content-Type: 'application/x-www-form-urlencoded'
    • Authorization: 'Basic'
  • HTTP Authentication: with username and empty password (the username and password are also required in POST data).

If it is a first login case (the refresh_token variable is null), we set EDP Authentication JSON Post message to options.form parameter. The string variable named result is use for setting an event name that the main application has registered an event to receive data.

// Get Authentication Token and Re-Refresh Token based on Machine ID and Password
public getToken = (url: string, refresh_token?: string) => {

    // Authentication Post body
    const data: IREST_Token = {
        username: this.username,
        password: this.password,
        grant_type: 'password',
        takeExclusiveSignOnControl: true,
        scope: 'trapi',
        client_id: this.client_id,
    };

    let result: string = '';

    // HTTP request options
    const authen_options: IREST_TokenPostData = {
        method: 'POST',
        uri: url,
        form: '',
        headers: {
            'Accept': 'application/json',
            'Content-Type': 'application/x-www-form-urlencoded',
            'Authorization': 'Basic',
        },
        json: true,
        resolveWithFullResponse: true,
        auth: {
            username: data.username,
            password: '',
        },
        simple: true,
    };

    if (!refresh_token) { // First Login
        authen_options.form = data;
        result = 'EDPREST_getTokenSuccess';
    }

}

Finally, we send a HTTP Authentication request message to EDP via a sendHttpRequest() function. If an authentication process success , an anonymous onFulfielled function will be called with a HTTP response. The application then uses EventEmitter to send a result to "EDPREST_getTokenSuccess" event listener.

If authentication process fail, an anonymous onRejected function will be called with a HTTP error response and sends error result to "EDPREST_getTokenFail" event listener.

// Get Authentication Token and Re-Refresh Token based on Machine ID and Password
public getToken = (url: string, refresh_token?: string) => {

    // Code from previous steps
    this.sendHttpRequest(authen_options).then((response: any) => {
        console.log('HTTP Request: EDP-GW Authentication success');
        this.emitter.emit(result, response); // Send response back to a main class
    }, (error: any) => {
        console.log(`HTTP Request: EDP-GW Authentication error ${error}`);
        this.emitter.emit('EDPREST_getTokenFail', error); // Send error back to a main class
    });
}

edp_rest_controller.ts application: Implmenting Pricing service discovery Process

Next, we implement a getServiceDiscovery() function to request EDP Pricing service discovery information. This EDP Pricing service discovery returns a list of ERT in Cloud WebSocket servers endpoints and ports. This function requires EDP Pricing service URL and EDP access token from a previous step.

// Get EDP streaming/pricing
public getServiceDiscovery = (url: string, data: IDataAuthen) => {

}

We create Request-Promise's options object to specify the HTTP request information (URL, method, outgoing data, etc) for EDP Pricing service. The EDP Pricing service discovery service requires the following operations and parameters:

  • HTTP method: GET
  • HTTP headers:
    • Accept: 'application/json'
    • Content-Type: 'application/json'
    • Authorization: 'Bearer ' + EDP Access Token
// Get EDP streaming/pricing
public getServiceDiscovery = (url: string, data: IDataAuthen) => {

    const Authorization: string = 'Bearer ' + data.access_token; // Set Authorization Header

    // HTTP request options
    const ERT_service_options: IREST_ServiceGetData = {
        method: 'GET',
        uri: url,
        qs: data,
        headers: {
            'Accept': 'application/json',
            'Content-Type': 'application/json',
            'Authorization': Authorization,
        },
        json: true,
        resolveWithFullResponse: true,
    };
}

Finally, we send a HTTP request to EDP via a sendHttpRequest() function. If a request success , an anonymous onFulfielled function will be called with the HTTP response. The application uses EventEmitter to send a result to "EDPREST_getServiceDiscoverySuccess" event listener.

If a request fail, an anonymous onRejected function will be called with the HTTP error response and sends error result to "EDPREST_getServiceDiscoveryFail" event listener.

// Get Authentication Token and Re-Refresh Token based on Machine ID and Password
public getServiceDiscovery = (url: string, data: IDataAuthen) => {

    // Code from previous steps
    console.log(`Sending EDP-GW service discovery request to ${url}`)

    this.sendHttpRequest(ERT_service_options).then((response: any) => {
        
        console.log('HTTP Request: EDP-GW service discovery success');
        this.emitter.emit('EDPREST_getServiceDiscoverySuccess', response);
    }, (error: any) => {
        console.log(`HTTP Request: EDP-GW service discovery error ${error}`);
        this.emitter.emit('EDPREST_getServiceDiscoveryFail', error);
    });
}

market_price_edpgw_service_discovery.ts application: Implmenting Authentication Process

Let get back to a main application implementation (market_price_edpgw_service_discovery.ts file). Firstly we import all require mododules and classes for EventEmitter, json_msg_interface.ts interface and RESTConnectionController class. We also initialize an EventEmitter object here.

// market_price_edpgw_service_discovery.ts

import * as EventEmitter from "events";
import { IDataAuthen } from "./json_msg_interface";
import { RESTConnectionController } from "./edp_rest_controller";
import { WSConnectionController } from "./edprt_websocket_controller";

// initiate EventEmitter class for asynchronous sending and receiving "events" between each class  
class MyEmitter extends EventEmitter { }
const myEmitter: MyEmitter = new MyEmitter();

Next, we decare all variables that will be used for HTTP REST requests process such as the EDP URL parameters, username and password. The auth_obj variable will be used for keeping EDP authentication and VIPs information inside the application and classes.

let edp_url: string = '';
let username: string = '';
let password: string = '';
let client_id: string = '';

let auth_hostname: string = 'api.edp.thomsonreuters.com';

let auth_port: string = '443';
let auth_path: string = 'auth/oauth2/beta1/token';
const discovery_path: string = 'streaming/pricing/v1/';

const auth_obj: IDataAuthen = {
    access_token: '',
    refresh_token: '',
    expires_in: '',
    transport: 'websocket',
    dataformat: 'tr_json2',
    position: '',
    appId: '256',
}

Next, we parse user's input command line arguments with Python optimist module. The application supports only a username and password parameters in this phase.

const argv = require("optimist").argv; // For getting command line arguments 

// get command line arguments for setting application parameters

if (argv.user) {
    username = argv.user.toString();
}
if (argv.password) {
    password = argv.password.toString();
}

if (argv.help) {
    console.log('Usage: $>node market_price_edpgw_service_discovery.js [--user user] [--password password] [--help]');
    process.exit();
}

Then we initialize the RESTConnectionController object and call RESTConnectionController.getToken() function to request EDP Authentication with username and password.

let rest_app: RESTConnectionController;

// Initiate REST controller class
rest_app = new RESTConnectionController(username, password, client_id, scope, myEmitter);

edp_url = `https://${auth_hostname}:${auth_port}/${auth_path}`;

// Get EDP Authentication token information
rest_app.getToken(edp_url);

Next we implement a callback function to handle an authentication success "EDPREST_getTokenSuccess" event. This "EDPREST_getTokenSuccess" callback function gets the authentication information from RESTConnectionController class and keeps it in an auth_obj object for later use. The EDP Authentication information contains the following properties:

  • access_token: the authentication token for login to EDP and ERT in Cloud service.
  • refresh_token: the token which used for periodically refresh the access_token in order to keep the token alive.
  • expires_in: access_token expiration period.
//Get Authentication Token (first request)
myEmitter.on('EDPREST_getTokenSuccess', (event) => {
    // console.log(`Get Token ${JSON.stringify(event)}`); 
    auth_obj['access_token'] = event['access_token'];
    auth_obj['refresh_token'] = event['refresh_token'];
    auth_obj['expires_in'] = event['expires_in'];
    console.log(`EDP-GW Authentication succeeded. RECEIVED: ${JSON.stringify(event, null, 2)}`);
});

Then we implement a callback functiont to handle an authentication fail "EDPREST_getTokenFail" event. This "EDPREST_getTokenFail" callback function gets authentication error information from RESTConnectionController class and print it to a console.

// Fail to get EDP Authentication Token
myEmitter.on('EDPREST_getTokenFail', (err_event) => {
    console.error(`Get Token Error ${err_event}`);
});

market_price_edpgw_service_discovery.ts application: Implmenting Service Discovery Process

After EDP login sucess, applciation can access the service entitled in EDP. To access ERT in Cloud service, application first needs to determine the global VIPs for accessing the service.

In an authentication success "EDPREST_getTokenSuccess" event callback, we pass the EDP Streaming Service Discovery URL (https://api.edp.thomsonreuters.com/streaming/pricing/v1/) and auth_obj variable to RESTConnectionController.getServiceDiscovery() function to request VIPs from EDP.

//Get Authentication Token (first request)
myEmitter.on('EDPREST_getTokenSuccess', (event) => {
    // console.log(`Get Token ${JSON.stringify(event)}`); 
    auth_obj['access_token'] = event['access_token'];
    auth_obj['refresh_token'] = event['refresh_token'];
    auth_obj['expires_in'] = event['expires_in'];
    console.log(`EDP-GW Authentication succeeded. RECEIVED: ${JSON.stringify(event, null, 2)}`);


    edp_url = `https://${auth_hostname}/${discovery_path}`;
    rest_app.getServiceDiscovery(edp_url, auth_obj);
});

Next we implement a callback function to handle VIPs information from "EDPREST_getServiceDiscoverySuccess" event. In this phase, we just print out incoming VIPs information to console. The VIPs information will be used for connecting to ERT in Cloud WebSocket connection in the next phase.

// Get EDP Pricing Service Discovery information, then initiates WebSocket connection
myEmitter.on('EDPREST_getServiceDiscoverySuccess', (event) => {
    // console.log(`Get Service Discovery ${JSON.stringify(event)}`); 
    console.log(`EDP-GW Service discovery succeeded. RECEIVED: ${JSON.stringify(event, null, 2)}`);
   
});

The services returned are associated with the following properties:

  • provider: Indicates public cloud provider. Now, only aws.
  • location: Indicates where the infrastructure is deployed. For AWS, it is in terms of availability zone (AZ). If more than one availability zone is stated, it means EDP-RealTime performs cross AZ failover on-behalf of client.
  • transport: Indicates which transport type to be used to access service.
  • dataFormat: Indicates the data format used. It is the presentation of wire protocol format.
  • port: Indicates IP port number used to establish connection.
  • endpoint: The DNS name of service access endpoint.

Application Setup

The application source code is available at GitHub page. You can get it via the following git command

$>git clone git@github.com:TR-API-Samples/Example.ERT.TypeScript.NodeJS.MarketPrice.git

This example requires the following dependencies softwares and runtime

  1. Node.js runtime - version 8.9.3 or higher.
  2. npm package manager (included in Node.js)
  3. TypeScript compiler (will be installed via npm install command)
  4. Webpack JavaScript module bundler (will be installed via npm install command)

please check the README.md in the source code project for more detail.

The EDP/ERT in Cloud access is required to run this example. Please contact your Refinitiv's Technical Account Manager or Technical Relationship Manager to help you to access EDP account and services. You will receive your Machine ID as a user name and a link to activate your machine account and set your password via the Welcome Email that you receive when you subscribe to ERT in Cloud.

How to run this example

  1. Unzip or download the example project folder into a directory of your choice
  2. Run $> npm install command in the command prompt to install all the dependencies required to run the sample in a subdirectory called node_modules/.
  3. If the machine is behind a proxy server, you need to configure Node.js uses proxy instead of a direct HTTP connection via the following command in command prompt: 
$> node market_price_edpgw_service_discovery.js --user <Your ERT in Cloud username> --password <Your ERT in Cloud password>

Upon execution, you will be presented with authentication and ERT in Cloud Service discovery processes via EDP Gateway REST API, then followed by initial WebSocket connection between the application and ERT in Cloud.

Sending authentication request with password to https://api.edp.thomsonreuters.com:443/auth/oauth2/beta1/token
HTTP Request: EDP-GW Authentication success
EDP-GW Authentication succeeded. RECEIVED: {
"access_token": "<Access Token>",
"refresh_token": "<Refresh Token>",
"expires_in": "300",
"scope": "trapi.streaming.pricing.read",
"token_type": "Bearer"
}
Sending EDP-GW service discovery request to https://api.edp.thomsonreuters.com/streaming/pricing/v1/
HTTP Request: EDP-GW service discovery success
EDP-GW Service discovery succeeded. RECEIVED: {
"services": [
    {
    "port": 443,
    "location": [
        "us-east-1a",
        "us-east-1b"
    ],
    "provider": "aws",
    "transport": "websocket",
    "endpoint": "amer-3.pricing.streaming.edp.thomsonreuters.com",
    "dataFormat": [
        "tr_json2"
    ]
    },
    {
    "port": 443,
    "location": [
        "us-east-1b"
    ],
    "provider": "aws",
    "transport": "websocket",
    "endpoint": "amer-2.pricing.streaming.edp.thomsonreuters.com",
    "dataFormat": [
        "tr_json2"
    ]
    },
    {
    "port": 443,
    "location": [
        "us-east-1a"
    ],
    "provider": "aws",
    "transport": "websocket",
    "endpoint": "amer-1.pricing.streaming.edp.thomsonreuters.com",
    "dataFormat": [
        "tr_json2"
    ]
    }
]
}

References

For further details, please check out the following resources:

For any question related to this article or EDP/ERT in Cloud API, please use the Developer Community Q&A Forum.