1. Home
  2. Article Catalog
  3. How to implement Refinitiv Real-Time Optimized Node.js application with TypeScript part 1: Authentication and Service Discovery with RDP REST Services

Article

How to implement Refinitiv Real-Time - Optimized Node.js application with TypeScript part 1: Authentication and Service Discovery with RDP REST Services

Wasin Waeosri
Developer Advocate Developer Advocate

Overview

Update: January 2021

Refinitiv Data Platform (RDP) gives you seamless and holistic access to all of the Refinitiv content (whether real-time or non-real-time, analytics or alternative datasets), commingled with your content, enriching, integrating, and distributing the data through a single interface, delivered wherever you need it.

As part of the Refinitiv Data Platform, Refinitiv Real-Time - Optimized (formerly known as ERT in Cloud) gives you access to best in class Real-Time market data delivered in the cloud. Refinitiv Real-Time - Optimized is a new delivery mechanism for RDP, using the AWS (Amazon Web Services) cloud. Once a connection to RDP is established using Refinitiv Real-Time - Optimized, data can be retrieved using Websocket API for Pricing Streaming and Real-Time Services aka WebSocket API.

This article is the first part of how to implement the Refinitiv Real-Time - Optimized Node.js application with TypeScript language articles. The application source codes are implemented in TypeScript language to authenticate with RDP gateway via HTTP REST API, then connects and consuming Refinitiv Real-Time - Optimized steaming market data through WebSocket connection. The source codes will be compiled into a readable JavaScript file and run in a console with Node.js runtime environment. This first part article focuses on how to implement a Node.js TypeScript application to authenticate and get Pricing service discovery from RDP REST API.

*Note: The application functionalities are based on Refinitiv Real-Time - Optimized market_price_edpgw_service_discovery.py Python example. Access to the Refinitiv Refinitiv Data Platform and ERefinitiv Real-Time - Optimized are required.

TypeScript Overview

TypeScript is an open-source programming language developed and maintained by Microsoft. The language lead architect is Anders Hejlsberg, lead architect of C# and creator of Delphi and Turbo Pascal. TypeScript is a typed superset of JavaScript that compiles to readable, standards-based JavaScript. The syntax is designed for application-scale JavaScript by adding optional types, classes, modules, ECMAScript 2015 features, and future proposals to JavaScript. TypeScript supports tools for large-scale JavaScript applications for any browser, for any host, on any OS. TypeScript is a first-class programming language in Microsoft Visual Studio and Angular web application platform. It also supported by various application frameworks like ReactNodeJS and Express frameworkASP.Net CoreVue.js, and more.

Application files

The project includes completed TypeScript source codes, NPM and Webpack. All dynamic dependencies for compiling and building Node.js JavaScript source file are defined in package.json file which can be installed via npm install command.

  • src/ folder: Application source code folder, all codes are written in TypeScript language
  • src/market_price_edpgw_service_discovery.ts: The main Node.js source code
  • src/edp_rest_controller.ts: The controller application file that handles RDP REST API
  • src/edprt_websocket_controller.ts: The controller application that handles Refinitiv Real-Time - Optimized WebSocket API
  • src/json_msg_interface.ts: The TypeScript interface file for all RDP and Refinitiv Real-Time - Optimized messages
  • dist/ folder: The folder that the compiled Node.js application file named market_price_edpgw_service_discovery.js will be generated and deployed
  • node_modules/ folder: NPM dependencies installation folder
  • LICENSE.md: License declaration file
  • README.md: readme file
  • package.json: Project's NPM dependencies file
  • tsconfig.json: Project's TypeSccript compiler configurations file
  • webpack.config.js: Project's Webpack compiler configurations file
  • assetinfo.json: Project ID file (for internal Refinitiv)

RDP and Refinitiv Real-Time - Optimized Application Implementation Process

The application requires the following steps to authentication and consume data from RDP and Refinitiv Real-Time - Optimized

  1. Send a HTTP POST request message with username and password information to RDP Authentication service to obtain access token, refresh token, expiration time and scope information
  2. Send a HTTP GET request message with access token to RDP Realtime Service Discovery service to get lists of Refinitiv Real-Time - Optimized WebSocket streaming server endpoints, ports and location information (VIPs)
  3. Connect to Refinitiv Real-Time - Optimized WebSocket endpoint from step (2)
  4. Once a WebSocket connection is established, sends a JSON Login request message with access token Refinitiv Real-Time - Optimized WebSocket server
  5. Once the Login request message is accepted by Refinitiv Real-Time - Optimized WebSocket server, the application can send requests and closes as required to main watch-list of subscribed instruments.
  6. Handle asynchronous responses and status changes when they occur, including “pings” and connection failures.
  7. Re-issue RDP Authentication service before token expiration to keep session open..
  8. Once the application receives a new access token, application must re-send a new JSON Login request message with a new access token and refresh attribute value false to Refinitiv Real-Time - Optimized WebSocket server
  9. All current item streams will remain opened

RDP REST API Application Code Walkthrough

All HTTP REST communication logic is implemented in edp_rest_controller.ts application class file. The edp_rest_controller.ts application use Node.js EventEmitter class to asynchronous communicate with market_price_edpgw_service_discovery.ts main application. The main application registers for specific edp_rest_controller's events to receive HTTP results from RDP HTTP REST services.

The edp_rest_controller.ts application uses NPM's Request-Promise module to make HTTP request asynchronously with RDP HTTP REST services.

All REST JSON messages structure are defined in json_msg_interface.ts file.

json_msg_interface.ts Code Walkthrough: Define JSON messages interface

Firstly, we define interfaces for all JSON messages in the json_msg_interface.ts file. The TypeScript's interfaces feature helps developers type-checking their variables and data in the implementation time to avoid data type error in a final JavaScript application. We use export keyword to let these interfaces can be used by other modules.

    	
            

export interface IREST_Token {

    username: string;

    password: string;

    grant_type: string;

    takeExclusiveSignOnControl: boolean;

    scope: string;

    client_id: string;

}

 

export interface IREST_RefreshToken {

    username: string;

    refresh_token: string;

    grant_type: 'refresh_token';

    takeExclusiveSignOnControl: boolean;

}

Next, we define interfaces for Request-Promise module's JSON parameters. 

    	
            

export interface IREST_TokenPostData {

    method: string;

    uri: string;

    form: any;

    headers: any;

    json: boolean;

    resolveWithFullResponse: boolean;

    auth: any;

    simple: boolean;

}

 

export interface IREST_ServiceGetData {

    method: string;

    uri: string;

    qs: IDataAuthen;

    headers: any;

    json: boolean;

    resolveWithFullResponse: boolean;

}

Finally, we define an interface for RDP authentification object which will be used for requesting further RDP and Refinitiv Real-Time Optimized services information. 

    	
            

export interface IDataAuthen {

    access_token: string;

    refresh_token: string;

    expires_in: string;

    transport: string;

    dataformat: string;

    position: string;

    appId: string;

}

edp_rest_controller.ts Code Walkthrough: Initiate the class

Firstly, we import all required interfaces and Node.js modules in the edp_rest_controller.ts file.

    	
            

import * as EventEmitter from "events";

import * as rp from "request-promise";

 

import { IREST_Token } from "./json_msg_interface";

import { IREST_TokenPostData } from "./json_msg_interface";

import { IDataAuthen } from "./json_msg_interface";

import { IREST_ServiceGetData } from "./json_msg_interface";

import { IREST_RefreshToken } from "./json_msg_interface";

Next, we create a RESTConnectionController class in the edp_rest_controller.ts file. This class handles all HTTP operations between the main application class (market_price_edpgw_service_discovery.ts file) and RDP HTTP REST services. We use export keyword to let this class can be used by other modules/classes. The main application needs to pass username, password, client_id and scope information to RESTConnectionController class via a constructor function. We also create the emitter variable as an EventEmitter object for communicating with the main class.

    	
            

export { RESTConnectionController };

 

class RESTConnectionController {

 

    private username: string;

    private password: string;

    private client_id: string;

    private emitter: EventEmitter;

    private scope: string;

 

    constructor(username: string, password: string, client_id: string, scope: string, emitter: EventEmitter) {

        this.username = username;

        this.password = password;

        this.client_id = client_id;

        this.scope = scope;

        this.emitter = emitter;

    }

 

}

Next, we implement a sendHttpRequest() function to asynchronous sends and receives HTTP request/response messages with RDP HTTP REST services via Request-Promise module. If a request success (receives HTTP Status 200 from RDP gateway), it returns HTTP response body to a caller. If a request fails, it returns HTTP error response to caller instead. The function returns result as a Promise object for both sucess and fail cases.

    	
            

// Send HTTP message using Node.js's request-promise library

private sendHttpRequest(post_message: any) {

 

    // console.log('sendHttpRequest()');

    return rp(post_message)

        .then((response) => { // Get HTTP response 200

            // console.log(`response.statusCode = ${response.statusCode}`);

            return response.body;

        }).catch((error) => { // Get HTTP Error response

            throw error; nvm

        });

}

edp_rest_controller.ts application: Implementing Authentication Process

Next, we implement a getToken() function to request RDP Authentication for both newly authentication and re-authentication cases. This function requires RDP Authentication service URL as a mandatory parameter. A refresh token parameter is an optional parameter for re-authentication case only. We create a "data" JSON object to keep authentication JSON POST message for later use.

    	
            

class RESTConnectionController {

    

    // Get Authentication Token and Re-Refresh Token based on Machine ID and Password

    public getToken = (url: string, refresh_token?: string) => {

 

        // Authentication Post body

        const data: IREST_Token = {

            username: this.username,

            password: this.password,

            grant_type: 'password',

            takeExclusiveSignOnControl: true,

            scope: 'trapi',

            client_id: this.client_id,

        };

 

    }

}

Then we create Request-Promise's options object to specify a HTTP request information (URL, method, outgoing data, etc) for RDP Authentication service. The service requires the following operations and parameters:

  • HTTP method: POST
  • HTTP headers:
    • Accept: 'application/json'
    • Content-Type: 'application/x-www-form-urlencoded'
    • Authorization: 'Basic'
  • HTTP Authentication: with username and empty password (the username and password are also required in POST data).

If it is a first login case (the refresh_token variable is null), we set RDP Authentication JSON Post message to options.form parameter. The string variable named result is use for setting an event name that the main application has registered an event to receive data.

    	
            

// Get Authentication Token and Re-Refresh Token based on Machine ID and Password

public getToken = (url: string, refresh_token?: string) => {

 

    // Authentication Post body

    const data: IREST_Token = {

        username: this.username,

        password: this.password,

        grant_type: 'password',

        takeExclusiveSignOnControl: true,

        scope: 'trapi',

        client_id: this.client_id,

    };

 

    let result: string = '';

 

    // HTTP request options

    const authen_options: IREST_TokenPostData = {

        method: 'POST',

        uri: url,

        form: '',

        headers: {

            'Accept': 'application/json',

            'Content-Type': 'application/x-www-form-urlencoded',

            'Authorization': 'Basic',

        },

        json: true,

        resolveWithFullResponse: true,

        auth: {

            username: data.username,

            password: '',

        },

        simple: true,

    };

 

    if (!refresh_token) { // First Login

        authen_options.form = data;

        result = 'EDPREST_getTokenSuccess';

    }

 

}

Finally, we send a HTTP Authentication request message to RDP via a sendHttpRequest() function. If an authentication process success , an anonymous onFulfielled function will be called with a HTTP response. The application then uses EventEmitter to send a result to "EDPREST_getTokenSuccess" event listener.

If authentication process fail, an anonymous onRejected function will be called with a HTTP error response and sends error result to "EDPREST_getTokenFail" event listener.

    	
            

// Get Authentication Token and Re-Refresh Token based on Machine ID and Password

public getToken = (url: string, refresh_token?: string) => {

 

    // Code from previous steps

    this.sendHttpRequest(authen_options).then((response: any) => {

        console.log('HTTP Request: EDP-GW Authentication success');

        this.emitter.emit(result, response); // Send response back to a main class

    }, (error: any) => {

        console.log(`HTTP Request: EDP-GW Authentication error ${error}`);

        this.emitter.emit('EDPREST_getTokenFail', error); // Send error back to a main class

    });

}

edp_rest_controller.ts application: Implmenting Pricing service discovery Process

Next, we implement a getServiceDiscovery() function to request RDP Pricing service discovery information. This RDP Pricing service discovery returns a list of Refinitiv Real-Time Optimized WebSocket servers endpoints and ports. This function requires RDP Pricing service URL and RDP access token from a previous step.

    	
            

// Get EDP streaming/pricing

public getServiceDiscovery = (url: string, data: IDataAuthen) => {

 

}

We create Request-Promise's options object to specify the HTTP request information (URL, method, outgoing data, etc) for RDP Pricing service. The RDP Pricing service discovery service requires the following operations and parameters:

  • HTTP method: GET
  • HTTP headers:
    • Accept: 'application/json'
    • Content-Type: 'application/json'
    • Authorization: 'Bearer ' + RDP Access Token
    	
            

// Get EDP streaming/pricing

public getServiceDiscovery = (url: string, data: IDataAuthen) => {

 

    const Authorization: string = 'Bearer ' + data.access_token; // Set Authorization Header

 

    // HTTP request options

    const ERT_service_options: IREST_ServiceGetData = {

        method: 'GET',

        uri: url,

        qs: data,

        headers: {

            'Accept': 'application/json',

            'Content-Type': 'application/json',

            'Authorization': Authorization,

        },

        json: true,

        resolveWithFullResponse: true,

    };

}

Finally, we send a HTTP request to RDP via a sendHttpRequest() function. If a request success , an anonymous onFulfielled function will be called with the HTTP response. The application uses EventEmitter to send a result to "EDPREST_getServiceDiscoverySuccess" event listener.

If a request fail, an anonymous onRejected function will be called with the HTTP error response and sends error result to "EDPREST_getServiceDiscoveryFail" event listener.

    	
            

// Get Authentication Token and Re-Refresh Token based on Machine ID and Password

public getServiceDiscovery = (url: string, data: IDataAuthen) => {

 

    // Code from previous steps

    console.log(`Sending EDP-GW service discovery request to ${url}`)

 

    this.sendHttpRequest(ERT_service_options).then((response: any) => {

        

        console.log('HTTP Request: EDP-GW service discovery success');

        this.emitter.emit('EDPREST_getServiceDiscoverySuccess', response);

    }, (error: any) => {

        console.log(`HTTP Request: EDP-GW service discovery error ${error}`);

        this.emitter.emit('EDPREST_getServiceDiscoveryFail', error);

    });

}

market_price_edpgw_service_discovery.ts application: Implmenting Authentication Process

Let get back to a main application implementation (market_price_edpgw_service_discovery.ts file). Firstly we import all require mododules and classes for EventEmitter, json_msg_interface.ts interface and RESTConnectionController class. We also initialize an EventEmitter object here.

    	
            

// market_price_edpgw_service_discovery.ts

 

import * as EventEmitter from "events";

import { IDataAuthen } from "./json_msg_interface";

import { RESTConnectionController } from "./edp_rest_controller";

import { WSConnectionController } from "./edprt_websocket_controller";

 

// initiate EventEmitter class for asynchronous sending and receiving "events" between each class  

class MyEmitter extends EventEmitter { }

const myEmitter: MyEmitter = new MyEmitter();

Next, we decare all variables that will be used for HTTP REST requests process such as the RDP URL parameters, username and password. The auth_obj variable will be used for keeping RDP authentication and VIPs information inside the application and classes.

    	
            

let edp_url: string = '';

let username: string = '';

let password: string = '';

let client_id: string = '';

 

let auth_hostname: string = 'api.refinitiv.com';

 

let auth_port: string = '443';

let auth_path: string = 'auth/oauth2/beta1/token';

const discovery_path: string = 'streaming/pricing/v1/';

 

const auth_obj: IDataAuthen = {

    access_token: '',

    refresh_token: '',

    expires_in: '',

    transport: 'websocket',

    dataformat: 'tr_json2',

    position: '',

    appId: '256',

}

Next, we parse user's input command line arguments with Python optimist module. The application supports only a username and password parameters in this phase. 

    	
            

const argv = require("optimist").argv; // For getting command line arguments 

 

// get command line arguments for setting application parameters

 

if (argv.user) {

    username = argv.user.toString();

}

if (argv.password) {

    password = argv.password.toString();

}

 

if (argv.help) {

    console.log('Usage: $>node market_price_edpgw_service_discovery.js [--user user] [--password password] [--help]');

    process.exit();

}

Then we initialize the RESTConnectionController object and call RESTConnectionController.getToken() function to request RDP Authentication with username and password.

    	
            

let rest_app: RESTConnectionController;

 

// Initiate REST controller class

rest_app = new RESTConnectionController(username, password, client_id, scope, myEmitter);

 

edp_url = `https://${auth_hostname}:${auth_port}/${auth_path}`;

 

// Get EDP Authentication token information

rest_app.getToken(edp_url);

Next we implement a callback function to handle an authentication success "EDPREST_getTokenSuccess" event. This "EDPREST_getTokenSuccess" callback function gets the authentication information from RESTConnectionController class and keeps it in an auth_obj object for later use. The RDP Authentication information contains the following properties:

  • access_token: the authentication token for login to RDP and Refinitiv Real-Time - Optimized service.
  • refresh_token: the token which used for periodically refresh the access_token in order to keep the token alive.
  • expires_in: access_token expiration period.
    	
            

//Get Authentication Token (first request)

myEmitter.on('EDPREST_getTokenSuccess', (event) => {

    // console.log(`Get Token ${JSON.stringify(event)}`); 

    auth_obj['access_token'] = event['access_token'];

    auth_obj['refresh_token'] = event['refresh_token'];

    auth_obj['expires_in'] = event['expires_in'];

    console.log(`EDP-GW Authentication succeeded. RECEIVED: ${JSON.stringify(event, null, 2)}`);

});

Then we implement a callback functiont to handle an authentication fail "EDPREST_getTokenFail" event. This "EDPREST_getTokenFail" callback function gets authentication error information from RESTConnectionController class and print it to a console.

    	
            

// Fail to get EDP Authentication Token

myEmitter.on('EDPREST_getTokenFail', (err_event) => {

    console.error(`Get Token Error ${err_event}`);

});

market_price_edpgw_service_discovery.ts application: Implmenting Service Discovery Process

After RDP login sucess, applciation can access the service entitled in RDP. To access Refinitiv Real-Time Optimized, application first needs to determine the global VIPs for accessing the service.

In an authentication success "EDPREST_getTokenSuccess" event callback, we pass the RDP Streaming Service Discovery URL (https://api.refinitiv.com/streaming/pricing/v1/) and auth_obj variable to RESTConnectionController.getServiceDiscovery() function to request VIPs from RDP.

    	
            

//Get Authentication Token (first request)

myEmitter.on('EDPREST_getTokenSuccess', (event) => {

    // console.log(`Get Token ${JSON.stringify(event)}`); 

    auth_obj['access_token'] = event['access_token'];

    auth_obj['refresh_token'] = event['refresh_token'];

    auth_obj['expires_in'] = event['expires_in'];

    console.log(`EDP-GW Authentication succeeded. RECEIVED: ${JSON.stringify(event, null, 2)}`);

 

 

    edp_url = `https://${auth_hostname}/${discovery_path}`;

    rest_app.getServiceDiscovery(edp_url, auth_obj);

});

Next we implement a callback function to handle VIPs information from "EDPREST_getServiceDiscoverySuccess" event. In this phase, we just print out incoming VIPs information to console. The VIPs information will be used for connecting to Refinitiv Real-Time Optimized WebSocket connection in the next phase.

    	
            

// Get EDP Pricing Service Discovery information, then initiates WebSocket connection

myEmitter.on('EDPREST_getServiceDiscoverySuccess', (event) => {

    // console.log(`Get Service Discovery ${JSON.stringify(event)}`); 

    console.log(`EDP-GW Service discovery succeeded. RECEIVED: ${JSON.stringify(event, null, 2)}`);

   

});

The services returned are associated with the following properties:

  • provider: Indicates public cloud provider. Now, only aws.
  • location: Indicates where the infrastructure is deployed. For AWS, it is in terms of availability zone (AZ). If more than one availability zone is stated, it means RDP-RealTime performs cross AZ failover on-behalf of client.
  • transport: Indicates which transport type to be used to access service.
  • dataFormat: Indicates the data format used. It is the presentation of wire protocol format.
  • port: Indicates IP port number used to establish connection.
  • endpoint: The DNS name of service access endpoint.

Application Setup

The application source code is available at GitHub page. You can get it via the following git command

    	
            
$>git clone git@github.com:Refinitiv-API-Samples/Example.ERT.TypeScript.NodeJS.MarketPrice.git

This example requires the following dependencies softwares and runtime

  1. Node.js runtime - version 8.9.3 or higher.
  2. npm package manager (included in Node.js)
  3. TypeScript compiler (will be installed via npm install command)
  4. Webpack JavaScript module bundler (will be installed via npm install command)

please check the README.md in the source code project for more detail.

The RDP/Refinitiv Real-Time - Optimized access is required to run this example. Please contact your Refinitiv's representative to help you to access RDP account and services. You will receive your Machine ID as a user name and a link to activate your machine account and set your password via the Welcome Email that you receive when you subscribe to Refinitiv Real-Time - Optimized.

How to run this example

  1. Unzip or download the example project folder into a directory of your choice
  2. Run $> npm install command in the command prompt to install all the dependencies required to run the sample in a subdirectory called node_modules/.
  3. If the machine is behind a proxy server, you need to configure Node.js uses proxy instead of a direct HTTP connection via the following command in a command prompt: 
    	
            
$> node market_price_edpgw_service_discovery.js --user <Your Real-Time Optimized username> --password <Your  Real-Time Optimized password>

Upon execution, you will be presented with authentication and efinitiv Real-Time Optimized discovery processes via RDP Gateway REST API, then followed by initial WebSocket connection between the application and efinitiv Real-Time Optimized.

    	
            

Sending authentication request with password to https://api.refinitiv.com:443/auth/oauth2/v1/token

    HTTP Request: RDP-GW Authentication success

    RDP-GW Authentication succeeded. RECEIVED: {

    "access_token": "<Access Token>",

    "refresh_token": "<Refresh Token>",

    "expires_in": "300",

    "scope": "trapi.streaming.pricing.read",

    "token_type": "Bearer"

    }

    Sending RDP-GW service discovery request to https://api.refinitiv.com/streaming/pricing/v1/

    HTTP Request: RDP-GW service discovery success

    RDP-GW Service discovery succeeded. RECEIVED: {

    "services": [

        {

        "port": 443,

        "location": [

            "us-east-1a",

            "us-east-1b"

        ],

        "provider": "aws",

        "transport": "websocket",

        "endpoint": "amer-3-t3.streaming-pricing-api.refinitiv.com",

        "dataFormat": [

            "tr_json2"

        ]

        },

        {

        "port": 443,

        "location": [

            "us-east-1b"

        ],

        "provider": "aws",

        "transport": "websocket",

        "endpoint": "amer-2-t3.streaming-pricing-api.refinitiv.com",

        "dataFormat": [

            "tr_json2"

        ]

        },

        {

        "port": 443,

        "location": [

            "us-east-1a"

        ],

        "provider": "aws",

        "transport": "websocket",

        "endpoint": "amer-1-t3.streaming-pricing-api.refinitiv.com",

        "dataFormat": [

            "tr_json2"

        ]

        }

    ]

    }