1. Home
  2. Article Catalog
  3. Setup a Simulated Real-Time Optimized Environment in 3 Steps

Article

Setup a Simulated Real-Time Optimized Environment in 3 Steps

Author:
Jirapongse Phuriphanvichai
Developer Advocate Developer Advocate

Real-Time Optimized (RTO) is Refinitiv conflated real-time content, hosted in the public cloud. It provides fast and simple access to our unparalleled content from hundreds of exchanges and OTC markets around the world. We can use WebSocket APIs or Real-Time SDKs to connect to Real-Time Optimized to consume real-time content. Real-Time Optimized is maintained by Refinitiv so we don’t have control over it. It means we are unable to shut down the services to test how applications handle unexpected behaviors originated by Real-Time Optimized or Refinitiv Data Platform services.

This article demonstrates how to set up a simulated Real-Time Optimized in 3 steps to test Refinitv Real-Time SDK’s applications. It also shows how to configure and run EMA examples to connect to this simulated Real-Time Optimized environment. 

Note: This simulated environment is for development and testing purposes only. 

Prerequisites

To follow the steps in this article the following applications are required.

1.      OpenSSL toolkit

2.      Node.js

3.      An RSSL interactive provider

Simulated Real-Time Optimized Environment

Refinitiv Real-Time Optimized is comprised of three secured services in the public cloud. The first two services are HTTPS REST services on the Refinitiv Data Platform for authentication and service discovery and the third service is a secured streaming service that provides financial streaming content. This article shows how to use a node application and an RSSL interactive provider application to simulate HTTPS REST services and a streaming service, respectively.

The following picture shows how the Refinitiv Real-Time SDK consumer applications connect to Refinitiv Real-Time Optimized and the simulated Real-Time Optimized environment.

There are three steps to set up a Simulated Real-Time Optimized Environment. To simplify the setup process, the Node.js application and RSSL interactive provider are running on the same machine. 

1.     Generate a certificate file and a private key file

Refinitiv Real-Time Optimized uses encrypted connections so we need to have a certificate file and a private key file for the simulated Real-Time Optimized environment. You may need to contact your IT team to generate those files for you. Otherwise, you can use the OpenSSL command to generate those files.  This is for development/testing purposes only.

Run the following command to generate a server certificate file and a server private key file for the www.test.com domain. 

    	
            openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365 -subj '/CN=www.test.com'
        
        
    

The key.pem file is the private key file and the cert.pem file is the certificate file.

Then, run the following command to remove the PEM pass phrase from the key.pem file.

    	
            

cp key.pem key.pem.org

openssl rsa -in key.pem.org -out key.pem

The key.pem and cert.pem files will be used by the Node.js and RSSL interactive provider application and the cert.pem file will be used by Refinitiv Real-time SDK consumer applications to connect to this simulated Real-Time Optimized environment.

The key.pem and cert.pem files used by this article are also available on GitHub. The description of the cert.pem file on GitHub is:

    	
            

Common Name : www.test.com

Valid From : Sep 30,2022

Valid To : Sep 27,2032

Serial Number : 18138038264291739402

2.     Run a Node.js application

We used a Node.js application with the Express framework to simulate REST services on the Refinitiv Data Platform. The example code is available on GitHub.

First, the application uses the private key (key.pem) and certificate (cert.pem) files created in the first step to start the HTTPS server on TCP port 433. 

    	
            

const https = require('https');

const fs = require('fs');

const bodyParser = require('body-parser');

const listen_port = 443

 

const app = express();

app.use(bodyParser.urlencoded({ extended: true }));

 

https.createServer(

        {

            key: fs.readFileSync("key.pem"),

            cert: fs.readFileSync("cert.pem"),

        },

        app

    )

    .listen(listen_port, () => {

      

        console.log(timestamp()+"serever is running at port "+ listen_port);

    });

You can change to other TCP ports.

Next, the application simulates the following Refinitiv Data Platform services. 

·        RDP Authentication Service V1

The endpoint of this service is auth/oauth2/v1/token. It is used to get and renew tokens. The application receives HTTP Post requests from this endpoint and then sends successful or failed responses. 

    	
            

const success_token = `{

            "access_token": "eyJ0eXAiOiJhdCtqd3QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImJlcGpHV0dkOW44WU9VQ1Nw",

                "refresh_token": "aa957c85-f1f2-453d-8211-adbbb30b8428",

                "expires_in": "600",

                "scope": "trapi.data.pricing.read trapi.streaming.pricing.read",

                "token_type": "Bearer"

        }`

 

const bad_request_token = '{"error": "invalid_grant"}'

 

 

app.post('/auth/oauth2/v1/token', (req, res) => {

   

    console.log(timestamp()+req.url)   

    console.log("grant_type: "+req.body.grant_type)

    console.log(req.body)

 

    if (req.body.grant_type == 'password') {

        res.send(success_token)

 

    } else if (req.body.grant_type == 'refresh_token') {

        res.send(success_token)

        //console.log("Send 400 Bad Request")

        //res.status(400).send(bad_request_token)

    } else {

        console.log("Unknown grant_type")

        res.status(400).send(bad_request_token)

    }

})

We can simulate failed requests by sending the HTTP 400 Bad Request response status code with the error message in the HTTP body.

·        RDP Authentication Service V2

The endpoint of this service is auth/oauth2/v2/token. It is used to get an access token. The application receives HTTP Post requests from this endpoint and then sends successful or failed responses. 

    	
            

const success_token_v2 = `{

                   "access_token": "eyJ0eXAiOiJhdCtqd3QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImJlcGpHV0dkOW44WU9VQ1Nw",

                 "expires_in": "7199",

                "scope": "trapi.data.pricing.read trapi.streaming.pricing.read",

                 "token_type": "Bearer"

        }`

 

 

 

app.post('/auth/oauth2/v2/token', (req, res) => {

    console.log(timestamp() + req.url)   

    console.log("grant_type: " + req.body.grant_type)

    console.log(req.body)

 

    if (req.body.grant_type == 'client_credentials') {

        res.send(success_token_v2)

 

    } else {

        console.log("Unknown grant_type")

        res.status(400).send(bad_request_token)

    }

 

})

We can simulate failed requests by sending the HTTP 400 Bad Request response status code with the error message in the HTTP body.

·        Streaming Service Endpoint Information

The endpoint of this service is /streaming/pricing/v1/. It is used to get available streaming servers. The Refinitiv Real-Time SDK consumer applications will connect to one of these streaming servers to retrieve conflated real-time content. The application receives HTTP GET requests from this endpoint and then sends successful responses that contain all available streaming servers. 

    	
            

const endpoints_data = fs.readFileSync('endpoints.json', 'utf8');

 

app.get('/streaming/pricing/v1/', (req, res) => {

    console.log(timestamp()+req.url)

    res.send(endpoints_data)

})

The application loads the response from the local file (endpoints.json) and then sends it in the HTTP response. The content of the endpoints.json looks like this:

    	
            

{

  "services": [

    {

      "port": 14002,

      "location": [

        "ap-northeast-1a"

      ],

      "transport": "tcp",

      "provider": "aws",

      "endpoint": "www.test.com",

      "dataFormat": [

        "rwf"

      ]

    },

    {

      "port": 14002,

      "location": [

        "ap-northeast-1a",

        "ap-northeast-1b"

      ],

      "transport": "tcp",

      "provider": "aws",

      "endpoint": "www.test.com",

      "dataFormat": [

        "rwf"

      ]

    },

The response informs the Refinitiv Real-Time SDK consumer applications to connect to www.test.com on TCP port 14002 to retrieve conflated real-time content. You can modify the hostname in the “endpoint” property.

The source code is available on GitHub. Download the source code and run “npm install” to install dependencies.

You can start the Node.js application by running the following command.

    	
            node app.js
        
        
    

The output looks like this:

    	
            

[root@www rdpservices]# node app.js

RDP Services Simulator

[6:6:29] server is running at port 443

3.     Run an RSSL interactive provider application

We used an ETA Provider example (Cpp-C\Eta\Applications\Examples\Provider) in the Refinitiv Real-Time SDK C/C++ package to simulate a Real-Time Optimized feed.  The example generates dummy real-time data and sends it to the consumer application. 

    	
            

UpdateMsg

    streamId="5"

    domain="MarketPrice Domain"

    updateTypeNum="0"

    name="IBM.N"

    serviceId="1"

    serviceName="ELEKTRON_DD"

    Payload dataType="FieldList"

        FieldList

            FieldEntry fid="6" name="TRDPRC_1" dataType="Real" value="1.09"

            FieldEntry fid="22" name="BID" dataType="Real" value="1.08"

            FieldEntry fid="25" name="ASK" dataType="Real" value="203.12"

            FieldEntry fid="32" name="ACVOL_1" dataType="Real" value="100000.00"

            FieldEntry fid="11" name="NETCHNG_1" dataType="Real" value="203.15"

            FieldEntry fid="267" name="ASK_TIME" dataType="Time" value="08:55:54:000:000:000"

        FieldListEnd

    PayloadEnd

UpdateMsgEnd

You can modify the code to test unexpected behavior, such as sending a login closed status message.

Run the example with the following arguments. We specify the key and certificate files created in the first step in the arguments. 

    	
            ./Provider -c encrypted -s ELEKTRON_DD -keyfile key.pem -cert cert.pem -p 14002 -runtime 99999999
        
        
    

The output looks like this:

    	
            

[root@www providers]# ./Provider -c encrypted -s ELEKTRON_DD -keyfile key.pem -cert cert.pem -p 14002 -runtime 99999999

portNo: 14002

serviceName: ELEKTRON_DD

serviceId: 1

Server IPC descriptor = 3 bound on port 14002

Instead of using an ETA interactive provider example, you can run an encrypted ADS or testserver tool, as a simulated Real-time Optimized feed. Testserver is a test tool in the Refinitiv Real-Time Distribution System package. You can run the testserver tool with the following arguments. We specify the key and certificate files created in the first step in the arguments. 

    	
            ./testserver -S ELEKTRON_DD -N 14002 -K -U 1 -serverCert cert.pem -serverPrivateKey key.pem -ct rssl -rct 1
        
        
    

To set up an encrypted ADS, please refer to the Configuring Encrypted Connections on ADS article. 

In summary, the settings for connecting to this simulated Refinitiv Real-Time Optimized are:

Service URL TCP Port
RDP Authentication Service V1 www.test.com/auth/oauth2/v1/token 443
RDP Authentication Service V2 www.test.com/auth/oauth2/v2/token 443
Streaming Service Endpoint Information (Service Discovery) www.test.com/streaming/pricing/v1/ 443
Real-Time Streaming www.test.com 14002

Run EMA Examples

In this section, I will demonstrate how to configure and run EMA C++ and Java examples on a Windows machine to connect to the simulated Real-Time Optimized environment running on the same machine (localhost). 

Run the Enterprise Message API C++ Example (Cons450)

450_MP_QueryServiceDiscovery (Cons450) is an OMM Consumer application example that demonstrates basic usage of the EMA library in accessing and parsing OMM MarketPrice data from Refinitiv Real-Time - Optimized. It illustrates how to query endpoints from RDP service discovery using the ServiceEndpointDiscovery class and use the location from the command line to select an endpoint. It uses the EMA's programmatic configuration to enable session management with the retrieved endpoint for establishing a connection with the Refinitiv Real-Time service and consuming data.

Please follow these steps to run the Cons450 example to connect to the simulated Real-Time Optimized environment. 

1.      Setup the Windows host file

The key and certificate files used by the node.js and RSSL interactive provider are created by using www.test.com as a common name and those applications are running on the same Windows machine as the Cons113 example. Therefore, to connect to those applications, we need to add the following entry into the Windows host file (C:\Windows\System32\drivers\etc\hosts) to resolve www.test.com to 127.0.0.1. 

    	
            127.0.0.1 www.test.com
        
        
    

Note: The administrator privilege is required to add a new entry into the Windows host file.

2.      Import a certificate file

To run the example, the certificate file (cert.pem) must be imported to the Windows Trusted Root Certification Authorities.

o   Run certmgr to open the Manage Computer Certificates window

o   Right click Trusted Root Certification Authorities to import the certificate file (cert.pem) 

o   Select the cert.pem file.

You can check that the certificate file is added in the Trusted Root Certification Authorities. 

Note: The administrator privilege is required to add a new entry into the Windows host file.

3.      Run the example

Run the Cons450 example with the following parameters.

o   Use RDP Authentication Service V1

    	
            -username <machine id> -password <password> -clientId <client id> -itemName IBM.N -tokenURLV1 https://www.test.com/auth/oauth2/v1/token -serviceDiscoveryURL https://www.test.com/streaming/pricing/v1/
        
        
    

o   Use RDP Authentication Service V2

    	
            -clientId <client id> -clientSecret <client secret>  -itemName IBM.N -tokenURLV2 https://www.test.com/auth/oauth2/v2/token -serviceDiscoveryURL https://www.test.com/streaming/pricing/v1/
        
        
    

The output looks like this:

    	
            

Services :

    Service :

        Provider : aws

        Transport : tcp

        Endpoint : www.test.com

        Port : 14002

        Data Format : rwf

        Location : ap-northeast-1a

    Service :

        Provider : aws

        Transport : tcp

        Endpoint : www.test.com

        Port : 14002

        Data Format : rwf

        Location : ap-northeast-1a ap-northeast-1b

    Service :

        Provider : aws

        Transport : tcp

        Endpoint : www.test.com

        Port : 14002

        Data Format : rwf

        Location : ap-northeast-1b

RefreshMsg

    streamId="5"

    domain="MarketPrice Domain"

    Solicited

    RefreshComplete

    ClearCache

    state="Open / Ok / None / 'Item Refresh Completed'"

    itemGroup="00 00"

    qos="RealTime/TickByTick"

    name="IBM.N"

    nameType="1"

    serviceId="1"

    serviceName="ELEKTRON_DD"

    Payload dataType="FieldList"

        FieldList

            FieldEntry fid="2" name="RDNDISPLAY" dataType="UInt" value="100"

            FieldEntry fid="4" name="RDN_EXCHID" dataType="Enum" value="155"

            FieldEntry fid="38" name="DIVPAYDATE" dataType="Date" value="22 OCT 2010"

            FieldEntry fid="6" name="TRDPRC_1" dataType="Real" value="1.00"

            FieldEntry fid="22" name="BID" dataType="Real" value="0.99"

            FieldEntry fid="25" name="ASK" dataType="Real" value="203.03"

            FieldEntry fid="32" name="ACVOL_1" dataType="Real" value="100000.00"

            FieldEntry fid="11" name="NETCHNG_1" dataType="Real" value="203.15"

            FieldEntry fid="267" name="ASK_TIME" dataType="Time" value="15:55:38:000:000:000"

            FieldEntry fid="14305" name="EVENT_T_NS" dataType="Time" value="12:14:07:966:111:999"

        FieldListEnd

    PayloadEnd

RefreshMsgEnd

UpdateMsg

    streamId="5"

    domain="MarketPrice Domain"

    updateTypeNum="0"

    name="IBM.N"

    serviceId="1"

    serviceName="ELEKTRON_DD"

    Payload dataType="FieldList"

        FieldList

            FieldEntry fid="6" name="TRDPRC_1" dataType="Real" value="1.01"

            FieldEntry fid="22" name="BID" dataType="Real" value="1.00"

            FieldEntry fid="25" name="ASK" dataType="Real" value="203.04"

            FieldEntry fid="32" name="ACVOL_1" dataType="Real" value="100000.00"

            FieldEntry fid="11" name="NETCHNG_1" dataType="Real" value="203.15"

            FieldEntry fid="267" name="ASK_TIME" dataType="Time" value="15:55:39:000:000:000"

            FieldEntry fid="14305" name="EVENT_T_NS" dataType="Time" value="12:14:07:966:111:999"

        FieldListEnd

    PayloadEnd

UpdateMsgEnd

Run the Enterprise Message API Java Example (ex450_MP_QueryServiceDiscovery)

ex450_MP_QueryServiceDiscovery is an OMM Consumer application example that demonstrates basic usage of the EMA library in accessing and parsing OMM MarketPrice data from Refinitiv Real-Time - Optimized. It illustrates how to query endpoints from Refinitiv Data Platform service discovery using the ServiceEndpointDiscovery class and use the location from the command line to select an endpoint. It uses the EMA's programmatic configuration to enable session management with the retrieved endpoint for establishing a connection with the Refinitiv Real-Time - Optimized service and consuming data.

Please follow these steps to run the ex450_MP_QueryServiceDiscovery example to connect to the simulated Real-Time Optimized environment.

1.      Setup the Windows host file

The key and certificate files used by the node.js and RSSL interactive provider are created by using www.test.com as a common name and those applications are running on the same Windows machine as the ex450_MP_QueryServiceDiscovery example. Therefore, to connect to those applications, we need to add the following entry into the Windows host file (C:\Windows\System32\drivers\etc\hosts) to resolve www.test.com to 127.0.0.1. 

    	
            127.0.0.1 www.test.com
        
        
    

Note: The administrator privilege is required to add a new entry into the Windows host file.

2.      Import a certificate file

To run the example, the certificate file (cert.pem) must be imported to the Java trust store file. For example, the Java trust store file for JDK 11 is at C:\Program Files\Java\jdk-11.0.11\lib\security\cacerts.

o   Copy the certificate file (cert.pem) to C:\Program Files\Java\jdk-11.0.11\lib\security\

o   Run a command prompt as administrator and change the directory to C:\Program Files\Java\jdk-11.0.11\lib\security\

o   Run the following command to import the certificate file (cert.pem) to the Java trust store file. “changeit” is the default password of the Java trust store file. 

    	
            keytool -import -alias www.test.com -keystore cacerts -file cert.pem -storepass changeit
        
        
    

You can run the following command to see the imported certificate file.

    	
            keytool -list -alias www.test.com -keystore cacerts -storepass changeit
        
        
    

You can delete it by using the following command.

    	
            keytool -delete -alias www.test.com -keystore cacerts -storepass changeit
        
        
    

3.     Run the example

Run the ex450_MP_QueryServiceDiscovery.Consumer example with the following parameters.

o   Use RDP Authentication Service V1

    	
            -username <machind id> -password <password> -clientId <client id> -itemName IBM.N -tokenURLV1 https://www.test.com/auth/oauth2/v1/token -serviceDiscoveryURL https://www.test.com/streaming/pricing/v1/
        
        
    

o   Use RDP Authentication Service V2

    	
            -clientId <client id> -clientSecret <client secret> -itemName IBM.N -tokenURLV2 https://www.test.com/auth/oauth2/v2/token -serviceDiscoveryURL https://www.test.com/streaming/pricing/v1/ 
        
        
    

The output looks like this:

    	
            

Services :

    Service :

        Provider : aws

        Transport : tcp

        Endpoint : www.test.com

        Port : 14002

        Data Format : rwf 

        Location : ap-northeast-1a 

    Service :

        Provider : aws

        Transport : tcp

        Endpoint : www.test.com

        Port : 14002

        Data Format : rwf 

        Location : ap-northeast-1a  ap-northeast-1b 

    … 

   

Oct 21, 2022 12:52:43 PM com.refinitiv.ema.access.ChannelCallbackClient reactorChannelEventCallback

INFO: loggerMsg

    ClientName: ChannelCallbackClient

    Severity: Info

    Text:    Received ChannelUp event on channel Channel_1

       Instance Name Consumer_1_1

       Component Version eta3.6.6.L1.win.rrg 64-bit Static

loggerMsgEnd

 

 

RefreshMsg

    streamId="5"

    domain="MarketPrice Domain"

    solicited

    RefreshComplete

    state="Open / Ok / None / 'Item Refresh Completed'"

    itemGroup="00 00"

    name="PTT.BK"

    nameType="1"

    serviceId="1"

    serviceName="ELEKTRON_DD"

    Payload dataType="FieldList"

        FieldList

            FieldEntry fid="2" name="RDNDISPLAY" dataType="UInt" value="100"

            FieldEntry fid="4" name="RDN_EXCHID" dataType="Enum" value="155"

            FieldEntry fid="38" name="DIVPAYDATE" dataType="Date" value="22 OCT 2010"

            FieldEntry fid="6" name="TRDPRC_1" dataType="Real" value="1.00"

            FieldEntry fid="22" name="BID" dataType="Real" value="0.99"

            FieldEntry fid="25" name="ASK" dataType="Real" value="203.03"

            FieldEntry fid="32" name="ACVOL_1" dataType="Real" value="100000.00"

            FieldEntry fid="11" name="NETCHNG_1" dataType="Real" value="203.15"

            FieldEntry fid="267" name="ASK_TIME" dataType="Time" value="12:52:43:000:000:000"

            FieldEntry fid="14305" name="EVENT_T_NS" dataType="Time" value="12:14:07:966:111:999"

        FieldListEnd

    PayloadEnd

RefreshMsgEnd

 

UpdateMsg

    streamId="5"

    domain="MarketPrice Domain"

    updateTypeNum="0"

    name="PTT.BK"

    serviceId="1"

    serviceName="ELEKTRON_DD"

    Payload dataType="FieldList"

        FieldList

            FieldEntry fid="6" name="TRDPRC_1" dataType="Real" value="1.01"

            FieldEntry fid="22" name="BID" dataType="Real" value="1.00"

            FieldEntry fid="25" name="ASK" dataType="Real" value="203.04"

            FieldEntry fid="32" name="ACVOL_1" dataType="Real" value="100000.00"

            FieldEntry fid="11" name="NETCHNG_1" dataType="Real" value="203.15"

            FieldEntry fid="267" name="ASK_TIME" dataType="Time" value="12:52:44:000:000:000"

            FieldEntry fid="14305" name="EVENT_T_NS" dataType="Time" value="12:14:07:966:111:999"

        FieldListEnd

    PayloadEnd

UpdateMsgEnd

 

Summary

This article demonstrates how to set up a simulated Real-Time Optimized to test Refinitiv Real-Time SDK’s applications. It depends on the OpenSSL toolkit, node.js, and RSSL interactive provider. The first step is running the OpenSSL command to generate a certificate file and a private key file. These files are used to run the node.js and RSSL interactive provider applications.  The next step is running the node.js application to handle REST services on the Refinitiv Data Platform including the RDP authentication service and streaming service endpoint information. The last step is running an RSSL interactive provider to provide real-time financial data. An RSSL interactive provider could be an ETA interactive provider, encrypted ADS, or testsever tool. Finally, it shows how to configure and run EMA C++ and Java examples on a Windows machine to connect to this simulated Real-Time Optimized environment. 

References

1.      Refinitiv Data Platform APIs (no date) Refinitiv Data Platform APIs | Refinitiv Developers. Available at: https://developers.refinitiv.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-platform-apis (Accessed: October 21, 2022).

2.      Refinitiv real-time C++ SDK (no date) Refinitiv Real-Time C++ SDK | Refinitiv Developers. Available at: https://developers.refinitiv.com/en/api-catalog/refinitiv-real-time-opnsrc/rt-sdk-cc (Accessed: October 21, 2022).

3.      Refinitiv real-time Java SDK (no date) Refinitiv Real-Time Java SDK | Refinitiv Developers. Available at: https://developers.refinitiv.com/en/api-catalog/refinitiv-real-time-opnsrc/rt-sdk-java (Accessed: October 21, 2022).