Elektron SDK - C/C++

API Family: Elektron

EMA NI Provider - Publishing our first Market Price

Download tutorial source code

Click here to download

Last update June 2018
Compilers

Visual Studio 2015

For other compilers settings, please check out the EMA C++ Compiler Settings tutorial.

Prerequisites EMA NI Provider - Introducing the EMA configuration file

Declare the NI_PUB and TEST_NI_PUB services in your TREP (see Before you start).

Tutorial purpose

The aim of this tutorial is to explain the concepts related to data item publication, and to teach you how to use EMA to publish a refresh message for a market price instrument.

To this aim we will go through the following sections:

Introduction

Now that we know how to properly connect a NI Provider application to a Refinitiv real-time distribution system, it is time to get to the meat and actually publish data for the consuming applications. In this tutorial you will learn how to use the EMA library to encode and publish data for a MarketPrice item.

Background information about Refinitiv real-time infrastructures

The aim of this section is to give you a brief overview of the Refinitiv real-time infrastructures, concepts and workflows. For further details please consult the Elektron API Concepts Guide and the EMA C++ RDM Usage Guide in References.

Distribution systems

Refinitiv real-time infrastructures like TREP or Elektron are efficient and reliable distribution systems that enable the exchange of data items between Provider and Consumer applications. Before you start learning how to publish data, it is important you become familiar with the concepts described below.

Services

The system distributes data items via services. Each service provides a universe of data items (a.k.a instruments) and domain models that are served by provider applications. Several providers can serve the same service and a provider can serve several services.

A service is said Up when at least one of its providers is running and connected to the system. If no provider is available, the service is said Down and none of its data items can be consumed.  

Different services may deliver different data and different domain models for the same item. For example, on The Refinitiv Enterprise Platform IDN_RDF may provide market price level1 data for MSFT.O while DF_NAS_L2 provides market by order level 2 data for the same item.

Domain Models

Services and providers deliver data items for one or several domain models (a.k.a capabilities). Some of these models are defined by Refinitiv. They are then called Reuters Domain Models (RDM) - Market price, order book and yield curve are well known Reuters Domain Models. Other domain models can be defined by users. They are then called User Domain Models (UDM).

Data items

Data items distributed on the system are identified by symbols. Refinitiv publishers use the RIC symbol type (Reuters Instrument Code) to identify the data items they publish. However, as the Elektron SDK is symbol type agnostic, Provider may use other types of symbols like ISIN, CUSIP or SEDOL.

A Data item may be distributed by several services and provided for different domain models. For this reason, a symbol alone is not enough to identify a data item. It must be completed by the service name and the domain model.

These three elements (Service / Symbol / Domain Model) uniquely identify the data items distributed by the infrastructure. In this tutorial we will publish Market Price data for the RIC SHARE-0 via the service NI-PUB.  

Note: The naming of the RIC with the  -0 (zero) suffix is intentional as in later tutorials we will demonstrate multiple items to publish, i.e. SHARE-0, SHARE-1, etc.

Non Interactive Providers

NI Provider applications publish data items that are cached by the infrastructure, and then distributed to the requesting consumers. Ni Providers can publish several data items of several domain models for one or several services. For this reason, they must indicate the service, the RIC and the domain model of the item they publish.

Consumers

When a consuming application requests a data item, it must specify the three elements described above (Service, RIC, Domain Model). In return, and if the item is available, a refresh message (a.k.a image) is sent back by the system. In the case of market price data items, the refresh message contains the state and the complete data (all the fields) of the item. If the request indicated an interest for streaming data, the consumer may also receive subsequent updates and statuses. 

Provider / Consumer message flow

The diagram below illustrates an example of a message flow between a consumer that requested a streaming MarketPrice data item published by a NI Provider. This provider is connected to a remote source system that provides the actual data.

  1. At start-up, the NI Provider connects to its source system and publishes the initial Refresh message for the item. This message contains all the item's fields. This data is cached by the TREP but not forwarded to anyone as no consumer application registered an interest for this item yet.
  2. Because the source system changed the item data, the NI Provider publishes an Update message that contains the fields that changed. The data cached by the TREP is then updated with the new fields' values.
  3. A consumer application that is interested by this item sends a streaming request for it. The item is identified by the Service, the RIC and the Domain Model.
  4. The TREP sends back a Refresh message based on the data it cached for the item.
  5. Because the source data changed again, the provider sends another Update message. The TREP updates its internal cache.
  6. As a consumer application is interested by this data item, the TREP forwards the update to this consumer.
  7. Because it detected a connection failure with its source system, the provider sends a Status message indicating that the data is suspect. Then it starts its failover process. The TREP updates its internal cache with the new status.
  8. The TREP forwards the suspect status to the consumer.
  9. After a while the provider recovers the connection to its source system. Then, it publishes a Refresh message. This message contains all the item's fields and a status that indicates the data is Ok. The TREP uses this message to refresh its internal cache.

    Note: Sending an unsolicited Refresh message after a connection recovery (either with the TREP or with the source of data) is a good practice that providers are expected to follow. This is an excellent way to ensure the TREP data cache is in sync with your data source.
  10. The TREP forwards the Refresh message to the consumer.
  11. Because the source data changed, the provider sends an Update message. The TREP updates its internal cache.
  12. The TREP forwards the update to the consumer.

The MarketPrice Reuters Domain Model

The MarketPrice domain allows to publish Level I market information such as trades, indicative quotes, and top-of-book quotes. All information is sent as a list of field/value pairs that contain the information related to the data item (i.e. net change, bid, ask, volume, high, low, or last price). Each field is identified by a unique id and has an associated value of a predefined data type. The exhaustive list of fields with their ids, names and data types are defined by a field dictionary held by the infrastructure. 

The diagram below illustrates the structure of a Market Price message. The FieldList conveys several field entries. Each FieldEntry holds a FieldId (a.k.a FID) and the current value of the related field. Details like the field name and the value type are defined in the TREP's field dictionary. In this example, the first FieldEntry is for the DSPLY_NAME field and contains a 16 characters long string.

Notes:

  • If your provider application requires field dictionaries (because it relies on field names instead of field ids, for example) you must ask your TREP administrator to provide you with the dictionaries used by your TREP. They are provided as text files that your application will have to parse in order to build its in-memory representation of the field dictionaries. You will have to take care that the files used by your application are always in sync with the dictionaries used by your TREP. 
  • Examples of field dictionaries can be found in the Elektron SDK package (e.g. Ema\Etc\RDMFieldDictionary) or in the EMA C++ GitHub repository.

For MarketPrice data items, NI Providers can publish Refresh messages, Update messages and Status messages.

For more details about the MarketPrice model and the related messages, please consult the Elektron API Concepts Guide and the EMA C++ RDM Usage Guide in References.

The Refresh message

For our first publication we will send a Refresh message for the SHARE-0 market price item on the NI_PUB service. In order to tell the infrastructure what data item this message is published for, we will use the following information:

  • Service name: “NI_PUB”. This is the default service name used by EMA for NI Providers. We will see in the next tutorial how to configure EMA in order to use another service name. But for now we will stick with this one. If you try another service name you will get an exception error when you try to send the message.
  • RIC: “SHARE-0”. This is an arbitrary name that we chose for this tutorial. You can change this name if you wish.
  • Domain Type: MMT_MARKET_PRICE. This is the domain type to use for market price data.

The message will also transport the item state, and the item payload made of a field list.

To create a refresh message using EMA, you just have to instantiate a RefreshMsg object and initialize it by calling its different methods. Then, when you finished setting the different parts of the message, you must call the complete() method to indicate that the message is complete. For example you could do something like:

    RefreshMsg refreshMsg;

    refreshMsg.serviceName("NI_PUB");   
    refreshMsg.name("SHARE-0");
    refreshMsg.domainType(MMT_MARKET_PRICE);
        .
        .
        .
    refreshMsg.complete();

The State

The state of the item is also set in the refresh message. This is done by calling the state() method like this:

    refreshMsg.state(OmmState::OpenEnum, OmmState::OkEnum, OmmState::NoneEnum, "UnSolicited Refresh Completed");

This method gets 4 parameters:

  • The stream state (e.g. Opened, Closed...).
  • The data item state (e.g. Ok, suspect…).
  • The status code (e.g. Not Found, Not Authorized, Timeout…).
  • The status explanation.

Please consult the EMA C++ Reference Manual and the RDM Usage Guide in References for more details about the market price state.

The Payload

The refresh message will transport a payload made of the following field list:

Name Fid Type Value
DSPLY_NAME 3 String "My Company Share"
BID 22 Real 14.400
ASK 25 Real 14.700
OPEN_PRC 19 Real 14.100
HST_CLOSE 21 Real 14.900

The payload is simply added to the refresh message by calling the payload() method. But before you do that you must build the market price payload itself. To this aim, you must instantiate a FieldList object and initialize it by adding the fields and their values one by one. The FieldList class provides one method per field type. For example, to add a string field you must call the addAscii() method, to add a real field you must call the addReal() method, etc. Then, when the field list is complete, you must call the complete() method. Here is how you could proceed for the field list described above:

    FieldList fieldList;

    fieldList.addAscii(3, "My Company Share");                  // FID 3  => DSPLY_NAME
    fieldList.addReal(22, 14400, OmmReal::ExponentNeg2Enum);    // FID 22 => BID
    fieldList.addReal(25, 14700, OmmReal::ExponentNeg2Enum);    // FID 25 => ASK
    fieldList.addReal(19, 14100, OmmReal::ExponentNeg2Enum);    // FID 19 => OPEN_PRC
    fieldList.addReal(21, 14900, OmmReal::ExponentNeg2Enum);    // FID 21 => HST_CLOSE
    fieldList.complete();

    refreshMsg.payload(fieldList);

The FieldList class provides a number of addXXX() methods that require different parameters. Please consult the EMA C++ Reference Manual (in References) for an exhaustive list and description of these parameters.

Sending the Refresh message

Once the refresh message is built and completed, it is time to publish it. This is done by calling the submit() method of the provider like this:

    UInt64 itemHandle = 1;

    provider.submit(refreshMsg, itemHandle);

The first parameter is the refresh message to publish. The second parameter is a handle that uniquely identifies the item stream. This number is chosen by the provider application and will be used for every subsequent message sent for this item stream. Let us say you want to publish an update for the same data item, then you must submit the update message using the exact same handle (1 in this case).

Putting it all together in the NiProvider

Now that we saw how to build the payload, the state and how to send a refresh message for a data item, we will put all this together in a new refresh() method of the NiProvider class.

void NiProvider::refresh(const std::string & serviceName, const std::string & itemName)
{
    if (!isConnected())
    {
        cerr << "  ERROR: Can't refresh " << itemName << ". The provider is not connected." << endl;
        return;
    }

    UInt64 itemHandle = 1;

    cout << "  Refreshing " << itemName << endl;

    _provider->submit(
                    RefreshMsg()
                        .serviceName(serviceName.c_str())
                        .name(itemName.c_str())
                        .domainType(MMT_MARKET_PRICE)
                        .state(
                            OmmState::OpenEnum, 
                            OmmState::OkEnum, 
                            OmmState::NoneEnum, 
                            "UnSolicited Refresh Completed")
                        .payload(FieldList()
                            .addAscii(3, "My Company Share")        // DSPLY_NAME
                            .addReal(22, 14400, OmmReal::ExponentNeg2Enum)  // BID
                            .addReal(25, 14700, OmmReal::ExponentNeg2Enum)  // ASK
                            .addReal(19, 14100, OmmReal::ExponentNeg2Enum)  // OPEN_PRC
                            .addReal(21, 14900, OmmReal::ExponentNeg2Enum)  // HST_CLOSE
                            .complete())
                        .complete(), 
                    itemHandle);
}

This method takes two arguments, the service name and the item name which is the actual RIC. It is a public method of the NiProvider class, that we will call in our main workflow.

First of all, we test if the provider is connected and exit the method if it's not.

Then, we build and publish the message. As you can see, we leveraged the chained method call facility provided by EMA. This allows us to write more concise and readable code. The code is indented in a way that you can easily tell what methods are called on what object. For example, from the indentation you can tell that the serviceName(), name(), domainType(), state(), payload() and the second complete() methods have been called on the RefreshMsg object that we created on the stack. From the indentation, you can also tell that the submit() method has two parameters: the RefreshMsg and the itemHandle.

Also to be mentioned:

  • The item handle and fields values are hardcoded. That is ok for now but will become an issue when we publish multiple items in later tutorials.
  • The fields are built using the numerical field ids. This is not easy to read and not good programming practice.

We chose to implement this tutorial this way for the sake of simplicity, and because we are publishing only one data item. We will improve this code in the next tutorials.

The main workflow

The main workflow leverages the new refresh() method of the NiProvider class to implement the following:

  • It creates the provider and connects it to the ADH
  • It waits for 5 seconds to make sure the provider is actually connected to the ADH before sending the Refresh message.

    Note: This wait is a bit disproportionate as the provider generally connects almost instantly to the ADH. If for any reason the refresh() method was called before the connection, it would detect it and print an Error message on the console (see the method details for more).   
  • It uses the refresh() method to send a Refresh message for the NI_PUB/SHARE-0 market price item. 
  • Once the message is published, the application waits for 60 seconds before exiting, so that you have time to subscribe to the item using the consumer application of your choice.
int main(int argc, char* argv[])
{
    .
    .
    .
        NiProvider provider;

        provider.connectAs("YOUR_PROVIDER_USER_NAME");

        waitFor(5);

        provider.refresh("NI_PUB", "SHARE-0");

        waitFor(60);

        provider.disconnect();
    .
    .
    .

}

Build and run the application

Build the application and start it. Please refer to the Build and Run section within the first tutorial of this series (A barebones EMA NIP application shell) for detailed instructions.

This is what you should get:

  1. A console application should open and display something like:
-------------------------------------------------------------------------------
|                    Non Interactive Provider EMA Tutorial                    |
|                                                                             |
|                Tutorial 5 - Publishing our first Market Price               |
-------------------------------------------------------------------------------
  Provider created
  Connecting Provider to ADH 10.2.43.49:14003 as nip-user
  Waiting for 5 seconds...
  Provider is connected. OmmState:Open / Ok / None / 'Refresh Completed'
  Refreshing SHARE-0
  Waiting for 60 seconds...
  Disconnecting...
  Provider is disconnected. OmmState:Open / Suspect / None / 'channel down'
  Exiting the application
Press any key to continue . . .
  1. During the 60 seconds wait, open a TREP consuming application and subscribe to the SHARE-0 market price item of the NI_PUB service. After a short while, you should receive values for the 5 fields (DSPLY_NAME/3), OPEN_PRC/19), HST_CLOSE/21), BID/22) and ASK/25) published by the NI Provider.

    As an example, this is a screenshot of the Eikon Quote object that we used to subscribe to NI_PUB/SHARE-0. In the Eikon configuration, the NI_PUB service has been associated to the 'N' key letter:

  2. After 60 seconds, the NI_PUB service goes down and the application exits.
  3. Press a key to close the console when you are prompted to.

Troubleshooting

Q: When I build the tutorial project, I get errors like:

error MSB3073: The command "copy "\Ema\Libs\WIN_64_VS140\Debug_MDd\Shared\*.*" .\Debug_WIN_64_VS140_Shared\

A: The ElektronSDKInstallPath environment variable is not set or set to the wrong path. See Setup the development environment.

 

Q: The application is stuck after the "Connecting Provider to ADH…" message is displayed.

After a while the application displays an error like: 

Exception Type='OmmInvalidUsageException', 
          Text='login failed (timed out after waiting 45000 milliseconds) for 10.2.43.149:14003)'

A: Verify that the ADH of your TREP infrastructure is up and that you properly set the host parameter in the EmaConfig.xml file. 

You can also use the telnet command tool to verify that your NIP application machine can connect to the ADH (telnet <ADH host> <port>). If the telnet succeeds but you still can’t connect, verify that you don’t have any firewall blocking the messages sent/received by the application.  

Ultimately, ask your TREP administrator to help you to investigate with TREP monitoring tools like adhmon.

 

Q: The “Waiting for 5 seconds...” message and the “Provider is connected” message get mixed up when displayed in the console.

A: This is perfectly normal and actually caused by the EMA background thread that prints the “Provider is connected” message at the same time the main application thread prints the “Waiting for 30 seconds...” message. This can be fixed either by choosing a mono threaded application model (see the Message Processing - dispatch section of the Requesting MarketPrice data EMA Consumer tutorial) or by printing these messages atomically (use critical sections or a single cout << call to print the whole message).

Tutorial Group: 
EMA NI Provider