Article

Discover our Refinitiv Data Platform Library (part 1)

December 05,2019

Umer Nalla

UPDATE as of JAN 2022:

Statement from the RDP Library team:

Refinitiv Data Platform Libraries (RDP libraries) have been redesigned and renamed to Refinitiv Data Libraries (RD libraries). Alpha versions of former RDP libraries will be maintained for a while, but for new development, it is recommended to use the latest RD libraries which incorporate new features and offer greater usability. Please refer to the following pages for more about the new RD libraries: RD library for Python, RD Library for TypeScript (Q1 2022), RD Library for .NET.

Original article:

As part of our 2019 Refinitiv Developer Days Series, I spent a few months travelling around Europe giving members of our developer community a sneak peek at our exciting new Refinitiv Data Platform Library.

The feedback was so positive, I felt it was only right to share this with our wider community - showcasing some of the key features which will make the Refinitiv Data Platform Library such a useful addition to our current set of APIs.

So, what is the Refinitiv Data Platform Library?

Before we begin, I need to point out that the Refinitiv Data Platform Library is in Alpha mode and was released as part of an Early Access Programme in the first weeks of January 2020.

The latest alpha version of the Python library can be obtained on PyPi - install by executing 'pip install refinitiv-dataplatform' - in your Python environment.

Likewise, the .NET version can be found on NuGet - base package and Content Layer - with installation instructions on the relevant pages.

You can also find sample code on GitHub in .NET and Python flavours.

Refinitiv Data Platform Libraries - Raison D'être

Whilst Refinitiv offers huge breadth and depth of data content, it often involves multiple technologies, delivery mechanisms and data formats - and therefore the use of multiple APIs to access this content.

We recognise this is not ideal and we are gradually moving as much of our content as possible onto our new Refinitiv Data Platform.

If we are going to place the content in one place, it would be ideal if a single library could be used to access that content too.

And with the rise of Citizen Developers - those whose primary role is not Computer Science related - access to the data should be as simple and easy as possible.

Therefore, the Refinitiv Data Platform Library has been designed with the following objectives in mind:

Simplify integration into all delivery platforms - Desktop, Deployed or Cloud
Provide a consistent API experience - learn once and apply across the wealth of Refinitiv content
Reach the largest audience of developers possible from Citizen developers to seasoned professionals

So, how does the Refinitiv Data Platform library deliver on those objectives?

One Library - Three Access Points

The Refinitiv Data Platform library will let consume data from the following platforms

Desktop - Eikon and the new Refinitiv Workspace
Cloud-based - Refinitiv Data Platform
Deployed - deployed or managed Refinitiv Real-Time Distribution System (RTDS formerly known as TREP)

Using the library you can access content from all 3 of the access points - all from within the same application if required. One example would be sourcing realtime streaming Pricing data from your RTDS server as well as say historical pricing events from the cloud-based Refinitiv Data platform.

Another unique advantage is that you could develop your application against one platform and then redeploy against a different platform as and when required. For example, you could develop an application consuming real-time streaming data from your internal RTDS and then switch that to the Realtime in Cloud service in the future.

OR, you could develop a Proof of Concept application against your Eikon Desktop and then potentially roll that out to other users within the organisation by redeploying against the Cloud. Changing an application to switch access point simply involves changing the session type, so instead of creating say a Desktop session, you would create a Cloud Platform session - see later for an example.

One Library - Four Delivery Modes

Currently, the vast breadth and depth of data we provide often requires accessing multiple services with different delivery modes and therefore learning and coding with multiple APIs

Using this single library, you will be able to consume data across those different delivery modes:

Request Response - Historical pricing event & summary data, news, fundamentals, analytics, research, ESG etc.
Streaming – Level 1 Trade & Quote data, Level 2 Order book data, internally published custom domains
Bulk File – End of Day reports
Queues – Alerts using the Amazon SQS Queue mechanism

One Library - Three Abstraction Layers

In order to cater for all developer types, the library offers 3 abstraction layers from the easiest to the slightly(!) more complex.

The three layers are as follows:

Function: Highest level - single function call to get data
Content: High level - Fuller response and Level 1 Streaming data too
Delivery: For data sets not supported by Function or Content Layer

Let us start with the easiest to use - the Function layer.

Function Layer

For the easiest access to the most commonly used data content, the Function layer is the place to start.

Firstly we need to connect to our data source/access point. At present most of the content offered by the Function layer is available from Eikon (desktop) or the Refinitiv Data Platform (cloud):

    	
                rdp.open_desktop_session(APP_KEY)
OR
    rdp.open_platform_session(
        APP_KEY, rdp.GrantPassword( username = RDP_LOGIN, password = RDP_PASSWORD )
    )

Python code to connect to Eikon(desktop) or Refinitiv Data Platform(cloud)

Once connected to the desired access point, we can go ahead and request some content. One content type that is currently available via the Function layer is Historical Pricing data - both Pricing Events and Pricing Summaries.

Pricing Events Data

To request Pricing Event data we use the following function:

    	
            rdp.get_historical_price_events(
    universe = 'VOD.L',
    start = datetime.timedelta(-1),
    end = datetime.timedelta(0),
    adjustments = [
        rdp.Adjustments.EXCHANGE_CORRECTION,
        rdp.Adjustments.MANUAL_CORRECTION
    ]
)

In the above snippet, we are requesting Pricing Events for the past 24hrs and using the optional adjustments parameter to apply any exchange and analyst supplied corrections.

The function returns a Dataframe:

In the above example, the Pricing Events data-set consists of 29 fields of data. We can request a subset of fields if required by specifying a fields list in the request - see the next function for an example of this.

Interday + Intraday Summary data

We can also get Pricing Interday + Intraday Summary data using the Function layer.

    	
            rdp.get_historical_price_summaries(
    universe = 'VOD.L',
    interval = rdp.Intervals.DAILY, # Supported intervals: DAILY, WEEKLY, MONTHLY, QUARTERLY, YEARLY.
    count = 20,
    fields = ['BID','ASK','OPEN_PRC','HIGH_1','LOW_1','TRDPRC_1','NUM_MOVES','TRNOVR_UNS']
)

Here we are requesting daily Interday for the past 20 days and optionally restricting the response to a subset of 8 required fields.

We can also use the same function to get Intraday data by changing the interval, so for example 1 minute Price summaries:

    	
            rdp.get_historical_price_summaries(
    universe = 'IBM.N',
    interval = rdp.Intervals.ONE_MINUTE, # Supported intervals: ONE_MINUTE, FIVE_MINUTES, 
    TEN_MINUTES, THIRTY_MINUTES, ONE_HOUR
    count = 500,
    sessions = [
        rdp.MarketSession.PRE,
        rdp.MarketSession.NORMAL,
        rdp.MarketSession.POST
    ]
)

Here we also used the optional sessions parameters to limit the response to data available during those session types.

As we did not specify a fields list, the complete set of 24 columns was returned

News Headlines and Stories

As well as Historical pricing data, the Function layer can also be used to snapshot News Headlines and the associated News Stories.

For example, we can request the 50 most recent headlines related to London using the following function call.

    	
            rdp.get_news_headlines(query = 'London', count = 50)

And this returns a Dataframe - sample extract below:

and if we want to read the full story behind any of the headlines, we can request this using the associated storyId:

    	
            rdp.get_news_story('urn:newsml:reuters.com:20191204:nNDLcdfrBN:1')

And this returns an HTML story text:

Market Price Snapshots

You can also request non-streaming Level 1 Market Price data i.e. snapshots using the Function layer. So, executing the following function call:

    	
            rdp.get_snapshot(
    universe = ['GBP=','JPY='],
    fields = ['BID','ASK'] )

will return a Dataframe like the following:

Language Versions

At launch, the Refinitiv Data Platform library will be available in the following flavours:

Refinitiv Supported Editions:
- Python
- TypeScript/JavaScript
Community Edition:
- C#

You will have noted in the above Function layer code snippets I only showed Python-based examples. This is because we believe the function layer makes more sense with scripting languages and is therefore excluded from the C# and Typescript edition. As you will see from the next section - the Content Layer requires very little more effort to get the same data.

Now that we have explored the Function layer, let's move onto the Content layer.

Content Layer

The Content Layer allows you to access the same content as above albeit is a more flexible manner:

Richer / fuller response e.g. metadata, sentiment scores - where available
Using Asynchronous or Event-Driven operating modes - in addition to Synchronous
Streaming Level 1 Market Price Data - as well as Snapshot requests

As earlier, we need to create a session before we can request data:

    	
            # Eikon/Workspace
    session =  DesktopSession.Definition().AppKey(Credentials.AppKey)
    .GetSession() 
OR
# Cloud Session
    session = PlatformSession.Definition().AppKey(Credentials.AppKey)
    .OAuthGrantType(new GrantPassword().UserName(Credentials.RDPUser)
    .Password(Credentials.RDPPassword))
    .TakeSignonControl(true)
    .GetSession()
OR
# ADS / other Provider
    session = PlatformSession.Definition().Host(Credentials.ADSHost).GetSession()

C# code to connect to Eikon(desktop), Refinitiv Data Platform (cloud) or RTDS (deployed)

Note how the above C# code is very similar to the equivalent python calls:

    	
                session = rdp.DesktopSession(APP_KEY)     # Eikon / Refinitiv Workspace
OR
    session = rdp.PlatformSession(     # Cloud session
        APP_KEY,
        rdp.GrantPassword( username = RDP_LOGIN,
                          password = RDP_PASSWORD ))
OR
    session = rdp.DeployedPlatformSession(     # ADS / other Provider
        APP_KEY,
        deployed_platform_host = DEPLOYED_PLATFORM_HOST,
        deployed_platform_username = DEPLOYED_PLATFORM_USER_NAME
)

Python code to connect to Eikon, Refinitiv Data Platform or RTDS

As illustrated above and as you will hopefully continue to observe with the later code snippets, we have tried to offer as uniform an interface as possible - between the different language versions. Which means that you can learn once and reapply - across the different platforms and across the language versions as well.

Richer Fuller Content

Depending on the data-set that you are requesting, the Content layer can often provide a richer / fuller response than the function layer.

One example of this is the NewsStory content; when we used the Function layer the response consisted of an HTML representation of the actual News Story. However, if we request a News Story using the Content Layer we get back the full News Story payload which includes some metadata related to the story.

As I mentioned previously, the Content Layer requires a little bit more effort on the part of the developer - so, instead of the single get_news_story(storyId) function call we need the following:

    	
            newsStory = rdp.NewsStory(session) 
story = newsStory.get_story("urn:newsml:reuters.com:20191205:nNRAadmv9e:1")

As you can see, it requires an additional step of creating the NewsStory object (passing in the relevant session). Once created, we can then use the NewsStory object to request as many stories as we are interested in.

The above code snippet was a Synchronous request, however, we can use the Async or Event-Driven mode if required.

    	
            newsStory = rdp.NewsStory(session, 
    on_response = lambda story, response : print("Event Driven Output : \n", response)
 ) 
story = newsStory.get_story("urn:newsml:reuters.com:20191205:nNRAadmv9e:1")

Example of Event-Driven mode

When the response is received, the lambda function will output the following:

However, if we wanted the metadata that is provided with a News Story we can access this also sing the data.raw property of the response e.g:

    	
            print(json.dumps(story.data.raw, indent=2))

Python code to access raw payload

This would dump the full payload, a partial extract of which is shown below:

By referring to the documentation for the News Story Response, we can then drill down and extract the required metadata.

You can also access News Stories using similar code in the .NET version of the library e.g.

    	
            var story = NewsService.GetStory(session, StoryId); 
if (story.IsSuccess) 
    Console.WriteLine($"Content Type: {story.Data.ContentType}\n{story.Data.NewsStory}")

C# code to request and display News Story

Streaming Market Price Data

The Content layer expands on the Function layer's Snapshot functionality by supporting real-time Streaming requests for Level 1 Market Price data.

So, for example if we wanted BID, ASK and a few other fields for a bunch of currency instruments we can use the following:

    	
            pricing = rdp.StreamingPrices(session = mySession,
instruments = ['CHF=','GBP=', 'EUR='],
fields = ['BID', 'ASK','DSPLY_NAME','OPEN_PRC','HST_CLOSE'],
    on_refresh = lambda streaming_price, name, fields :
        print("Refresh for {} :\n{}".format(name,json.dumps(fields, indent=2))),
    on_update = lambda streaming_price, name, fields : display_update(name,fields),
    on_status = lambda streaming_price, name, status :
        print("Status for {} : {}".format( name, status))
)

Python code to define Streaming Level 1 Price request

The above code is using the Event driven mode of the Content Layer to specify lambda functions to handle the various response types. We can action the request by opening the Streaming Price object.

    	
            pricing.open()

Which generates something like the following initial Refresh and Status output:

Which is then followed by the tick updates as and when they occur:

These tick updates are displayed using the display_update() helper which looks likes this:

    	
            def display_update(name, fields): 
    currentTime = datetime.datetime.now() 
    print("{}: {} : ({}/{}) - : {}".format(currentTime.strftime("%H:%M:%S"), 
        name, fields['BID'], fields['ASK'], fields['DSPLY_NAME']))

Python helper to display tick update fields

We can request Streaming data using the C# version of the library in a very similar fashion:

    	
            using (var stream = Pricing.CreateStreamingPrices(
    new StreamingPrices.Params()
    .Universe("CHF=", "EUR=", "USD=")
    .WithFields("DSPLY_NAME", "BID", "ASK")
    .OnRefresh((o, item, refresh) => Console.WriteLine(refresh))
    .OnUpdate((o, item, update) => DisplayUpdate(item, update))
    .OnStatus((o, item, status) => Console.WriteLine(status))))
{
    stream.Open();
}

C# code to define and open Streaming Price request

    	
             private static void DisplayUpdate(string item, JObject fields) 
{
 // Display the quote for the asset we're watching 
    Console.WriteLine($"{ DateTime.Now.ToString("HH:mm:ss")}: 
        {item} ({fields["BID"],6}/{fields["ASK"],6}) - {fields["DSPLY_NAME"]}"); 
}

C# helper to display tick update fields

As you will note from the code snippets, we have tried to offer as uniform an interface as possible - between the different language versions.

Streaming Price Cache

In the above example, we are handling every single tick the server sends out by way of the Display Update lambda functions. Sometimes, however, we don't need to handle every update - perhaps we only want to update our GUI every 10 secs or when someone interacts with the GUI.

We can do this by utilising the Caching feature of the StreamingPrice object - in other words we create and open our streaming request, but we don't specify any lambda function.

    	
            prices = rdp.StreamingPrices(session = mySession,
    instruments = ['EUR=','CHF=', 'AED='],
    fields = ['BID', 'ASK','DSPLY_NAME', 'QUOTIM'])
prices.open()

Similarly in C#:

    	
            var streaming = Pricing.CreateStreamingPrices(new StreamingPrices.Params()
    .Universe("AED=", "CAD=", "USD=")
    .WithFields("DSPLY_NAME", "BID", "ASK"))
streaming.Open()

When the above code is executed, the library sends our request to the server and opens the stream. As the tick data arrives from the server, the library caches the values internally.

When we then want to get the latest values from the cache we take a snapshot:

    	
                 #python 
    prices.get_snapshot() 
OR 
    # C Sharp 
    streaming.GetSnapshot()

Selective Snapshot

There may be situations where we don't want a snapshot of all the RICs and/or Fields that we defined in our StreamingPrice object. This can also be easily achieved by optionally specifying a subset of RICS and/or Fields in the snapshot call:

    	
                #python 
    prices.get_snapshot( 
        universe = ['EUR=','AED='], 
        fields = ['BID', 'ASK']) 
OR 
    # C Sharp 
    streaming.GetSnapshot(new CacheSnapshot.Params() 
        .WithUniverse("AED=")
         .WithFields("DSPLY_NAME","BID"))

which returns a subset like:

The Delivery Layer...

So that concludes my look at the Function and Content layer - as I mentioned at the start, the Refinitiv Data Platform library is currently available as an alpha version release. We will continue to develop the library and hope to offer Function and Content layer support for other data content such as Environmental, Social and Governance (ESG) data, Search, Symbology, Streaming Chains, Bonds and so on.

In the meantime, you can access much of that Data using the Delivery Layer. So, join me again for the 2nd part of this article where we will explore the Delivery layer with usage examples covering ESG Data, Level 2 Full Depth Order Book, Surfaces, Curves and Alerts subscriptions.

Any questions/suggestions or issues related to the library, please feel free to post on the Refinitiv Data Platform Library Q&A Forum.