Use Case One

Refinitiv Data Platform News - Middle Office Compliance Use Cases

May 09,2022

Zoya Farberov

Introduction to Use Case

How a user can pass along a large list of identifiers: CUSIP, ISIN and or SEDOLs, and retrieve a company PermID | OrgName and status as an API response or as a bulk file

Approach

There are several possible approaches, including RDP API, to implement this Refinitiv Data Platform (RDP) News use case.

As we would like to showcase an approach in Python, and as ease of use is one of our primary considerations, the selected integration approach takes advantage of Refinitiv Data Library for Python- a brand-new, light, ease-of-use RDP integration offering, currently in pre-release.

Refinitiv Data Library for Python is documented on developer portal (see References section) and is positioned as a Refinitiv-supported member ofr Refinitiv Data Libraries family, its sister libraries being Refinitiv Data Library for .NET, and Refinitiv Data Library for TypeScript that is being prepared.

We are going to use Desktop session.

The Two Examples

Data Library - Symbol Conversion via Search
Data Library - Endpoint using Symbology

The reason we wish to show both approaches is that RD Library Symbol Conversion requires the RDP user to have permissions to RDP Search service. Whereas RDP Library Endpoint access only requires RDP user to have permissions to the specific endpoint in use, in this case RDP Symbology endpoint.

Data Library - Symbol Conversion via Search

Import Refinitiv Data Library
Open RDP Session ( Workspace)
Setup short default list with 3 IDs
Read working list with 2k+ IDs ( SEDOLS)
Convert the short list of symbols to PermIDs using Symbol Conversion
Request organization's common name and status
Convert 2K list to PermIDs using Symbol Conversion
Chunk the required list- define a helper function
Run chunked request and produce single result

References section links to the complete example code on GitHub. In the next discussion we are going to focus on the specifics aspects of conversion and lookup using RD Library.

Convert the Short List to PermIDs

    	
            # Symbol types:
# - symbol_conversion.RIC => RIC
# - symbol_conversion.ISIN => IssueISIN
# - symbol_conversion.CUSIP => CUSIP
# - symbol_conversion.SEDOL => SEDOL
# - symbol_conversion.TICKER_SYMBOL => TickerSymbol
# - symbol_conversion.OA_PERM_ID => IssuerOAPermID
# - symbol_conversion.LIPPER_ID => FundClassLipperID
 
response = symbol_conversion.Definition(
    symbols=defaultIDs,
    from_symbol_type=symbol_conversion.SymbolTypes.SEDOL,
    to_symbol_types=[
        symbol_conversion.SymbolTypes.RIC,
        symbol_conversion.SymbolTypes.OA_PERM_ID
    ],
).get_data()
 
response.data.df

Request Company Common Name and Organization Status

Very quick and easy, adding in Revenue or any other available and required field as well.

    	
            rd.get_data(
    universe=defaultIDs,
    fields=['TR.CommonName','TR.OrganizationStatusCode','TR.Revenue']    
)

resulting in:

Convert 2K List

If the requirement is to convert a much larger list, we may need to chunk it but 2K converts conveniently, in one request

    	
            response = symbol_conversion.Definition(
    symbols=myIDs,
    from_symbol_type=symbol_conversion.SymbolTypes.SEDOL,
    to_symbol_types=[
        symbol_conversion.SymbolTypes.RIC,
        symbol_conversion.SymbolTypes.OA_PERM_ID
    ],
).get_data()
 
response.data.df

Chunking Long List- Define a Helper Function

This is a good point of time to introduce chunking. The helper function can be used whenever it is to our advantage to partition the request into smaller pieces.

The output is a convenient list of chunks of our requested size of chunk

    	
            def list_to_chunks(long_list, chunk_size):
    chunked_list = list()
    for i in range(0, len(long_list), chunk_size):
        chunked_list.append(long_list[i:i+chunk_size])
    return chunked_list

Request CommonName and OrganizationStatusCode- Submit 2K List

    	
            dfResp = rd.get_data(
        universe=myIDs,
        fields=['TR.CommonName','TR.OrganizationStatusCode','TR.Revenue']    
    )
dfResp

Request CommonName and OrganizationStatusCode - Loop Permid Chunks

For larger lists, the request may need to be partitioned into chunks of size N (default is 1000), as requesting the full list of a larger size may result in an error. We append results into a list of results and once all results are received, restructure into a single dataframe in one shot, rather then concatenating to the dataframe and re-allocating and copying detaframe.

    	
            N = 1000
permids =response.data.df['IssuerOAPermID'].astype(str).values.tolist()
myPermidChunks = list_to_chunks(permids, N)
dataAll = []
dfAll = pd.DataFrame()
for i in range(len(myPermidChunks)):
    print('Iteration #',i)
    dfResp = rd.get_data(
        universe=myPermidChunks[i],
        fields=['TR.CommonName','TR.OrganizationStatusCode','TR.Revenue']    
    )
    dataAll.append(dfResp)
 #   print(dfResp)
dfAll = pd.concat(dataAll,ignore_index=True)
print('Displaying full result:')
display(dfAll)

results:

Let us next briefly review the same use case implementation using

Data Library - Endpoint Using Symbology

Import Refinitiv Data Library
Define a helper function listToChunks
Open RDP Session ( Workspace)
Setup short default list with 3 IDs
Read working list with 2k+ IDs ( SEDOLS)
Define Symbology Lookup Request Body Skeleton
Request lookup of the short list of symbols to PermIDs using Endpoint
Requests Lookup for Longer List of IDs using Endpoint (chunk)
Request organization's common name and status of the short list
Request organization's common name and status of the short list (chunk)

References section links to the complete example code on GitHub. In the next discussion we are going to focus on the specifics aspects of conversion and lookup uisng RD Library.

Define Request Body Skeleton

    	
            reqBodyPERMID  =  {         # Specify body parameters
        "from": [
            {
                "identifierTypes": [
                    "SEDOL"
                ],
                "values": [
                    
                ]
            }
        ],
        "to": [
            {
                "objectTypes": ["anyinstrument"],
                 "identifierTypes": ["PermID"]
            }
        ],
        "type": "auto"
    }

Request Lookup with defaultIDs

Enrich request body skeleton with defaultIDs

    	
            defaultIDs =  ['BDZDRC5','BFZP189','BGY7Q00']
 
reqBodyPERMID['from'][0]['values'] = defaultIDs 
request_definition = rd.delivery.endpoint_request.Definition(
    method = rd.delivery.endpoint_request.RequestMethod.POST,
    url = 'https://api.refinitiv.com/discovery/symbology/v1/lookup',
    body_parameters = reqBodyPERMID
)
 
response = request_definition.get_data()
response.data.raw
df = pd.json_normalize(response.data.raw,  record_path =['data'])
df

Resulting in:

Requests Lookup for Longer List of IDs

In chunks and display the single result

    	
            dataAll = []
dfCompleteResult = pd.DataFrame()
 
for i in range(len(myIDChunks)):
    reqBodyPERMID['from'][0]['values'] = myIDChunks[i] 
    request_definition = rd.delivery.endpoint_request.Definition(
        method = rd.delivery.endpoint_request.RequestMethod.POST,
        url = 'https://api.refinitiv.com/discovery/symbology/v1/lookup',
        body_parameters = reqBodyPERMID
    )
 
    response = request_definition.get_data()
    response.data.raw
    df = pd.json_normalize(response.data.raw,  record_path =['data'])
    
    print("Requested chunk #", i)
#    print(df)
    dataAll.append(df)
dfCompleteResult = pd.concat(dataAll,ignore_index=True)
print('Displaying result:')
display(dfCompleteResult)

our result looks like:

Extract PermIds

    	
            serAll = dfCompleteResult['output'].str.get(0)
permids = []
serAll
for val in serAll:
    if not pd.isna(val):
        permids.append(val.get('value'))
print(permids)

Request CommonName and OrganizationStatusCode - Loop Permid Chunks

The request is partitioned into chunks of size N (default is 1000), as the full list of a larger size may result in an error

    	
            N = 1000
myPermidChunks = list_to_chunks(permids, N)
dataAll = []
dfAll = pd.DataFrame()
for i in range(len(myPermidChunks)):
    print('Iteration #',i)
    dfResp = rd.get_data(
        universe=myPermidChunks[i],
        fields=['TR.CommonName','TR.OrganizationStatusCode','TR.Revenue']    
    )
    dataAll.append(dfResp)
    #print(resp)
dfAll = pd.concat(dataAll,ignore_index=True)
print('Displaying full result:')
display(dfAll)

The results being the same Dataframe of the full required results list as we have reviewed before, we would like to conclude this brief discussion of the use case with helpful