Messenger Bot API and Eikon Data API Introduction
Update: September 2021
Refinitiv's Messenger application is a free to use, compliant and secure messaging platform. It is a powerful communication tool that provides desktop, mobile, and web access, and allows sharing messages, data, files, screenshots, and emoticons with your contacts. The Messenger Bot API provides a set of available API calls to build automated workflows or bots for the Messenger application. The Bot API allows your applications to connect with and pass information into Eikon’s Messenger service programmatically or interact with a bot via a WebSocket connection.
The Eikon Data API (or DAPI) for Python provides simple access for users who require programmatic access to Refinitiv data on the desktop. These APIs are intended for Refinitiv Workspace/Eikon Desktop application users. The API lets users access to the following Refinitiv content sets such as Fundamental and reference data, Price time series, News, Symbology conversion and etc. It is designed for ease of use and for rapid application development by leveraging Refinitiv data and entitlements to simplify market data management and reporting.
Symbology Conversion Chat Bot Demo overview
This article shows how to integrates Messenger Bot API with Eikon Data API to be a basic Python chat bot application. The symbology chat bot receives a symbology conversion request from users in the chatroom, then uses Eikon Data API coverts the requested symbol and lets the Bot API response to user.
This demo application supports the following instrument codes conversion:
- RIC
- ISIN
- SEDOL
- CUSIP
- lipperID
- OAPermID
Note: The Messenger Bot API uses Refinitiv Data Platform APIs (RDP APIs) for authentication process. The APIs provide simple web based API access to a broad range of content for developers. Please refer to RDP APIs page for more detail.
Eikon Data API Usage and Limits Guideline
The Eikon Data API provides access to certain Refinitiv Workspace/Eikon data. Your access and use of Refinitiv Workspace/Eikon, including the Eikon Data API and any data included therein, is subject to the terms and conditions of your organization’s agreement with us. Refinitiv Workspace/Eikon is an information service that is licensed for individual use and requires a log in session. A log in session controls entitlements and reasonable use for a single user. In line with single user workflow demands, the Eikon Data API imposes limits to ensure that our overall platform can sustain the usage rates of all individual Eikon users accessing data via APIs.
Please refer to Eikon Data API Usage and Limits Guideline page for the full detail of API usage limit and how to handle it. The examples of usage limits are shown below:
- Requests per second: The number of requests sent to the platform via the Eikon Data APIs cannot exceed this limit. This number of requests is counted across all client applications connected to the same Eikon instance. For example if 4 applications connected to the same Eikon send 1 request per second, the throttling mechanism will consider an aggregated rate of 4 requests per seconds.The current limit value (10-Oct-2019) is 5 requests per second.
- Requests per day - The number of requests sent to the platform per day via the Eikon Data APIs cannot exceed this limit. This number of requests is counted across all client applications connected to the same Eikon instance.The current limit value (10-Oct-2019) is 10,000 requests per day.
Messenger Chat Bot Message Rate Throttling
Please note that A throttling mechanism limits messages to a maximum rate of one message per second.
An error is returned if this limit is exceeded.
Prerequisite
This demo project requires the following dependencies.
- Refinitiv Workspace with Eikon Data API access.
- Messenger Bot API access and license.
- Python compiler and runtime.
- Python's Eikon Data API, requests, websocket-client libraries. (They will be automatic installed via pip install -r requirements.txt command) .
Note:
- The Python demo applications has been qualified with Python version 3.7.7.
- Please refer to the pip installation guide page if your environment does not have the pip tool installed.
Getting the AppKey value
Once you have setup your Messenger user, please access the AppKey Generator Tool via Eikon Desktop/Refinitiv Workspace application (go to the Search Bar and type APP KEY, then select the AppKey Generator). Please access with your Refinitiv Workspace/Eikon account, not your bot account.
You can generate your AppKey via the following steps:
- Enter an App Display Name.
- Select the tick box next to 'EDP API' and 'Eikon Data API' as the API Types.
- Click ‘Register’ New App button.
You will then see a row for your new app with an AppKey item, which is your client_id for the Refinitiv Data Platform (RDP) and Eikon Data API Authentication.
Note: You can create a different App Key (client_id) for 'EDP API' and 'Eikon Data API' by just creating two applications for each API type.
Setting the Messenger application
Please refer to Messenger BOT API Quick Start Guide regarding how to add the Bot and create a Chatroom for you and your Bot.
Messenger Bot API Application Workflow
The Messenger Bot API application workflow combines the usage of RDP APIs HTTP REST API with the Bot REST and WebSocket APIs. Please see a full detail below:
- Authenticate with Refinitiv Data Platform API (RDP API) and obtain access token, refresh token and session expiration interval for your bot.
- Get the list of chat rooms from Messenger Bot API via HTTP REST.
- Join the selected chat room via HTTP REST API call.
- Connect to Messenger Bot API WebSocket server.
- Send connect request message using RDP access token to the WebSocket server.
- Receive chat messages from the chat room via the WebSocket event.
- Send message back to the chat room via HTTP REST API call.
- Re-authenticate with RDP API using a refresh token before token expiration to get a new access token.
- Send authenticate request message using newly received RDP access token to the WebSocket server token expiration to keep session open.
- Close the connection to finish.
RDP Authentication Process
Refinitiv Data Platform entitlement check is based on OAuth 2.0 specification. The first step of an application work flow is to get a token, which will allow access to the protected resource, i.e. data REST API's. For the full detail and explanation of RDP Authentication process application workflow, please refer to the following RDP APIS tutorials:
- Introduction to the Request-Response API.
- Authorization - All about tokens.
- Authorization in Python.
The chatbot_demo_symbology.py application file utilizes RDPTokenManagement class of rdp_token.py Python module file to manage all RDP operations and authentication process. This module source code is based on RDP Python Quickstart Python source code implemented by Gurpreet Bal with modification to support the WebSocket connection scenario.
Joining the Chat Room, Send and Receive Chat Room Messages.
Once the application success authenticated with RDP Auth Service, the application can send 1-to-1 message to user in Messenger application, joins a chat room, receives and posts a message to a joined chat room.
Please refer to Messenger BOT API Quick Start Guide and Bot API Developer document pages which provide a full detail of Messenger Bot API usages. You need to understand the HTTP request and response message, Websocket implementation for interactive Bot along with the structure of data, and limitation of the API.
Eikon Data API integration
The chatbot_demo_symbology.py application file utilizes DAPISessionManagement class of dapi_session.py Python module file to manage all Eikon Data API operations. The DAPISessionManagement class receives the Eikon Data API App Key from the application, then it imports the library and set the App Key of the application. The application is now ready to send data retrieval requests to Refinitiv.
# dapi_session.py file
# Import the required libraries for Eikon and JSON operations
import eikon as ek
import logging
import json
class DAPISessionManagement:
dapi_app_key = ''
# Constructor function
def __init__(self, app_key):
self.dapi_app_key = app_key
ek.set_app_key(self.dapi_app_key)
The DAPISessionManagement class also lets the chat bot application checks if it success initiate session with Refinitiv Workspace/Eikon Desktop application or not via the verify_desktop_connection() function. This function uses Eikon Data API get_port_number() which returns the port number used to communicate with the Eikon Data API Proxy. If the connection success, it always returns a default port "9000".
# dapi_session.py file
class DAPISessionManagement:
# ... Constructor function
# verify if Eikon Data API is connect to Refinitiv Workspace/Eikon Desktop application
def verify_desktop_connection(self):
ek_port_number = ek.get_port_number()
if int(ek_port_number) == 9000:
return True
else:
return False
Eikon Data API Symbology Conversion
Lastly, the DAPISessionManagement class uses Eikon Data API get_data() function to get the request instrument code of the inputted symbol via Refinitiv instrument code fields such as TR.RIC, TR.ISIN, TR.SEDOL.
Please note that you can choose Eikon Data API get_symbology() function which returns instrument names converted into another instrument code instead in your own implementation. This demo application choose get_data() function because user does not require to input an instrument code to convert from parameter like the get_symbology() function.
# dapi_session.py file
class DAPISessionManagement:
# ... Constructor function
'''
convert symbol to targe instrument code type with ek.get_data function. The supported fields are
- TR.RIC
- TR.ISIN
- TR.SEDOL
- TR.CUSIP
- TR.LipperRICCode
- TR.OrganizationID
'''
def convert_symbology(self, symbol, target_symbol_type = 'TR.ISIN'):
converted_result = True
try:
response = ek.get_data(symbol,target_symbol_type, raw_output = True)
if 'error' in response or not response['data'][0][1]: # The get_data can returns both 'error' and just empty/null result
converted_result = False
return converted_result, response
except Exception as ex:
logging.error('Data API: get_data exception failure: %s' % ex)
return False, None
By default, the Eikon Data API get_data() function returns data as DataFrame object. However, since this is a console application, so the DAPISessionManagement class set raw_output = True parameter to get_data() function which makes the function returns data as JSON message instead. If the JSON response message contain error attribute or an empty result, it means the conversion is fail.
Now the DAPISessionManagement class of dapi_session.py is ready to handle all Eikon Data API operations for the main application.
Import DAPISessionManagement into Chat Bot Application
The chatbot_demo_symbology.py application then imports DAPISessionManagement class from dapi_session.py module. The application also checks if the Eikon Data API success connect to Refinitiv Workspace/Eikon Desktop application by checking the verify_desktop_connection() function.
# chatbot_demo_symbology.py
from dapi_session import DAPISessionManagement # Module for managing Eikon Data API session
# Input your Refinitiv Workspace/Eikon Desktop Eikon Data API App Key
data_api_appkey = '---YOUR DATA API APPKEY---'
dapi = None
# =============================== Main Process ========================================
print('Setting Eikon Data API App Key')
# Create and initiate DAPISessionManagement object
dapi = DAPISessionManagement(data_api_appkey)
if not dapi.verify_desktop_connection(): #if init session with Refinitiv Workspace/Eikon Desktop success
print('Please start Refinitiv Workspace in your local machine')
# Abort application
sys.exit(1)
else:
print('Initiate Eikon Data API success')
Preparing the Chat Bot Interaction
The application uses regular expression to detect symbology conversion request message from users. All reply messages use Python String Template feature for easy messages customizable for each scenario.
from string import Template
import re
# =============================== Data API and Symbology Variables ========================================
# Conversion request message Regular Expression pattern
symbology_request_pattern = r'Please convert (?P<symbol>.*) to (?P<target_symbol_type>.*)'
# Response messages templates
response_template = Template('@$sender, the $target_symbol_type instrument code of $symbol is $converted_symbol')
response_error_template = Template('@$sender, the $target_symbol_type instrument code of $symbol is not available')
response_unsupported_type_template = Template('@$sender, unsupported <target symbol type> $target_symbol_type\n'
'The supported <target symbol type> are: CUSIP, ISIN, SEDOL, RIC, lipperID and OAPermID\n')
response_unsupported_command = Template('@$sender, unsupported command. Please use the following command to convert instrument code\n'
'"Please convert <symbol> to <target symbol type>"\n'
'\n'
'Example:\n'
'Please convert IBM.N to ISIN')
# Dictionary to map between input <target symbol type> and Refinitiv Workspace instrument type fields
symbol_dict = {'RIC':'TR.RIC','ISIN':'TR.ISIN','SEDOL':'TR.SEDOL',
'CUSIP':'TR.CUSIP','lipperID':'TR.LipperRICCode','OAPermID':'TR.OrganizationID'}
# Help/Instruction Message
help_message = ('You can ask me to convert instrument code with this command\n'
'"Please convert <symbol> to <target symbol type>"\n'
'The supported <target symbol type> are: CUSIP, ISIN, SEDOL, RIC, lipperID and OAPermID\n'
'\n'
'Example:\n'
'Please convert IBM.N to ISIN')
The symbol_dict dictionary will be used for mapping from readable instrument code types (RIC, ISIN, OAPermID) to the Eikon Data API fields (TR.RIC, TR.ISIN, TR.OrganizationID).
Sending The Welcome Message
Once the application success authenticates with RDP Auth Service and joins the associate chat room, the application sends a welcome message to notify user regarding how to use symbology conversion command. The application calls post_message_to_chatroom() function which sends a message to the joined chat room via the Messenger Bot Rest API. The API information is following:
- API Endpoint URL 1: https://api.refinitiv.com/messenger/beta1/chatrooms/<chatroomId>/post
- API Endpoint URL 2: https://api.refinitiv.com/messenger/beta1/managed_chatrooms/<chatroomId>/post
- Method: POST
- Header:
- Authorization = Authorization: "Bearer " + <access token>
Please notice the space between the Bearer and Access Token values.
The application uses Python requests library to send the HTTP Post message to the Messenger Bot Rest API.
# Posting Messages to a Chatroom via HTTP REST
def post_message_to_chatroom(access_token, joined_rooms, room_id=None, text='', room_is_managed=False):
if room_id not in joined_rooms:
joined_rooms = join_chatroom(access_token, room_id, room_is_managed)
if joined_rooms:
if room_is_managed:
url = '{}{}/managed_chatrooms/{}/post'.format(
gw_url, bot_api_base_path, room_id)
else:
url = '{}{}/chatrooms/{}/post'.format(
gw_url, bot_api_base_path, room_id)
body = {
'message': text
}
# Print for debugging purpose
logging.info('Sent: %s' % (json.dumps(
body, sort_keys=True, indent=2, separators=(',', ':'))))
response = None
try:
# Send a HTTP request message with Python requests module
response = requests.post(
url=url, data=json.dumps(body), headers={'Authorization': 'Bearer {}'.format(access_token)})
except requests.exceptions.RequestException as e:
logging.error('Messenger BOT API: post message to exception failure: %s ' % e)
if response.status_code == 200: # HTTP Status 'OK'
joined_rooms.append(room_id)
print('Messenger BOT API: post message to chatroom success')
# Print for debugging purpose
logging.info('Receive: %s' % (json.dumps(
response.json(), sort_keys=True, indent=2, separators=(',', ':'))))
else:
print('Messenger BOT API: post message to failure:',
response.status_code, response.reason)
print('Text:', response.text)
pass
# =============================== Main Process ========================================
# Send Greeting message
post_message_to_chatroom( access_token, joined_rooms, chatroom_id, 'Hi, I am a chatbot symbology converter.\n\n' + help_message)
The result is following:
Receiving Chat Room Chat Messages
Once the chat bot application connects to the Messenger Bot API WebSocket server, all chat messages from users in the chat room will be available to the application via WebSocket's on_message event in JSON format. The JSON message format is following:
{
"event":"chatroomPost",
"post":{
"chatroomId":"<chat room id>",
"chatroomType": "chatroom",
"sender":{
"email":"<user email>"
},
"message":"<user chat message>",
"messageId":"<message id>",
"timestamp":"<timestamp>"
}
}
The Symbology chat bot application then passes incoming JSON message to process_message() function to process users message.
# =============================== WebSocket functions ========================================
def on_message(_, message): # Called when message received, parse message into JSON for processing
message_json = json.loads(message)
logging.debug('Received: %s' % (json.dumps(message_json, sort_keys=True, indent=2, separators=(',', ':'))))
process_message(message_json)
# Process incoming message from a joined Chatroom via the WebSocket connection
def process_message(message_json):
message_event = message_json['event']
...
Symbology Conversion Message Operations
Lets focus on how the process_message() function handles symbology request message from the user. The chat bot demo needs to support various use cases such as the following:
- User request for the help message via /help command.
- User post a valid conversion request message.
- User post a valid message but wrong instrument code type.
- User post invalid message to a chat room.
The below diagram show the symbology conversion workflow. The blue box is a bot response to a chat room.
Process Help Request Message
If users forgot symbology conversion command, they can send /help message to the chat room to get an instruction message from the chat bot. The process_message() function just detects /help message and then sends the help message to the chat room for all users via post_message_to_chatroom() function.
def process_message(message_json): # Process incoming message from a joined Chatroom via the WebSocket connection
message_event = message_json['event']
if message_event == 'chatroomPost':
try:
incoming_msg = message_json['post']['message']
print('Receive text message: %s' % (incoming_msg))
if incoming_msg == '/help': # if users request for help, response with a help message
post_message_to_chatroom(access_token, joined_rooms, chatroom_id, help_message)
else: # otherwise, check incoming message patter
...
except Exception as error:
logging.error('Post message to a Chatroom fail : %s' % error)
The result is following:
Handles Symbology Request Message
This part focus heavily on the Regular Expression matching result of user's post message and 'Please convert (?P<symbol>.*) to (?P<target_symbol_type>.*)' command pattern. The input symbol will be available via <symbol> group and the target instrument code will be available <target_symbol_type> group.
The application also get the user email information from incoming JSON message to let the Bot response to a correct person who post the conversion request.
# process_message() function
if incoming_msg == '/help': # if users request for help, response with a help message
post_message_to_chatroom(access_token, joined_rooms, chatroom_id, help_message)
else: # otherwise, check incoming message patter
try:
sender = message_json['post']['sender']['email'] # Get message's sender
match = re.match(symbology_request_pattern, incoming_msg, flags=re.IGNORECASE) # match incoming message with Regular Expression
response_message = None
if match: # If incoming message match r'Please convert (?P<symbol>.*) to (?P<target_symbol_type>.*)' pattern, it is a symbology convert request message.
symbol = match.group('symbol') # get requested symbol
target_symbol_type = match.group('target_symbol_type') # get target_symbol_type
if target_symbol_type in symbol_dict: # check if user input a supported instrument code type
# convert symbology with Eikon Data API in DAPISessionManagement class
result, converted_response = dapi.convert_symbology(symbol, symbol_dict[target_symbol_type])
# process result.
else: # if user request for an unsupported instrument code type
response_message = response_unsupported_type_template.substitute(sender = sender, target_symbol_type = target_symbol_type)
else: # If user input other messages
response_message = response_unsupported_command.substitute(sender = sender)
# Send a message (convert result, or unsupported symbol type) to the chatroom.
post_message_to_chatroom( access_token, joined_rooms, chatroom_id, response_message)
except AttributeError as attrib_error:
logging.error('IOError Exception: %s' % attrib_error)
The application also needs to handle a symbology conversion success and fail case differently in order to give a proper response message to user via a chat room. The DAPISessionManagement.convert_symbology() function returns 2 values to the application, the conversion result (True or False) and Eikon Data API JSON response message in converted_response variable.
The chat bot choose a response message template that will be posted to a chat room based on the result value.
# process_message() function
# if target_symbol_type in symbol_dict:
# convert symbology with Eikon Data API in DAPISessionManagement class
result, converted_response = dapi.convert_symbology(symbol, symbol_dict[target_symbol_type])
if result: #Convert success
response_message = response_template.substitute(sender = sender,
converted_symbol = converted_response['data'][0][1], # Get converted symbol result
symbol = symbol,
target_symbol_type = converted_response['headers'][0][1]['displayName'])
else: # convert fail or not found a match
response_message = response_error_template.substitute(sender = sender, target_symbol_type = target_symbol_type, symbol = symbol)
Now the chat bot is ready to receive symbology conversion request message from user and post the response to a chat room.
Running The Demo Application
Please note that the Refinitiv Workspace/Eikon application integrates a Data API proxy that acts as an interface between the Eikon Data API Python library and the Eikon Data Platform. For this reason, the Refinitiv Workspace/Eikon application must be running when you use the Eikon Data API Python library.
The first step is unzip or download the example project folder from GitHub into a directory of your choice, then choose how to run application based on your environment below.
- Open a command prompt or console.
- Run $>pip install -r requirements.txt command in a console to install all the dependencies libraries.
- Open the chatbot_demo_symbology.py demo application source code in the src folder with your editor and input the following parameters.
- app_key: Your AppKey
- bot_username: Your Bot username
- bot_password: Your Bot password
- data_api_appkey : Your Eikon Data AppKey
- Open a command prompt and go to folder src, then run the demo application with the $>python chatbot_demo_symbology command.
Example Result
Once the demo application connects to Refinitiv Workspace/Eikon desktop application and show the following message if it can initial session with the desktop application.
Initiate Eikon Data API success
Then demo application continues authentication process, get an associate chatroom, then join that chatroom. The bot will send a greeting message with the symbology command instruction.
You can ask the chatbot to convert instrument code type for you with Please convert <symbol> to <target_symbol_type> command. The supported <target_symbol_type> are CUSIP, ISIN, SEDOL, RIC, lipperID and OAPermID
If you forget the command, you can send /help message in to a Chatroom to see an example command.
Next Step
The Messenger Bot API provides a set of APIs calls to build automated workflows or bots for the Messenger application. The API can integrates with other Refinitiv APIs such as Eikon Data API to extend Interactive Chat Bot capability for users in Refinitiv Workspace/Eikon Desktop application. There are many open opportunities to integrate with the chat bot to maximize the chat bot usages and provides assistant for both the business and the consumer.
This demo project demonstrates a simple chat bot interaction mechanism for a basic request-response chat message only. For more advance chat bot interaction, please see How to build Refinitiv Messenger Interactive ChatBot with Python Machine Learning and Messenger Bot API article which show how to integrate Chat Bot with Machine Learning for a better bot interaction experience.
References
For further details, please check out the following resources:
- Refinitiv Messenger Bot API page on the Refinitiv Developer Community website.
- Refinitiv Messenger Bot API: Quick Start.
- Refinitiv Messenger Bot API: Documentation page.
- Refinitiv Messenger Bot API Demo application - JavaScript.
- Refinitiv Messenger Bot API Demo application - Python.
- Refinitiv Eikon Data API page on the Refinitiv Developer Community website.
- Eikon Data API Quick Start Guide page.
- Eikon Data API Tutorial page.
- Python Quants Video Tutorial Series for Eikon API.
- Eikon Data API Python Reference Guide.
- Eikon Data API Troubleshooting article.
- Eikon Data API Usage and Limits Guideline page.
- Refinitiv Data Platform APIs page.
For any question related to the Messenger Bot API, please use the Developers Community Q&A Forum for Messenger Bot API and Eikon Data API.
