ARTICLE

Intelligent Tagging | Extract Information from API Response

September 28,2019

Chavalit Jintamalit

Introduction

This article will walk you through steps to make an API call to Intelligent Tagging(TRIT) with an unstructured text.

It then guides you to extract useful information from the Intelligent Tagging response.

This article provides a Python sample code in Jupyter Notebook and explains how it works.

A sample Jupyter Notebook file is available for download at the GitHub repository.

Prerequisites reading

Please read the Quick Start before reading this article.

You also need to register for an API key before you can make an API call. To do so, register for MyRefinitiv. Then login to PermID.org with your new credentials. An Open Calais API Key is automatically e-mailed to you.

1. Make API calls to TRIT

This session provides guidance on how to make API calls using Python.

Please follow these detailed steps:

1.1 Import required libraries

    	
            import requests
import json

1.2 Prepare content

    	
            #This article is taken from https://www.reuters.com/article/us-tomtom-stocks/tomtom-shares-lifted-by-bmw-peugeot-contract-wins-idUSKCN1MC23N
 
contentText = "TomTom shares lifted by BMW, Peugeot contract wins. AMSTERDAM (Reuters) - TomTom shares jumped 4 percent on Tuesday after the Dutch navigation company said it had struck deals with carmakers BMW and Peugeot.
TomTom said at the Paris motor show that it had extended its contract as a supplier of digital maps and traffic information for Peugeot, Citroen, Opel and Vauxhall models. 
It also said it would start to provide traffic information for BMW, Mini and Rolls Royce vehicles. 
Although TomTom did not provide financial details, the contract wins came as a relief for investors, worried about the entrance of Google as a competitor.
Last month TomTom shares lost a quarter of their value in one day as Google announced a far-reaching supply deal with a group of carmakers including Renault, Nissan and Mitsubishi.
TomTom shares recovered half of those losses last week as the company said it was considering selling its fleet-management business to focus on the head-to-head competition with the internet technology giant.
Shares in TomTom traded up 4 percent to 7.43 euros at 1530 GMT, 12 percent lower than before Google’s Sept. 18 announcement."
 
headType = "text/raw"

1.3. Make an API calls to TRIT with the prepared content

    	
            token = '<a_valid_api_token>'
url = "https://api-eit.refinitiv.com/permid/calais"
payload = contentText.encode('utf8')
headers = {
    'Content-Type': headType,
    'x-ag-access-token': token,
    'x-calais-selectiveTags': 'Company,socialtags,City,topic-all',
    'outputformat': "application/json"
    }
 
TRITResponse = requests.request("POST", url, data=payload, headers=headers)

2. Load content into JSON object

After step 1.3, you will get a response back into the TRITResponse variable and its content is in the text property.

You can load the content into a JSON object by following this sample code:

    	
            JSONResponse = json.loads(TRITResponse.text)
 
print(json.dumps(JSONResponse, indent=4, sort_keys=True))

This is an example of the content.

Note that the content is quite large so only a few keys and value are being displayed here:

    	
            {
    "doc": {
        "info": {
            "calaisRequestID": "<some_id>",
            "docDate": "2019-10-08 06:45:08.360",
            "docId": "http://d.opencalais.com/dochash-1/183f5b8d-63c4-312c-987b-f2e6852ba06c",
            "docTitle": "",
            "document": "TomTom shares lifted by BMW, Peugeot contract wins. AMSTERDAM (Reuters) - TomTom shares jumped 4 percent on Tuesday after the Dutch navigation company said it had struck deals with carmakers BMW and Peugeot. TomTom said at the Paris motor show that it had extended its contract as a supplier of digital maps and traffic information for Peugeot, Citroen, Opel and Vauxhall models. It also said it would start to provide traffic information for BMW, Mini and Rolls Royce vehicles. Although TomTom did not provide financial details, the contract wins came as a relief for investors, worried about the entrance of Google as a competitor. Last month TomTom shares lost a quarter of their value in one day as Google announced a far-reaching supply deal with a group of carmakers including Renault, Nissan and Mitsubishi. TomTom shares recovered half of those losses last week as the company said it was considering selling its fleet-management business to focus on the head-to-head competition with the internet technology giant. Shares in TomTom traded up 4 percent to 7.43 euros at 1530 GMT, 12 percent lower than before Google\u2019s Sept. 18 announcement.",
            "id": "http://id.opencalais.com/M8R48JD25HW0SZ3Gxf1hqA",
            "ontology": "http://trit-us-east-1-sharedamd.int.refinitiv.com/owlschema/12.10.02/onecalais.owl.allmetadata.xml"
        },
        "meta": {
            "contentType": "text/raw",
            "language": "English",
            "processingVer": "AllMetadata",
            "serverVersion": "12.9.216:216",
            "signature": "digestalg-1|U268MDiRV/4irO6y4UnybiHOwC8=|Uqk29bCONe1sEDOIBjkxR+DI+OlJBJa+QOGITOix9csax2f6Y49wFg==",
            "stagsVer": "defaultVersion",
            "submissionDate": "2019-10-08 06:45:08.074",
            "submitterCode": "0ca6a864-5659-789d-5f32-f365f695e757"
        }
    },
    "http://d.opencalais.com/comphash-1/230f54f0-634d-38c3-a0a2-51b7aec9becc": {
        "_type": "Company",
        "_typeGroup": "entities",
        "_typeReference": "http://s.opencalais.com/1/type/em/e/Company",
        "confidence": {
            "aggregate": "0.764",
            "dblookup": "0.0",
            "resolution": "1.0",
            "statisticalfeature": "0.857"
        },
        "confidencelevel": "0.764",
        "forenduserdisplay": "false",
        "instances": [
            {
                "detection": "[ to provide traffic information for BMW, Mini and ]Rolls Royce[ vehicles. Although TomTom did not provide]",
                "exact": "Rolls Royce",
                "length": 11,
                "offset": 457,
                "prefix": " to provide traffic information for BMW, Mini and ",
                "suffix": " vehicles. Although TomTom did not provide"
            }
        ],
        "name": "Rolls Royce",
        "recognizedas": "name",
        "relevance": 0.2,
        "resolutions": [
            {
                "commonname": "Rolls Royce",
                "id": "https://permid.org/1-5040697296",
                "ispublic": "false",
                "name": "ROLLS-ROYCE MOTOR CARS LIMITED",
                "permid": "5040697296",
                "score": 1
            }
        ]
    },
    "some_more_keys.": {},
    "some_more_keys..": {},
    "some_more_keys...": {}
}

So we now have the TRIT response content in the JSONResponse variable and the datatype is JSON object.

3. Extract useful information from the JSON response

To extract the information the JSON response, you can follow these examples below:

3.1 Social Tag

Here is the sample JSON object:

    	
            {
    "http://d.opencalais.com/dochash-1/183f5b8d-63c4-312c-987b-f2e6852ba06c/SocialTag/1": {
        "_typeGroup": "socialTag",
        "forenduserdisplay": "true",
        "id": "http://d.opencalais.com/dochash-1/183f5b8d-63c4-312c-987b-f2e6852ba06c/SocialTag/1",
        "importance": "1",
        "name": "Automotive industry",
        "originalValue": "Automotive industry",
        "socialTag": "http://d.opencalais.com/genericHasher-1/e078688f-9d13-384b-b29e-f12219e01f34"
    }
}

Here is the sample code:

    	
            #Get socialTags
 
#Print Header
print('====Social Tags====')
print('Name, Importance')
 
for key in JSONResponse:
    if ('_typeGroup' in JSONResponse[key]):
        if JSONResponse[key]['_typeGroup'] == 'socialTag':
            print(JSONResponse[key]['name'] + ", " + JSONResponse[key]['importance'])

Here is the sample output

    	
            ====Social Tags====
Name, Importance
Automotive industry, 1
Transport, 1
Companies, 1
TomTom, 2
CAC 40, 2
Renault, 2
Peugeot, 2
BMW, 2
Traffic reporting, 2

3.2 Topic

Here is the sample JSON object:

    	
            {
    "http://d.opencalais.com/dochash-1/183f5b8d-63c4-312c-987b-f2e6852ba06c/cat/1": {
        "_typeGroup": "topics",
        "forenduserdisplay": "false",
        "name": "Business_Finance",
        "score": 1
    }
}

Here is the sample code:

    	
            #Get Topics
 
#Print Header
print('====Topics====')
print('Topics, Score')
 
for key in JSONResponse:
    if ('_typeGroup' in JSONResponse[key]):
        if JSONResponse[key]['_typeGroup'] == 'topics':
            print(JSONResponse[key]['name'] + ", " + str(JSONResponse[key]['score']))

Here is the sample output:

    	
            ====Topics====
Topics, Score
Business_Finance, 1
Technology_Internet, 0.926

3.3 Entity

Here is the sample JSON object:

    	
            {
 "http://d.opencalais.com/genericHasher-1/56fc901f-59a3-3278-addc-b0fc69b283e7": {
        "_type": "City",
        "_typeGroup": "entities",
        "_typeReference": "http://s.opencalais.com/1/type/em/e/City",
        "forenduserdisplay": "true",
        "instances": [
            {
                "detection": "[carmakers BMW and Peugeot. TomTom said at the ]Paris[ motor show that it had extended its contract as]",
                "exact": "Paris",
                "length": 5,
                "offset": 227,
                "prefix": "carmakers BMW and Peugeot. TomTom said at the ",
                "suffix": " motor show that it had extended its contract as"
            }
        ],
        "name": "Paris",
        "relevance": 0.2,
        "resolutions": [
            {
                "containedbycountry": "France",
                "latitude": "48.8742",
                "longitude": "2.346954",
                "name": "Paris,France",
                "rcscode": "G:12V",
                "shortname": "Paris"
            }
        ]
    }
}

Here is the sample code:

    	
            #Get Entities
 
#Print Header
print('====Entities====')
print('Type, Name')
 
for key in JSONResponse:
    if ('_typeGroup' in JSONResponse[key]):
        if JSONResponse[key]['_typeGroup'] == 'entities':
            print(JSONResponse[key]['_type'] + ", " + JSONResponse[key]['name'])

Here is the sample output:

    	
            ====Entities====
Type, Name
City, AMSTERDAM
Company, Nissan
Company, Mitsubishi
Company, BMW
IndustryTerm, internet technology giant
Company, TomTom
Company, Google
Company, Peugeot
City, Paris
Company, OPEL
Company, Rolls Royce
Company, Renault

3.4 RIC

Here is the sample JSON object:

    	
            {
 "commonname": "Nissan",
 "id": "https://permid.org/1-4295877341",
 "ispublic": "true",
 "name": "NISSAN MOTOR CO.,LTD.",
 "permid": "4295877341",
 "primaryric": "7201.T",
 "score": 0.94075924,
 "ticker": "7201"
}

Here is the sample code:

    	
            #Get RIC code
 
#Print Header
print('====RIC====')
print('RIC')
 
for entity in JSONResponse:
    for info in JSONResponse[entity]:
        if (info =='resolutions'):
            for companyinfo in (JSONResponse[entity][info]):
                if 'primaryric' in companyinfo:
                    print(companyinfo['primaryric'])

Here is the sample output:

    	
            ====RIC====
RIC
7201.T
8058.T
BMWG.DE
TOM2.AS
PEUP.PA
RENA.PA

At this stage, you should be able to extract useful information from the Intelligent Tagging response.

Source Code

Github - Jupyter notebook

Related APIs

Intelligent Tagging - RESTful API

A Practical Approach to Understanding and Ingesting Intelligent Tagging Output for Your Use Case

Mitchell Van Eps

Enhancing your Outlook emails with Intelligent Tagging

Nick Zincone

Explore all Articles

Articles

Developer Tools

Use Cases

Upcoming Events

Past Events

About Us

Intelligent Tagging | Extract Information from API Response

Introduction

Prerequisites reading

1. Make API calls to TRIT

2. Load content into JSON object

3. Extract useful information from the JSON response

3.1 Social Tag

3.2 Topic

3.3 Entity

3.4 RIC

Related Articles