1. Home
  2. Article Catalog
  3. Intelligent Tagging | Extract Information from API Response

ARTICLE

Intelligent Tagging | Extract Information from API Response

Chavalit Jintamalit
Developer Advocate Developer Advocate

Introduction

This article will walk you through steps to make an API call to Intelligent Tagging(TRIT) with an unstructured text.

It then guides you to extract useful information from the Intelligent Tagging response.

This article provides a Python sample code in Jupyter Notebook and explains how it works.

A sample Jupyter Notebook file is available for download at the end of this article.

 

Prerequisites reading

Please read the Quick Start before reading this article.

You also need to register for an API key before you can make an API call. To do so, register for MyRefinitiv. Then login to PermID.org with your new credentials. An Open Calais API Key is automatically e-mailed to you.

 

1. Make API calls to TRIT

This session provides guidance on how to make API calls using Python.

Please follow these detailed steps:

1.1 Import required libraries

    	
            

import requests

import json

1.2 Prepare content

    	
            

#This article is taken from https://www.reuters.com/article/us-tomtom-stocks/tomtom-shares-lifted-by-bmw-peugeot-contract-wins-idUSKCN1MC23N

 

contentText = "TomTom shares lifted by BMW, Peugeot contract wins. AMSTERDAM (Reuters) - TomTom shares jumped 4 percent on Tuesday after the Dutch navigation company said it had struck deals with carmakers BMW and Peugeot.

TomTom said at the Paris motor show that it had extended its contract as a supplier of digital maps and traffic information for Peugeot, Citroen, Opel and Vauxhall models.

It also said it would start to provide traffic information for BMW, Mini and Rolls Royce vehicles.

Although TomTom did not provide financial details, the contract wins came as a relief for investors, worried about the entrance of Google as a competitor.

Last month TomTom shares lost a quarter of their value in one day as Google announced a far-reaching supply deal with a group of carmakers including Renault, Nissan and Mitsubishi.

TomTom shares recovered half of those losses last week as the company said it was considering selling its fleet-management business to focus on the head-to-head competition with the internet technology giant.

Shares in TomTom traded up 4 percent to 7.43 euros at 1530 GMT, 12 percent lower than before Google’s Sept. 18 announcement."

 

headType = "text/raw"

1.3. Make an API calls to TRIT with the prepared content

    	
            

token = '<a_valid_api_token>'

url = "https://api-eit.refinitiv.com/permid/calais"

payload = contentText.encode('utf8')

headers = {

    'Content-Type': headType,

    'X-AG-Access-Token': token,

    'outputformat': "application/json"

    }

 

TRITResponse = requests.request("POST", url, data=payload, headers=headers)

2. Load content into JSON object

After step 1.3, you will get a response back into the TRITResponse variable and its content is in the text property.

You can load the content into a JSON object by following this sample code:

    	
            

JSONResponse = json.loads(TRITResponse.text)

 

print(json.dumps(JSONResponse, indent=4, sort_keys=True))

This is an example of the content.

Note that the content is quite large so only a few keys and value are being displayed here:

    	
            

{

    "doc": {

        "info": {

            "calaisRequestID": "<some_id>",

            "docDate": "2019-10-08 06:45:08.360",

            "docId": "http://d.opencalais.com/dochash-1/183f5b8d-63c4-312c-987b-f2e6852ba06c",

            "docTitle": "",

            "document": "TomTom shares lifted by BMW, Peugeot contract wins. AMSTERDAM (Reuters) - TomTom shares jumped 4 percent on Tuesday after the Dutch navigation company said it had struck deals with carmakers BMW and Peugeot. TomTom said at the Paris motor show that it had extended its contract as a supplier of digital maps and traffic information for Peugeot, Citroen, Opel and Vauxhall models. It also said it would start to provide traffic information for BMW, Mini and Rolls Royce vehicles. Although TomTom did not provide financial details, the contract wins came as a relief for investors, worried about the entrance of Google as a competitor. Last month TomTom shares lost a quarter of their value in one day as Google announced a far-reaching supply deal with a group of carmakers including Renault, Nissan and Mitsubishi. TomTom shares recovered half of those losses last week as the company said it was considering selling its fleet-management business to focus on the head-to-head competition with the internet technology giant. Shares in TomTom traded up 4 percent to 7.43 euros at 1530 GMT, 12 percent lower than before Google\u2019s Sept. 18 announcement.",

            "id": "http://id.opencalais.com/M8R48JD25HW0SZ3Gxf1hqA",

            "ontology": "http://trit-us-east-1-sharedamd.int.refinitiv.com/owlschema/12.10.02/onecalais.owl.allmetadata.xml"

        },

        "meta": {

            "contentType": "text/raw",

            "language": "English",

            "processingVer": "AllMetadata",

            "serverVersion": "12.9.216:216",

            "signature": "digestalg-1|U268MDiRV/4irO6y4UnybiHOwC8=|Uqk29bCONe1sEDOIBjkxR+DI+OlJBJa+QOGITOix9csax2f6Y49wFg==",

            "stagsVer": "defaultVersion",

            "submissionDate": "2019-10-08 06:45:08.074",

            "submitterCode": "0ca6a864-5659-789d-5f32-f365f695e757"

        }

    },

    "http://d.opencalais.com/comphash-1/230f54f0-634d-38c3-a0a2-51b7aec9becc": {

        "_type": "Company",

        "_typeGroup": "entities",

        "_typeReference": "http://s.opencalais.com/1/type/em/e/Company",

        "confidence": {

            "aggregate": "0.764",

            "dblookup": "0.0",

            "resolution": "1.0",

            "statisticalfeature": "0.857"

        },

        "confidencelevel": "0.764",

        "forenduserdisplay": "false",

        "instances": [

            {

                "detection": "[ to provide traffic information for BMW, Mini and ]Rolls Royce[ vehicles. Although TomTom did not provide]",

                "exact": "Rolls Royce",

                "length": 11,

                "offset": 457,

                "prefix": " to provide traffic information for BMW, Mini and ",

                "suffix": " vehicles. Although TomTom did not provide"

            }

        ],

        "name": "Rolls Royce",

        "recognizedas": "name",

        "relevance": 0.2,

        "resolutions": [

            {

                "commonname": "Rolls Royce",

                "id": "https://permid.org/1-5040697296",

                "ispublic": "false",

                "name": "ROLLS-ROYCE MOTOR CARS LIMITED",

                "permid": "5040697296",

                "score": 1

            }

        ]

    },

    "some_more_keys.": {},

    "some_more_keys..": {},

    "some_more_keys...": {}

}

So we now have the TRIT response content in the JSONResponse variable and the datatype is JSON object.

3. Extract useful information from the JSON response

To extract the information the JSON response, you can follow these examples below:

3.1 Social Tag

Here is the sample JSON object:

    	
            

{

    "http://d.opencalais.com/dochash-1/183f5b8d-63c4-312c-987b-f2e6852ba06c/SocialTag/1": {

        "_typeGroup": "socialTag",

        "forenduserdisplay": "true",

        "id": "http://d.opencalais.com/dochash-1/183f5b8d-63c4-312c-987b-f2e6852ba06c/SocialTag/1",

        "importance": "1",

        "name": "Automotive industry",

        "originalValue": "Automotive industry",

        "socialTag": "http://d.opencalais.com/genericHasher-1/e078688f-9d13-384b-b29e-f12219e01f34"

    }

}

Here is the sample code:

    	
            

#Get socialTags

 

#Print Header

print('====Social Tags====')

print('Name, Importance')

 

for key in JSONResponse:

    if ('_typeGroup' in JSONResponse[key]):

        if JSONResponse[key]['_typeGroup'] == 'socialTag':

            print(JSONResponse[key]['name'] + ", " + JSONResponse[key]['importance'])

Here is the sample output

    	
            

====Social Tags====

Name, Importance

Automotive industry, 1

Transport, 1

Companies, 1

TomTom, 2

CAC 40, 2

Renault, 2

Peugeot, 2

BMW, 2

Traffic reporting, 2

3.2 Topic

Here is the sample JSON object:

    	
            

{

    "http://d.opencalais.com/dochash-1/183f5b8d-63c4-312c-987b-f2e6852ba06c/cat/1": {

        "_typeGroup": "topics",

        "forenduserdisplay": "false",

        "name": "Business_Finance",

        "score": 1

    }

}

Here is the sample code:

    	
            

#Get Topics

 

#Print Header

print('====Topics====')

print('Topics, Score')

 

for key in JSONResponse:

    if ('_typeGroup' in JSONResponse[key]):

        if JSONResponse[key]['_typeGroup'] == 'topics':

            print(JSONResponse[key]['name'] + ", " + str(JSONResponse[key]['score']))

Here is the sample output:

    	
            

====Topics====

Topics, Score

Business_Finance, 1

Technology_Internet, 0.926

3.3 Entity

Here is the sample JSON object:

    	
            

{

"http://d.opencalais.com/genericHasher-1/56fc901f-59a3-3278-addc-b0fc69b283e7": {

        "_type": "City",

        "_typeGroup": "entities",

        "_typeReference": "http://s.opencalais.com/1/type/em/e/City",

        "forenduserdisplay": "true",

        "instances": [

            {

                "detection": "[carmakers BMW and Peugeot. TomTom said at the ]Paris[ motor show that it had extended its contract as]",

                "exact": "Paris",

                "length": 5,

                "offset": 227,

                "prefix": "carmakers BMW and Peugeot. TomTom said at the ",

                "suffix": " motor show that it had extended its contract as"

            }

        ],

        "name": "Paris",

        "relevance": 0.2,

        "resolutions": [

            {

                "containedbycountry": "France",

                "latitude": "48.8742",

                "longitude": "2.346954",

                "name": "Paris,France",

                "rcscode": "G:12V",

                "shortname": "Paris"

            }

        ]

    }

}

Here is the sample code:

    	
            

#Get Entities

 

#Print Header

print('====Entities====')

print('Type, Name')

 

for key in JSONResponse:

    if ('_typeGroup' in JSONResponse[key]):

        if JSONResponse[key]['_typeGroup'] == 'entities':

            print(JSONResponse[key]['_type'] + ", " + JSONResponse[key]['name'])

Here is the sample output:

    	
            

====Entities====

Type, Name

City, AMSTERDAM

Company, Nissan

Company, Mitsubishi

Company, BMW

IndustryTerm, internet technology giant

Company, TomTom

Company, Google

Company, Peugeot

City, Paris

Company, OPEL

Company, Rolls Royce

Company, Renault

3.4 RIC

Here is the sample JSON object:

    	
            

{

"commonname": "Nissan",

"id": "https://permid.org/1-4295877341",

"ispublic": "true",

"name": "NISSAN MOTOR CO.,LTD.",

"permid": "4295877341",

"primaryric": "7201.T",

"score": 0.94075924,

"ticker": "7201"

}

Here is the sample code:

    	
            

#Get RIC code

 

#Print Header

print('====RIC====')

print('RIC')

 

for entity in JSONResponse:

    for info in JSONResponse[entity]:

        if (info =='resolutions'):

            for companyinfo in (JSONResponse[entity][info]):

                if 'primaryric' in companyinfo:

                    print(companyinfo['primaryric'])

Here is the sample output:

    	
            

====RIC====

RIC

7201.T

8058.T

BMWG.DE

TOM2.AS

PEUP.PA

RENA.PA

At this stage, you should be able to extract useful information from the Intelligent Tagging response.

Downloads

TRIT-Response_Extraction.zip