Release Notes Digest

This document is a digest of all Intelligent Tagging release notes since R9.5.

New Features

Known Issues and Fixed Bugs

 

 

New Features

The following table lists the new features and enhancements introduced since R9.5.
Please note that most releases also incorporate new RCS topic classifiers and infrastructure improvements, not listed here.

New Feature/Enhancement - Relevant to both On Premise and Hosted Intelligent Tagging unless specified - Note that releases 12.10 and higher are not currently available to On Premise customers Release
New in this release, classification topics trained for specific (not standard) use cases are disabled by default, reducing noise in the topic classification output. 13.6
Currency tagging - Improved precision and recall for extraction. Quality improvements to the relevance feature. 13.4
MarketIndex tagging - Improved precision and recall for extraction. Quality improvements to the relevance feature. 13.4
CurrencyPair tagging  - Improved precision and recall for extraction. Quality improvements to the relevance feature. 13.4
x-calais-selectiveTags header - The list of valid values supported by the header now includes all Intelligent Tagging entities, relationships, and classification options. Detailed information about the updated x-calais-selectiveTags header... 13.3
Currency tagging - Currency tagging is once again supported, with precision and recall improvements. 13.3
MarketIndex tagging - Precision and recall improvements. 13.3
CurrencyPair tagging - Precision and recall improvements. 13.3

New URL to Intelligent Tagging on the Developer Community:

https://developers.refinitiv.com/en/api-catalog/open-perm-id/intelligent-tagging-restful-api

13.1
New documents on the Dev Portal Documentation Tab: RCS Topic Tagging; Synonyms for Entity Relation Types; Linking to Eikon; Supported Sources. (This content was previously in the Supplementary Guide for Premium Users, now retired.) 13.1

Files moved from the Downloads Tab of the Dev Portal to MyRefinitiv:

List of all supported RCS Topics

Hierarchical Tree View of all RCS Topics

On Premise Installation and Administration Guide - available with the software package on MyRefinitiv

13.1

Files moved to Sharepoint:

German Company Tagging - Company Lexicon

Supplementary Guide for internal Intelligent Tagging (for internal Refinitiv customers who do not connect through API Gateway)

13.1
New content class - Transcripts. New support for optimized tagging of transcript documents in XML format. Trigger this feature with x-calais-contentClass:transcripts. 13.0 HF
New resource URL: https://api-eit.refinitiv.com/permid/calais. Relevant to all external hosted and free Open Calais users. API requests to both the new and old resource URL are supported until February 28th, 2020. After that, only the new URL is supported.  13.0+
Release Notes: Release notes are now published online, on the Documentation tab of the Developer Portal. 13.0

Person tagging: Infrastructure and quality improvements to Person tagging in English.

13.0

Market Index tagging (em/e/MarketIndex): Overall Quality improvements, plus quality improvements that increase precision by 50% for GDAX (Global Digital Asset Exchange) and SET (Stock Exchange of Thailand).

13.0

Currency Pair tagging: Overall quality improvements that boost precision by 5%.

13.0

The em/e/CurrencyPair (CurrencyPair entity) tag no longer outputs the rcscode attribute.

13.0

Company and Person tagging for Chinese language input is supported for all Intelligent Tagging users. About company tagging, see the Intelligent Tagging for Non-English Languages for details about the reference list of approximately 8000 supported companies.

13.0

Company tagging for German language input is supported for all Intelligent Tagging users. See the Intelligent Tagging for Non-English Languages for details about the reference list of approximately 1700 supported companies.

13.0

Company tagging for Japanese language input is supported for all Intelligent Tagging users. See Intelligent Tagging for Non-English Languages for details about the reference list of supported companies.

13.0

Enhancements to user documentation:

- New JSON output examples throughout the documentation

- RDF output format is documented in a dedicated guide.

- New guide: Intelligent Tagging for Non-English Languages.

- Relevant to internal Refinitiv users only - New Intelligent Tagging documentation page on SharePoint.

13.0

Relevant to internal Intelligent Tagging customers: The x-calais-bodyTag input header is no longer supported.

13.0

Classification – List of supported RCS topics – This list has been moved from the Supplementary Guide for Premium Users to an Excel file to make it easier to search, sort, and filter. The file includes an up-to-date list of all the RCS topics supported by Intelligent Tagging classification, plus several pre-filtered lists of topics, for example, topics optimized for news, topics optimized for research, geographical topics, and so on. Download the list of supported topics from MyRefinitiv. 

12.10

New User Documentation available online on the Developer Community:

API User Guide – New streamlined API User Guide, relevant to all Intelligent Tagging users (internal, external, free, basic, premium, hosted, Azure Cloud, On Premise).

Intelligent Tagging Semantic Metadata Tags -- A new reference guide with detailed information about all supported semantic metadata tags and their attributes. Previously this information was found in the API User Guide.

Input Headers --A new stand-alone guide that describes all supported input headers. Previously this information was found in the API User Guide.

12.10

New Quick Start on the Developer Community.

12.10

New page on the Developer Community, How Does Intelligent Tagging work. Provides a detailed overview of Intelligent Tagging along with running output examples.

12.10

New Python code example in Jupyter Notebook format. Includes simple instructions for installing and working with Jupyter Notebook.

12.10

x-calais-selectiveTags header – new valid values: marketindex (filters for MarketIndex tags); currency (filters for Currency and CurrencyPair tags).

12.9 hotfix

Person tagging – Overall quality improvements to Person tagging for English language input.

12.9

Two attributes of the em/e/Person entity tag are deprecated. The PersonType and Nationality attribute values will now always be “N/A.” This is relevant to English and French language input. Spanish input is not affected.

12.9

x- calais-EnableTickerRecallOriented – A new header for premium users. Triggers recall-oriented company tagging for research content such as analyst emails. (This header is not optimized for documents like long research reports.) This header must be enabled in additio to the x-calais-EnableTickerExtraction header.

12.8

New entity metadata type: Currency Pair (em/e/CurrencyPair tag). (A premium metadata type.)

12.8

Hierarchical tree view of all RCS topics (RCS Eikon News TRBC 2012 View Plus Tree ) is available for download on MyRefinitiv.

12.8

Market Index Entity – enhanced relevance scoring. Improved algorithms provide more accurate relevance scores.

12.6

Intelligent Tagging Demo Tool – New upload folder function.

12.5

Slugline Tagging - New capability which uses Reuters slug lines to classify news documents consistently across multiple sources. Available to premium users.

12.4

Market Index Entity – New confidencelevel attribute of the em/e/MarketIndex tag indicates the probability that the extracted entity is indeed a market index. Values range from 0 to 1. The higher the value, the higher the probability.

The consuming application can use this score to achieve higher accuracy results by ignoring instances with confidence scores below a specified level.

12.4

Intelligent Tagging Demo Tool – Overall improvements. Improved upload function.

12.4

New resource for developers – new Intelligent Tagging space on the Refinitiv Developer Community

12.4

A single API User Guide is now created and maitained for Intelligent Tagging. There is no longer a separate API User Guide for Open Calais (the free, limited version of Intelligent Tagging). The API User Guide is available on the Documentation Tab of the Developer Community Intelligent Tagging space.

12.4

[Update - This guide has been retired. See "New Features" under R13.1 for further information.]

A new supplementary guide for premium users is available for download on the Downloads tab of the Developer Community Intelligent Tagging space.

12.4

A new Intelligent Tagging FAQ is available on the Documentation Tab of the Developer Community Intelligent Tagging space.

12.4

A new Demo Program is available on the Downloads tab of the Developer Community Intelligent Tagging space. This is a simple program, written in Java, that provides a concrete example of how to trigger the Intelligent Tagging API. A Read Me file with instructions is included.

12.4

Quality improvements to ticker extraction (company tagging based on ticker mentions in the text).

R12.3

Significant quality improvements to the Market Index Entity (em/e/MarketIndex metadata tag).

R12.2

x-calais-DocumentTitle- A new header you can use to specify the title of the document, to optimize tagging output for text files.

R12.2

Intelligent Tagging Demo Tool in the Local Management Console now lets you preview Slugline tagging. Previously, the Slugline tagging capability could only be previewed with the Intelligent Tagging Demo Tool on PermID.org.

R12.2

List of RCS classification topics in the API User Guide now includes descriptions.

R12.2

Intelligent Tagging Demo Tool – Enhancement - When Document Type: News is selected, an XML template is displayed, making it simple to submit text in the format that returns the best tagging results.

R12.1

API User Guide – Sections that describe the PermID and ForEndUserDisplay attributes have been moved to Chapter 4: Semantic Metadata Tags.

R12.1

API User Guide – New section – Metadata Tags that are Actively Enhanced and Supported.

R12.0

Intelligent Tagging Demo Tool - New ability to trigger either basic or premium processing. Overall improvements.

R12.0

Intelligent Tagging Demo Tool - New support for uploading multiple files to the Demo tool for tagging or viewing. Overall improvements.

R11.9

RIC extractions – New company extraction based on RIC (Reuters Instrument Code) mentions in the text increases company tagging coverage. This functionality is enabled for premium users.

R11.8

x-calais-EnableTickerExtraction –New header that triggers company extraction based on ticker mentions in the text. Available to premium users.

R11.8

em/e/Company – new “recognizedasattribute indicates if the company extraction is based on a company RIC, a company ticker, or a company name being found in the text.

R11.8

Tagging vertical text in charts –Intelligent Tagging now applies metadata tags to vertical text in charts in PDF files.

R11.8

New article available on Dev Portal: A Practical Approach to Understanding and Ingesting TRIT Output for Your Use Case

R11.8

API UG- New appendix that describes the PermID attribute.

R11.7

Supported RCS topics list now available on MyRefinitiv in CSV format, making it possible to download the (regularly updated) RCS topic list programmatically.

R11.6

New company tagging algorithm improves the quality of company resolution.

R11.5

x-calais-suppressSocialTagsFin header - updated list of social tags that are suppressed when this header is set to True. Enables more focused Social Tagging for financial documents.

R11.5

Intelligent Tagging Demo Tool – new support for processing text in tables.

R11.3

API User Guide – updated list of supported sources.

R11.3

Abstraction Layer Developer Guide – additional code examples.

R11.3

On Premise installation now requires a minimum of 61GB RAM

R11.3

Research Reports – Improved algorithms for RCS topic classification of research reports.

R11.2

x-calais-source header – Optimized extraction for two new sources: Fitch Financial Services; BMI Research.

R11.2

Social Tags – A new header , x-calais-SuppressSocialTagsFin. Used to exclude from the tagging output specific generic social tags that do not add value to financial users. Recommended for use when tagging financial documents.

R10.8

API User Guide – Added the list of sources (news providers and investment banks) supported by the x-calais-source header.

R10.8

DocCat (Topic) tags – new shortName attribute.

R10.8

New permid attribute for metadata tag types. The PermID can be used when building a knowledge graph.

R10.8

Intelligent Tagging Demo Tool – enhancement – RCS codes no longer displayed next to topic names.

R10.6

New openpermid attribute of the er/Person (person resolution) tag gives you direct access to high-quality, curated Refinitiv People data.

R10.5

New ispublic attribute of the er/Company (company resolution) tag and the er/TopmostPublicParentCompany tag indicates whether the company is public (true) or private (false).

R10.5

Intelligent Tagging Demo Tool supports input files of up to 5MB. (Relevant to hosted Intelligent Tagging.)

R10.5

Improved tagging quality for PDF documents.

R10.4

Improvements to the Intelligent Tagging Demo Tool: Overall improvements, bug fixes, new support for optimized tagging of research documents.

R10.4

Overall improvements to Social Tagging.

R10.3

HTTPS proxy server supported for data updates.

R10.3

New x-calais-source input header.

R10.3

Improved company extraction.

R10.2

Social tags – overall quality improvements.

R10.2

Improvements to performance and stability.

R10.2

Social tags – overall quality improvements.

R10.1

Improvements to performance and stability.

R10.1

New Intelligent Tagging Viewer (aka Calais Viewer, aka Intelligent Tagging Demo Tool)

R10

(Relevant to On Premise users) The Docker Engine on AWS installation procedure was successfully tested on a smaller AWS service: r3.2xlarge with 8 CPU, 61 GB RAM, and 1 x 160 (SSD). (Installing on a smaller AWS service reduces hosting fees.)

R10

New x-calais-pdfTagZone header enables tagging to tables in PDF documents.

R9.9

Social tags – overall quality improvements.

R9.9

x-calais-socialTagsImportanceThreshold: New header enables excluding from the output, social tags with importance scores below a specified threshold.

R9.7

x-calais-socialTagsResultSize: New header enables limiting the total number of social tags in the output.

R9.7

API User Guide – new appendix provides a list of synonyms for Enty Relation Types.

R9.7

(Relevant to On Premise users) New disaster recovery license.

R9.7

(Relevant to On Premise users) Availability of On Premise Basic and Premium packages

R9.6

(Relevant to Hosted Intelligent Tagging) The tagging output includes new HTTP response headers.

x-permid-quota-daily: Indicates the daily quota defined by your license.

x-permid-quota-used: Indicates the number of submissions already made.

R9.5

 

 

Known Issues and Fixed Issues

The following table provides a list of known issues since R9.5, along with the release version in which the issue was fixed, when relevant.

Issue - Relevant to both On Premise and Hosted Intelligent Tagging Unless Specified Release in which issue resolved
PDF input - InstanceInfo tag - Offset attribute: For PDF input, the offset values of extracted instances may be incorrect. For example, the offset values for phrases extracted from tables and charts may be incorrect. The offset values for phrases extracted from highlighted text are consistently incorrect. (ETRIT-1589  

Slugline tagging: Currently slugline tagging is triggered by default. Slugline tagging should be triggered only if the x-calais-useSlugline header is in use or the x-calais-selectiveTags:slugline parameter is passed. (ETRIT-526).

Please note that if you use the x-calais-selectiveTags header without defining the "slugline" value, then slugline tagging is turned off, and slugline tags will not be present in the tagging output. In other words, implementing the x-calais-selectiveTags header resolves the issue.
 
Product tagging: If you use the x-calais-selectiveTags header to trigger Product tagging, then you must also trigger PharmaceuticalDrug tagging for best results. (ETRIT-622)  
PharmaceuticalDrug tagging: If you use the  x-calais-selectiveTags header to trigger PharmaceuticalDrug tagging, then you must also trigger Product tagging. Otherwise the ownerPermID attribute will be missing from the PharmaceuticalDrug tag. (ETRIT-578)  
x-calais-selectiveTags header: The currency value triggers both Currency and CurrencyPair tagging. CurrencyPair is not a supported header value.

 

x-calais-selectiveTags header: Currently, you cannot use the x-calais-selectiveTags header to output these metadata types: MedicalCondition (ETRIT-625);  MedicalTreatment (ETRIT-626); PoliticalRelationship (ETRIT-659); PollsResult (ETRIT-660); VotingResult (ETRIT-663).

 

Local Management Console (On Premise Users) – the Configure Entity Tuning Server option on the Administration tab is not currently in use.

 

Complex input documents may cause timeout errors. Timeout errors may be generated if an input document is too complex (contains too many entities and relations) to be processed within the defined time limit. In this case, try splitting the document into smaller parts for processing.

 

Social Tagging - an issue relevant to On Premise users only – There is no automatic data update mechanism for the Wikipedia data set (the reference set of topics used by Social Tagging). Thus Social Tags are not extracted for current events. Each new release of Intelligent Tagging includes an updated Wikipedia data set.

 

Industry Tag – In some instances, the trbccode attribute of the Industry tag exposes a more specific TRBC code than it did in R10.1 and earlier releases. For example, instead of getting the TRBC code for Advanced Medical Equipment, you might get the TRBC code for Laser Equipment (a specific type of Advanced Medical Equipment). (CU-2952)

 

Currency tagging: This issue is relevant to non-English language tagging, and to internal Intelligent Tagging customers using the CalaisDirect profile. The permid attribute of the Currency entity tag is exposing the PermID of the Currency metadata type instead of the PermID of the extracted currency. (ESINGAPORE-911)

R13.4

Currency entity is not output. This is a temporary issue which will be resolved soon.  R13.3

Slugline tags are missing from JSON output. (ETRIT-77)

R13.0

er/Person (person resolution ) tag – the id attribute is missing from the JSON output. The id attribute value is a link to the relevant person page on Open PermID. Workaround—use the PAID attribute value to form the link to the relevant person page on Open PermID. (ETRIT-219)

R13.0

Market Index Tagging: Two key fixes that increase precision by 50% across a select number of classes. (ESINGAPORE-198; ESINGAPORE-162; ESINGAPORE-144; ESINGAPORE-213-217)

R13.0

Currency Pair Tagging: A number of fixes improve precision by 5% across all classes. (ESINGAPORE-220; ESINGAPORE-205)

R13.0

Sending requests with the SuppressSocialTagsFin header cause all processing requests to fail with error HTTP 500.

R12.10

FANG index incorrectly identified as New York Financial Index.

R12.7

STI index incorrectly identified as SET. (TMSSIN-1386)

R12.7

Quality issues related to Person tagging in French language input. Person extraction may not be done properly on HTML and PDF files, and on XML and text files larger than approximately 10KB.

R12.4

RIC attribute of em/e/MarketIndex tag – missing period at the beginning of the RIC.

R12.2

Disable Data Updates function not working properly. Upgrading to R11.8 resolves the issue. (TMSCAL-5216)

R11.8

Duplicate Data in the JSON output of On Premise users (TMSCAL-5042)

R11.8

TopmostPublicParent value missing from the JSON output of On Premise users. (TMSCAL-4939)

R11.8

Company extraction - confidencelevel attribute missing from the em/e/Company tag in the tagging output.

R11.3

Logs generated by the On Premise Local Management Console are corrupted. (TMSCAL-4134)

R11.3

Intelligent Tagging Demo Tool –On the results page, incorrect PermIDs displayed in the Person details popups. (TMSCAL-3815)

R11.1

If you activate a license while the API is initializing, the tagging mechanisms are not always fully functional. From R11.0, you are automatically blocked from activating a license key when the API is initializing. (TMSCAL-3813)

R11.0

Local Management Console (On Premise users) – Admin Tab –The Component Last Update Times are not consistently displayed according to the time zone defined by the client machine.

R10.8

Intelligent Tagging Demo Tool - When you hover over a highlighted entity mention, the popup dialog should indicate the total number of times this specific entity is mentioned in the text and indicate which one you are hovering over. These numbers are not always correct. (CU-2862)

R10.5

Intelligent Tagging Demo Tool - When you hover over a highlighted entity mention, left and right arrows appear in the popup dialog, enabling you to scroll through the entity mentions. This function does not always work properly. (CU-2862)

R10.5

(Relevant to On Premise users) Local Management Console - Home page – Intelligent Tagging Demo: Instances of entities and relations are not always highlighted properly in the text. (CU-2863)

R10.4

(Relevant to On Premise users) Local Management Console - Home page – Intelligent Tagging Demo: When you select/unselect the check box to the left of the Entities heading, all of the entities in the list are selected/unselected. Note that although the Top Mentioned Entities are indeed selected/unselected as expected, the check box to the left of Top Mentioned Entities does not reflect the change.

R10.4

(Relevant to On Premise users) Local Management Console - Home page – Intelligent Tagging Demo: Sometimes the Top Mentioned Entities section is missing from the Found In Document pane. (CU-2953)

R10.4

On Premise License Activation issue – sometimes it”s necessary to restart the service in order to activate a new or updated license. (On Premise only)

R10.4

(Relevant to On Premise users) Local Management Console - Home page: If you paste text into the Intelligent Tagging Demo window and click TAG IT, tagging results may not be consistent with a real production environment. Currently the Demo converts the pasted text into text/xml format for processing, and applies the x-calais-contentClass:news header.

R10.4

Company Tagging – a company tagging issue may prevent entire documents from being tagged. Installing or Upgrading to version 10.2 or late resolves the issue. (Relevant to On Premise users)

R10.2

(Relevant to On Premise users) Ticker attribute not output for the following metadata tags: er/company; er/TopmostPublicParentCompany. (CU-1827)

R10.2

(Relevant to On Premise users) The table that appears on the Admin tab of the Local Management Console displays a list of the components that have been updated and the last update time. When you restart the server, some of the data disappears from the table. (CU-2088)

R9.6