We’ve just (re-)launched our named entity recognition service. This builds on the NICTA Named-Entity-Recogniser work of Scott Sanner, Kishor Gawande, William Han, Paul Rivera and Kin Hon Chan.
The new version has been improved by Mats Henrikson, and is open sourced under the GPLv3.0. The public web page demo is hosted on Amazon at http://ner.t3as.org
We’ve just made a new version of the patent classification service live.
Changes are:
We’ve now done some extensive browser testing for the web-demo of the patent classification service. The web-demo works on
The service itself is agnostic to browsers.
As IE8 doesn’t support manipulation of XML elements embedded in HTML (which are used extensively by the demo) IE8 users will need to wait for an update – either from us, or by downloading a recent version of IE.
Now with configuration options!
We’ve just launched the latest update for our SnoMed-CT text coder We now allow users to select options to choose the semantic types.
SnoMed coder with options
At present we are using a heuristic to filter the semantic types that MetaMap returns to restrict the hits to clinical terms. We will improve the heuristic over time.
The new feature allows the user to select which semantic types are used by MetaMap. Our defaults (the heuristic defaults) are listed, but perhaps you really want to include a search for codes that match fish. So then, press the “configure button, select the semantic type “fish” and re-analyse the text.
As this is a demo system, we don’t store cookies: so if you open new browser window, you’ll have to re-configure
We’ve released our work on Snomed-CT text analysis, our first step is a public web service that can analyse English text and report any SNOMED CT concepts that are detected. The service is available at http://snomedct.t3as.org/
The service uses the National Library of Medicine’s software product MetaMap and the National Library of Medicine’s Unified Medical Language System (UMLS). The screenshot below is generated from the web page, using the input text of Figure 3, of Sager et al. Automatic encoding of Snomed III
Snomed Coder Screenshot
If you would like to try out some other clinical texts, you might try the Medical History texts from Monash University.
Please note: This demonstration service runs on a publicly accessible server that is not geographically constrained. All text entered in the web page is sent in clear text to the Amazon service. Please do not upload private clinical documents.
The page is a simple front-end for the public web service that can analyse English text and report SNOMED CT concepts that are detected. At present we are using a heuristic to filter the semantic types that MetaMap returns to restrict the hits to clinical terms. We will improve the heuristic over time.
The next feature we add will be to also display the semantic type of each returned concept to feed into a system that allows the user to view/search-over specific types with some defaults set.
For developers, have a look at the service page to find out how to use the service as an API. We will shortly be releasing the source code as well.
SNOMED CT or SNOMED Clinical Terms is a systematically organized computer processable collection of medical terms providing codes, terms, synonyms and definitions used in clinical documentation and reporting. Wikipedia
More information on the Australian terminology SnoMed CT-AU can be found at the NEHTA site.
MetaMap is the most common tool in clinical text analysis applications to codify and relate medical concepts. We found only 1 hit on PubMed for the similar systems of Mgrap, and 12 hits for KnowledgeMap. There were 74 hits for MetaMap. MetaMap has resulted in substantial performance gains. For example,
With thanks to Mats Henrikson, Hanna Suominen and Neil Bacon
The USPTO, CPC and IPC have each recently updated their classification systems. We have updated our underlying classification data:
This brings the patent classification API up to date with current international classification documentation.
In this project, we developed a searchable index of patent classification codes that allows search by text and by code. We also extended this to allow users to explore the classification hierarchy. This blog entry page describes the demonstration web page.
Patent Classification pat-clas.t3as.org
For those unfamiliar with reading patents, we refer to The Lens and the tutorial, how to read a patent. Within patents, classification codes provide significant benefit in understanding and searching patents: patents with similar codes are likely to refer to similar content. From the British Library emphasis added
“The usefulness of patent classification as a means of searching for patents information is a by-product of its primary purpose as a tool for patent examiners. Using patent classification as part of a search to identify patents in a particular field can help the non-expert searcher to focus and refine his search and produce a useful set of references… However, it is a massive and complex tool designed for an expert user group and when it is used by anyone outside that user group it should be applied with care.”
For the non-expert, classification codes are difficult to use. For example, a patent for locomotive on the Lens, has two IPC classification codes associated with it: B61C17/04 and B61D27/00. What do these codes mean?
The text analytics service for this project is hosted at pat-clas.t3as.org and has a public Github repository for all code. The web page provides an open html file, that accesses the service API’s and presents a simple interface for text- or code- based search of CPC, IPC, and USPTO classification codes.
Free text search
The first field of the web page allows users to enter free text, and return classification codes. Following our example, let’s choose IPC, and enter “locomotive”. The search returns all IPC codes that contain “locomotive” in their text. The list is sorted based on relevance (rank), with all relevant search terms highlighted in the codes. The search returns at most the top 50 items.
The next field allows the user to find the context of a given code. Let’s try B61C17/04:
Code context
The context is build up of the the parent codes in the classification hierarchy, with their associated text stubs. Explore can also be used to view the hierarchy of the classification system – for example to find siblings of the code B61C17/04.
Classification hierarchy for Locomotive
The screen flow below outlines how to use the web page.
All the code is available on Public GitHub. If you are interested in developing applications that use this code, you should read the README on GitHub.
This is a very simple database app with XML processing to populate the database.
We’ll post more details soon.
This combines work from Neil Bacon, Gabriela Ferraro and Mats Henrikson
Welcome to the Text Analytics as a Service (t3as) blog.
We want to deliver an open ability in Text Analysis to end users via open source, web-based services. Text Analytics is a significant bottleneck for data analysis: analytics of unstructured text is needed as text data remains largely unused outside research projects.
Text Analytics is an artisanal (or cottage) industry that does not yet lend itself to engineered processes. Text analytics lacks the standardisation required to deploy technology solutions composed of “off the shelf” components.
We want to build a framework that delivers the benefits of open-source text analytics, whilst overcoming the barriers of open data analysis.
Text Analysis as a Service will support multiple sub-projects. The services are hosted on Amazon Web Services, with domain www.t3as.org Projects will deliver:
We maintain a private GitHub repository. Projects that reach an appropriate level of maturity, including NICTA open source requirements, will be transitioned to Open Source, under GNU General Public License v3 (GPLv3) and public GitHub
These API’s will be useable without commercial license. The hosted API’s are designed to deliver trusted code, but not necessarily high bandwidth. The API’s are versioned, and will support remote interrogation by other services: this allows composition of our web services. If/when load becomes an issue we will develop a free API key access that supports monitoring and potentially limits usage per user.
We will host web pages that wrap the API’s to show the services. We expect the web page to give a tutorial in the API calls, as well as allowing developers to duplicate the web page, and experiment with API calls, before developing their own software that calls the API
The services work in both remote (as a server-side application) and local (service deployed in-process within the user’s application) modes.
Text analytics as a service 