{"id":96,"date":"2024-11-08T10:00:00","date_gmt":"2024-11-08T16:00:00","guid":{"rendered":"https:\/\/douglasstarnes.dev\/?p=96"},"modified":"2024-11-04T23:43:23","modified_gmt":"2024-11-05T05:43:23","slug":"azure-ai-language-service-named-entity-recognition","status":"publish","type":"post","link":"https:\/\/douglasstarnes.dev\/index.php\/2024\/11\/08\/azure-ai-language-service-named-entity-recognition\/","title":{"rendered":"Azure AI Language Service: Named Entity Recognition"},"content":{"rendered":"\n

In the previous post<\/a> on the Azure AI Language, you saw how to provision and configure an instance of the Azure AI Language service and a simple example of using sentiment analysis and key phrase extraction. In this post we will look as another feature of the Language service, Named Entity Recognition or NER.<\/p>\n\n\n\n

The NER feature of the Language service detects notable entities in a body of text such as people, places and organizations. For example, Satya Nadella would be an example of a person recognized by NER while Microsoft would be an example of an organization. In total, NER can recognize 14 different entity categories. A complete list can be found here<\/a>.<\/p>\n\n\n\n

The prerequisites for using the NER feature are the same as using the sentiment analysis feature. You need to provision an instance of the Language service and get the endpoint and keys to create a TextAnalyticsClient<\/code> that you will use to communicate with the Language service. So we’ll start from there.<\/p>\n\n\n\n

Using the NER feature<\/h2>\n\n\n\n

As I said in the previous post about the Language service, most of the “hard work” of getting a prediction from an AI model is reduced to a single line of code. The NER feature is no exception. Once you have a client, simple call the recognize_entities<\/code> method and pass it of a list of strings, which will be referred to as documents<\/em>. For example, let’s use this short passage about interesting things in Nashville, Tennessee. (courtesy ChatGPT)<\/p>\n\n\n\n

\n

Nashville, Tennessee, is often known as “Music City,” a title that rings true thanks to its rich history and vibrant cultural scene. While Nashville has become synonymous with country music, the city is also celebrated for its notable residents, groundbreaking music venues, and contributions to art, education, and civic life.<\/p>\n\n\n\n

Nashville\u2019s story is one of creativity, resilience, and tradition. From the legacies of Dolly Parton and Andrew Jackson to the lively scenes on Broadway Street, Nashville continues to attract visitors and inspire artists worldwide. Whether you’re a music lover, a history buff, or simply someone looking to experience a slice of Southern culture, Nashville offers a warm welcome and countless stories to uncover.<\/p>\n<\/blockquote>\n\n\n\n

Assuming a TextAnalyticsClient<\/code> instance named client<\/code>, the single line of code that does the “hard work” is this.<\/p>\n\n\n\n

<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
results <\/span>=<\/span> client<\/span>.<\/span>recognize_entities<\/span>(<\/span><\/span>\n  <\/span>documents<\/span>=<\/span>[<\/span>nashville_document<\/span>]<\/span><\/span>\n)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n
The return value will contain one result for each document.  Inside each result is a entities<\/code> attribute.  Each entity contains data such as the entity text<\/code>, the category<\/code> and a confidence_score<\/code>.<\/p>\n\n\n\n
<\/circle><\/circle><\/circle><\/g><\/svg><\/span><\/path><\/path><\/svg><\/span>
entity_categories <\/span>=<\/span> <\/span>{}<\/span><\/span>\n  <\/span>for<\/span> entity <\/span>in<\/span> document<\/span>.<\/span>entities<\/span>:<\/span><\/span>\n    <\/span>if<\/span> entity<\/span>.<\/span>category <\/span>not<\/span> <\/span>in<\/span> entity_categories<\/span>:<\/span><\/span>\n      entity_categories<\/span>[<\/span>entity<\/span>.<\/span>category<\/span>]<\/span> <\/span>=<\/span> <\/span>[]<\/span><\/span>\n    entity_categories<\/span>[<\/span>entity<\/span>.<\/span>category<\/span>].<\/span>append<\/span>(<\/span><\/span>\n      <\/span>{<\/span><\/span>\n        <\/span>"<\/span>text<\/span>"<\/span>:<\/span> entity<\/span>.<\/span>text<\/span>,<\/span> <\/span><\/span>\n        <\/span>"<\/span>confidence_score<\/span>"<\/span>:<\/span> entity<\/span>.<\/span>confidence_score<\/span><\/span>\n      <\/span>}<\/span><\/span>\n    <\/span>)<\/span><\/span><\/code><\/pre><\/div>\n\n\n\n
Here is an excerpt of the entities extracted:<\/p>\n\n\n\n
{\n    \"Location\": [\n        {\n            \"text\": \"Nashville\",\n            \"confidence_score\": 1.0\n        },\n        {\n            \"text\": \"Tennessee\",\n            \"confidence_score\": 0.98\n        }\n    ],\n    \"PersonType\": [\n        {\n            \"text\": \"residents\",\n            \"confidence_score\": 0.88\n        }\n    ],\n    \"Skill\": [\n        {\n            \"text\": \"art\",\n            \"confidence_score\": 0.91\n        }\n    ],\n    \"Quantity\": [\n        {\n            \"text\": \"one\",\n            \"confidence_score\": 0.8\n        }\n    ],\n    \"Person\": [\n        {\n            \"text\": \"Dolly Parton\",\n            \"confidence_score\": 1.0\n        },\n        {\n            \"text\": \"Andrew Jackson\",\n            \"confidence_score\": 1.0\n        }\n    ],\n    \"Address\": [\n        {\n            \"text\": \"Broadway Street\",\n            \"confidence_score\": 0.94\n        }\n    ]\n}<\/code><\/pre>\n\n\n\n
The NER feature found entities in 6 different categories:<\/p>\n\n\n\n