Content Tagging using Sitecore Cortex and Azure Machine Learning

Imagine having to dedicate content resources to read/edit/proofread the content manually line by line. And, imagine the plight of a content writer or developer when the volume of content increases in numbers on a daily basis. In the 21^st century, marketers are continually moving towards the age of automation where Artificial Intelligence(AI) and Machine Learning(ML) are becoming the best friends of the wealth of data collected by the xDB (Experience Database).

There are explicit benefits to having xDB. Historical data is important to create a full-fledged personalization experience in Sitecore that targets every stage of the customer journey.

Sitecore 9.1 introduced Sitecore Cortex^TM that enables implementation of ML/AI–based technology by any product/customer on experience data in the xDB. In version 9.1 it provides Content Tagging Out of Box that tags content items based on the content of the item. The Sitecore Cortex Content Tagging feature integrates the Sitecore CMS with Machine Learning (ML) based Natural Language Processing (NLP) engines. By default it provides integration with OpenCalais.

Did you know: Open Calais is a sophisticated web service that attaches intelligent metadata-tags to your unstructured content. The Open Calais natural language processing engine automatically analyzes and tags your input files.

Auto-Tagging Setup Using Cortex

Let’s understand what was missing with the default Sitecore Cortex Content Tagging feature-

Open Calais NLP Limitations: Sitecore Cortex takes the information from all the fields present in the item, which is sent to the third party where the Open Calais NLP Algorithm extracts the tags from the data and returns back to Sitecore.

This facility wasn’t really working in favor of the developers, as Sitecore Cortex was giving us the data or tags from all the fields (as against the specific fields) that are not even close to the context of the content. There used to be a lot of manual work at any developer’s end to implement tagging in all older versions.

Irrelevant Tags shown in Sitecore Cortex

Why Azure ML

Having faced the above-mentioned limitations of Sitecore Cortex Content Tagging, we have strategically overwritten two content pipelines - the retrieve content pipeline and get tags pipeline.

• We collected the tokens from the data and removed stoppers. Stoppers are the most frequently used words such as ‘and’, ‘the’, etc.
• Extracted Keywords – words that help us differentiate one item from another
• Output the tags