AEM Integration with Coveo ‑ The High Level Approach

Search is considered one of the major features of a website used to retrieve content with speed and relevance. There are multiple frameworks in the search space which are either paid or open-source. Many of these frameworks support different approaches to integrate search with websites. 

We will understand more about what steps we have done to integrate a search framework with AEM.

Steps Involved in Search Integration with Websites 

1) Selecting The Search Tool

There are various search frameworks available, either open source or paid. The most used open-source search engines are Solr and Elasticsearch. The paid category list contains leaders like Coveo Relevance Platform, Lucidworks Fusion, IBM Watson Discovery, Sinequa ES, etc. The considerable factors for choosing a search engine mainly relies on the below factors.

• Open source vs paid
• Flexibility of the tool
• Integration capability like extensive API support
• Learning curve & development team capability
• Horizontal & vertical scaling
• Clustering through Cloud
• Maturity & community support
• Rich full text query syntax
• AI/ML Features
• Performance & Administrative abilities

2) Setting Up The Search Engine and Indexing Options 

Once we identify the tool, the next step is to plan for additional capabilities to reap the complete benefits provided. We need to ensure the below factors are discussed with stakeholders and taken an appropriate decision before setting up the system.

• Hosting: This depends on the search tools hosting capabilities (on premise vs cloud)
• Indexing: Indexing the data based on requirements; Almost all frameworks support customized indexing of content
• Index frequency: How often we index the items  (e.g. Daily, Monthly)

3) Configuration 

Basic configuration is necessary for smooth indexing of the content. Below are the not-to-be-missed configurations.

• Metadata: Metadata is used for faceting, sorting, ranking, relevance
• Breadcrumbs & Pagination: For a better page navigation
• Sorting & Facets: Facets & Sorting provides better refinements on search results; Recent Searches, Search Suggestions (Did you mean), Auto Complete - These are the basic search assist capability provided by frameworks
• AI/ML Configurations: Modern search tools utilize the user behaviour and patterns to provide better results based on Artificial intelligence (AI) & Machine Learning (ML) capabilities

4) Fine-Tuning Results

As the last step, once the search results are stable, we need to fine-tune the search by analyzing the results further. ‘Quality of search’ - is the way of improving the search results, by fine tuning the relevancy, applying analytical and artificial intelligence capabilities etc. Some of the options provided by the search frameworks are given below.

• Promotion & Banners: Based on the configurations, we can promote the results
• Dictionaries & Synonyms: Dictionaries help us to configure how terms are stored in the suggestion dictionary, where in a synonym is a user configured word or phrase which means exactly(or nearly the same thing) as another word or phrase
• Redirects: Based on a criteria or search term, users can be directly redirected to a specific page

Our AEM Framework Search Capability 

Bounteous developed an Activate framework which provides digital marketing offering around Adobe Experience Cloud. This framework minimizes the cost and time it takes to launch digital customer experiences with Adobe Experience Manager, Adobe Analytics, and Adobe Target. As a native AEM option, Activate has the search capability which utilises AEMs built-in lucene search.

Our Decision on Search Integration with AEM Framework 

We get some of the advantages when we separate search out of AEM’s boundary. For example, on sites with a huge volume of content pages the search may degrade the performance of the entire AEM. Also the third party searches provide many additional capabilities built-in like faceting, sort options, paginations, auto suggestions, AI/ML capabilities etc.

We have decided to integrate AEM with Coveo as a first entry into Activate’s non-native search capability. The decision factors were around non-open source cloud hosted search with advanced indexing features and result fine tuning abilities.

Why Did We Choose Coveo?

Coveo is a software-as-a-service (SaaS) search engine which provides intelligent search, delivering the most relevant results possible from a potentially large volume of data. Coveo is also one of the notable leaders in Gartner’s magic quadrant consistently for the past few years.

Our requirements were around flexible ways of indexing which includes REST API support, cloud hosted solution which has the capability to utilise the AI/ML features.

In the section below we will cover high level concepts of a Coveo search integration with AEM CMS.

AEM Integration with Coveo

There are multiple ways to integrate the AEM with Coveo. The commonly used approaches are AEM connector provided by Coveo and generic connector like sitemap.

Generic Way of Content Index Within Coveo

Coveo search supports various index options to facilitate smooth indexing of content based on various technical requirements. 

Crawler Based Indexing

Coveo provides a module which scans content to index,  extracts their contents and indexes within itself.

Sitemap Based Indexing

The Sitemap connector helps to index web pages from a Sitemap. The connector also supports authentication methods for secured websites. The supported formats are XML, Text, RSS 2.0, and Atom 1.0.

Push API

Assume we need to customize the indexing to ensure only defined data gets indexed. In this case the Push APIs give more flexibility to control the indexing.

The Push API, exposes a set of services which allow us to push items and their permission models into a source, and security identities into a security identity provider. But there are certain limits on request size, request frequency, and on security identities with respect to push mechanisms.

Indexing The Content from AEM to Coveo - AEM Connector 

Since we are working for an AEM based indexing, we have chosen to use the AEM Connector for content indexing.

AEM connector is a tool which enables easier content indexing from AEM to Coveo. This tool comes as a package which can be installed through the AEM’s package manager. Once installed, we will have to configure a set of workflows to enable the various indexing within AEM. 

The installation & configuration details can be found here

With minimal configurations, Adobe Experience Manager page based activities like publishing, un-publishing, and deleting pages automatically trigger indexing within Coveo.

Authentication & Content Retrieval from Coveo to AEM

The commonly used authentication methods are API key authentication (public & private) and search token authentication. Each configured API can be restricted or enabled to access any section of the resource, which gives fine-tuned control on the operation allowed within Coveo.

For the integration with AEM, we recommend the JWT based authentication which ensures the protected search.

Coveo supports various Search APIs to interact between the website hosted system to retrieve the content through a predefined authentication mechanism. These APIs support content retrieval by passing a few sets of query parameters and providing a huge variety of results which includes Facets, Sort, Auto Suggest, Pagination etc.

A complete swagger search API list can be found here.

Coveo Delivers The Results

Our decision on choosing Coveo as a search framework never disappointed us. As outlined in the previous sections, Coveo has given a lot more flexibility compared to some of the open source frameworks with respect to results grouping, analytical capability, AI/ML features and report dashboards. Coveo definitely has a learning curve, but we get more features when we explore and understand the tool better and search can be blazing fast hereafter.