How to add a search index

This article shows how to add a new search index to the project.

To setup a search index that can be used for searching the developer needs to implement a configuration, document, index document builder and event listener.
The search and indexing is using the NEST library that is maintained by the company behind Elasticsearch as an Open Source Project.

Document
The document is the part that describes what properties Elasticsearch should handle. The document will be used both for index and searching. All documents should inherit the marker interface Litium.Search.IDocument.
When creating the document, you create properties with the type of data that you want to use for index or searching. The document can support child and nested collections to support advanced search scenarios. For documentation about Nest and how to build and maintain documents you can read more in the documentation on https://www.elastic.co/

Configuration
Each index can be configured in either no lingual or multilingual. If configured to use multilingual an index in ES will be created for each language that you have in your installation.
The configuration is also responsible to be able to trigger a full rebuild of the search index, this is useful if you change the document or the creation of the document for indexing.
The following base classes will take care of the creation of the Nest.CreateIndexDescriptor that is used to configure the index.

  • No lingual index - creates an index in ES your configuration class should inherit Litium.Search.Indexing.IndexConfigurationBase.
  • Multilingual index - creates a multilingual index in ES your configuration class should inherit Litium.Search.Indexing.MultilingualIndexConfigurationBase.

The base classes have two methods that can help to configure how Elasticsearch will handle the different properties during index and searching,

  • BuildIndexDescriptor - configures global settings for the index like analyzers;
  • BuildTypeMapDescriptor  - configures the properties and how they should be mapped together with the attribute mapping that Nest supporting.

Index Document Builder
Index document builder implements either IndexDocumentBuilderBase or MultilingualIndexDocumentBuilderBase depend on if the index should be multilingual or not.
Both implementations are working in the same way and converting the Queue item into documents, either for index or deletions.

Event Listener
The purpose of the event listener is to convert the events that exist in Litium to different queue items that Elasticsearch should index.
The event listener should implement the marker interface Litium.Search.Indexing.IIndexQueueHandlerRegistration and use the Litium.Search.Indexing.IndexQueueService for adding items to the indexing queue.

Let say we want to search organizations on a site that exists in the system.
Next classes must be added to the Litium.Accelerator.Elasticsearch application:

Organization structure.PNG

1. OrganizationDocument

using System;
using System.Collections.Generic;
using Litium.Search;
using Nest;

namespace Litium.Accelerator.Search
{
    public class OrganizationDocument : IDocument
    {
        [Keyword(Ignore = true)] public string Id => OrganizationSystemId.ToString();

        public Guid OrganizationSystemId { get; set; }

        public ISet<string> Content { get; set; }

        public string Name { get; set; }
    }
}

2. OrganizationIndexConfiguration, it will be no lingual index.

using System.Threading.Tasks;
using Litium.Customers;
using Litium.Data;
using Litium.Search;
using Litium.Search.Indexing;
using Microsoft.Extensions.Localization;

namespace Litium.Accelerator.Search.Indexing.Organizations
{
    public class OrganizationIndexConfiguration : IndexConfigurationBase<OrganizationDocument>
    {
        private readonly DataService _dataService;
        private readonly IStringLocalizer _localizer;

        public OrganizationIndexConfiguration(IndexConfigurationDependencies dependencies, DataService dataService, IStringLocalizer<IndexConfigurationActionResult> localizer)
            : base(dependencies)
        {
            _dataService = dataService;
            _localizer = localizer;
        }

        protected override Task<IndexConfigurationActionResult> QueueIndexRebuildAsync(IndexQueueService indexQueueService)
        {
            using (var query = _dataService.CreateQuery<Organization>())
            {
                foreach (var systemId in query.ToSystemIdList())
                {
                    indexQueueService.Enqueue(new IndexQueueItem<OrganizationDocument>(systemId));
                }
            }

            return Task.FromResult(new IndexConfigurationActionResult
            {
                //Need to add "index.organizations.queued" to Administration.resx
                Message = _localizer.GetString("index.organizations.queued")
            });
        }
    }
}

3. OrganizationIndexDocumentBuilder

using System;
using System.Collections.Generic;
using System.Globalization;
using Litium.Accelerator.Indecies;
using Litium.Customers;
using Litium.Search;
using Litium.Search.Indexing;

namespace Litium.Accelerator.Search.Indexing.Organizations
{
    public class OrganizationIndexDocumentBuilder : IndexDocumentBuilderBase<OrganizationDocument>
    {
        private readonly OrganizationService _organizationService;
        private readonly ContentBuilderService _contentBuilderService;

        public OrganizationIndexDocumentBuilder(IndexDocumentBuilderDependencies dependencies, OrganizationService organizationService, ContentBuilderService contentBuilderService) : base(dependencies)
        {
            _organizationService = organizationService;
            _contentBuilderService = contentBuilderService;
        }
        public override IEnumerable<IDocument> BuildIndexDocuments(IndexQueueItem item)
        {
            var organization = _organizationService.Get(item.SystemId);
            if (organization == null)
            {
                yield break;
            }

            yield return new OrganizationDocument
            {
                OrganizationSystemId = organization.SystemId,
                Content = _contentBuilderService.BuildContent<OrganizationFieldTemplate, CustomerArea>(organization.FieldTemplateSystemId, CultureInfo.CurrentUICulture, organization.Fields),
                Name = organization.Name,
            };
        }
        public override IEnumerable<IDocument> BuildRemoveIndexDocuments(IndexQueueItem item)
        {
            yield return RemoveByFieldDocument.Create<OrganizationDocument, Guid>(x => x.OrganizationSystemId, item.SystemId);
        }
    }
}

4. OrganizationEventListener

using Litium.Events;
using Litium.Search.Indexing;
using Litium.Customers.Events;

namespace Litium.Accelerator.Search.Indexing.Organizations
{
    public class OrganizationEventListener : IIndexQueueHandlerRegistration
    {
        public OrganizationEventListener(EventBroker eventBroker, IndexQueueService indexQueueService)
        {
            eventBroker.Subscribe<OrganizationCreated>(x => indexQueueService.Enqueue(new IndexQueueItem<OrganizationDocument>(x.Item.SystemId)));
            eventBroker.Subscribe<OrganizationUpdated>(x => indexQueueService.Enqueue(new IndexQueueItem<OrganizationDocument>(x.Item.SystemId)));
            eventBroker.Subscribe<OrganizationDeleted>(x => indexQueueService.Enqueue(new IndexQueueItem<OrganizationDocument>(x.SystemId) { Action = IndexAction.Delete }));
        }
    }
}

The OrganizationDocument index will appear in the settings for Elasticsearch, now you can build indices for organizations

Indices.PNG

Please read How to add a searcher to Accelerator article where is described what need to do to start search in a site.