e-Spirit AG


1. Introduction

Search functions are just one of many features that customers expect from an online presence. They must be intuitive to use and deliver relevant results.

SmartSearch bundles these requirements and represents a high-performance search solution for them, which can be used on extensive websites. It offers both a high hit quality and an optimal search comfort and thus binds customers to the website.

By means of SmartSearch Connect, the functionalities of SmartSearch Connect and FirstSpirit can be optimally combined. With very little effort it is possible to provide the website created with FirstSpirit with a high-performance search. Changes are directly reflected in the search results.

The goal of SmartSearch Connect is to create a simple and fast link between SmartSearch and FirstSpirit. Care has been taken to create as little installation and configuration effort as possible.

To ensure the simplicity of the module it was deliberately avoided to be compatible with every FirstSpirit project. For example, it is currently not yet possible to use projects in which fragments are used. In most cases the functionality should be more than sufficient.

2. Installation and configuration

For the deployment of SmartSearch Connect a project specific configuration is required. This is done via the project component to be added to the used project. To do this, open the ServerManager and select the Project Properties area → Project Components..

Figure 1. Selection for configuring the project component

The main panel shows a list of all existing project components. After clicking Add, select the SmartSearch ConnectProjectApp and confirm your selection with OK. The project component is then added to the list in the main panel and must be configured (see figure Selection for configuring the project component). To do so, select the entry in the list and open the corresponding configuration dialog via Configure (see figure Dialog der Projekt-Komponente).

Figure 2. Dialog der Projekt-Komponente

First of all, a name for the Datagenerator and PreparedSearch to be generated automatically in SmartSearch can be configured here. If the field is left empty, the current project name is used for this purpose.

Via the button Initialize in SmartSearch! below, the above configured Datagenerator and the PreparedSearch can be created at any time ("Initialization"). Here, as well, the currently stored project name is used, should the above field be empty. The result of the call can be obtained from the server log, as this is where the initialization is triggered.

In the project configuration it is necessary to synchronize the configuration of the URL Creator from the corresponding generation. It is therefore necessary to select a URL Creator and to add the prefix to the URL Creator.

In the media file extension list, you can select which media should be transferred to SmartSearch using SmartSearch Connect. In this list, only file extensions of textually processable files that are present in the project are displayed. This list may have to be extended during the use of a project.

By default, SmartSearch delivers the URLs of the original resolutions for images. If you want to use URLs of other resolutions, you can select them in the Image resolutions. SmartSearch Connect does not ensure that the images are accessible from the outside. This must be ensured in the project.

Only media referenced in FirstSpirit are processed. This ensures that only media that can be reached from outside are processed.

By default, the documents in SmartSearch contain only the metadata of the corresponding element in FirstSpirit. If the metadata is to be inherited, then the check mark Inherit metadata must be set.

Inheritance does not refer to the mechanism to exclude elements from the SmartSearch index. At this point, inheritance in the structure takes effect even without this configuration.

2.1. Exclude elements from SmartSearch index

SmartSearch Connect writes all elements to the SmartSearch index by default. If the possibility should be given that this can be prevented directly in FirstSpirit, then it is necessary to extend the project.

2.1.1. Hide pages

For hiding pages it is necessary to add an input component to the metadata. This must be a CMS_INPUT TOGGLE and must have the name md_smartsearch_noindex.

Example of the input component in the metadata
<CMS_INPUT_TOGGLE name="md_smartsearch_noindex" type="radio" hFill="yes">
        <LANGINFO lang="*" label="Hide"/>
        <LANGINFO lang="*" label="On" description="Hide this site from search index."/>

If the metadata are included as described above, they are also displayed in the page store, for example. However, it is only displayed in the site store. Therefore, it makes sense to display the metadata only in the site store by a rule.

Example of the rule to create metadata only in the site store.
                        <PROPERTY name="STORETYPE" source="#global"/>
                <PROPERTY name="VISIBLE" source="md_smartsearch_noindex"/>
Hiding pages also works via folders. Please note that changes to folders only take effect after a full generation or a change to an affected page.

2.1.2. Hide datasets

Because datasets do not have metadata, it is necessary here to add an additional column of type boolean to the corresponding tables in the database schema. It is important to note that the column must be named smartsearch_noindex. In order to edit the columns data, an input component must be added to the table template.

Pages and datasets that already exist in the index but are to be hidden by setting the corresponding field are only deleted from the index after a full generation. Eventing does not take effect at this point.

3. Full generation

For the initial filling of SmartSearch a full generation is necessary. For this purpose a new job is created in the job administration of the project configuration and added under the actions SmartSearch Connect push.

Figure 3. Selection of the full generation action

Executing this job creates a new data generator and a new Prepared Search in SmartSearch Connect (name of the FirstSpirit project with the prefix FS_).

4. Partly generation

To keep SmartSearch up to date after a partial generation it is necessary to add a script type action to the job. This script must be included after the actual generation and have the following content:

#! executable-class
Figure 4. Script for the partial generation

5. Eventing

To keep the contents in SmartSearch as up to date as possible, there is eventing in addition to generation. Here, changes (creation, deletion, modification of elements) are sent to SmartSearch directly after the elements have been released.

In order to trigger event-driven transfer of data in the direction of SmartSearch by editorial adjustments in ContentCreator, certain requirements must be met by the workflow used. For example, dependent objects must be released by the workflow as soon as an object is released (example: when a page change is released, the dependent page references must be released). To achieve this behavior, the BasicWorkflows module can be installed, for example.

6. What happens in SmartSearch?

If a FirstSpirit project with the exemplary name "Demo" sends data to the SmartSearch for the first time, the elements required for use are automatically created there:

A DataGenerator of type "API" with the name "FS_Demo" (FS_<project name>):

Figure 5. Automatically generated DataGenerator

A PreparedSearch with the name "FS_Demo

Figure 6. Automatically generated PreparedSearch

The corresponding DataGenerator is already preconfigured at this location:

Figure 7. Automatically generated link between PreparedSearch and DataGenerator

6.1. DataGenerator

The DataGenerator(s) generated by the module are permanently set to the status "API Ready", except for the short moments when data is received. If this data reception is very fast, the status change is not visible in the cockpit. Also in the log the reception of data is only visible at loglevel DEBUG, here a corresponding entry for logback-spring.xml:

<logger name="de.arithnea.haupia.datageneration.crawler.api.reactive" level="DEBUG"/>

DataGenerators of type "API" cannot be started and stopped because they are permanently waiting for data.

Data transmitted to the DataGenerator will be transferred to the SOLR index after 30 seconds. This time period can be reduced (for test purposes) to e.g. one second using the environment variable API_CRAWLER_COMMIT_WITHIN on the SmartSearch system, but this is not intended for productive use: export API_CRAWLER_COMMIT_WITHIN=1000

6.2. PreparedSearch

The preconfigured PreparedSearch can be retrieved via REST immediately after receipt of the data. Facets, for example, can be maintained on it as usual. The facet fsType contains the type of the entity exported from FirstSpirit:

Figure 8. Facets language and fsType

A possibly important configuration for using the PreparedSearch would be e.g. adding the field link to the output, so that when querying the interface the result is output around the URLs generated by FirstSpirit:

Figure 9. Added field link

7. Known problems and limitations

7.1. FirstSpirit-project name

Currently, the project name in FirstSpirit should not contain any spaces if possible. While the export to SmartSearch with a space character works, there may be some unpredictable side effects, since spaces are not allowed for DataGenerator and PreparedSearch names. This will be changed in the future to use the FirstSpirit project ID and to be able to use a display name in SmartSearch Connect accordingly.

7.2. Renaming projects in FirstSpirit

To make the data generators and PreparedSearches automatically created by the FirstSpirit module in SmartSearch recognizable, follow the naming scheme "FS<Project name>" above.

As a result, the FirstSpirit project must not be renamed, as SmartSearch cannot follow this customization. For SmartSearch, data transferred afterwards would result in the creation of a new data generator and a new PreparedSearch, and thus a new "index".

As described above, this will be adjusted in the future in the direction of using the Project ID, which will then make renaming possible.

7.3. Filter for media

In the module configuration the media types can be maintained, which are to be processed by SmartSearch. With this function there is a restriction, if for a medium in different languages different medium types are deposited. In this case, media that are not stored in the configuration are also transferred if there is a medium in another language that is to be transferred.

SmartSearch Connect is a product of e-Spirit AG, Dortmund, Germany. Only a license agreed upon with e-Spirit AG is valid with respect to the user for using the module. Only a license agreed upon with e-Spirit AG is valid for using the module.

Details regarding any third-party software products in use but not created by e-Spirit AG, as well as the third-party licenses and, if applicable, update information can be found in the file THIRD-PARTY.txt included with the module.

9. Help

The Technical Support of the e-Spirit AG provides expert technical support covering any topic related to the FirstSpiritâ„¢ product. You can get and find more help concerning relevant topics in our community.

10. Disclaimer

This document is provided for information purposes only. e-Spirit may change the contents hereof without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. e-Spirit specifically disclaims any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. The technologies, functionality, services, and processes described herein are subject to change without notice.