UX-Bridge - Developer Documentation

e-Spirit AG

FirstSpirit Version 5.x

2017-01-17
Table of Contents

1. Introduction

The UX-Bridge module is a response to the trend of dynamic websites. Whenever content cannot be pre-generated, the CMS content has to be accessed dynamically. In this case, dynamically means that the content can change for every website user and at any point in time. UX-Bridge provides an infrastructure for the requirement for a dynamic content delivery platform. Consequently, the module expands on the hybrid architecture approach by adding a standard component for dynamic content delivery.
Additional information can be found in the UX-Bridge Whitepaper.

This documentation is intended to support development with the UX-Bridge infrastructure and to provide a look at the concept.

The second section describes the general steps to consider when using UX-Bridge. The chapter covers the topics of installation, thoughts on data exchange format, setup in FirstSpirit, routing, and adapter and web application integration.

Two Tutorials explain the use of UX-Bridge. Here, we will cover implementation in FirstSpirit, the creation of an adapter, and the web application step-by-step, with concrete examples.

2. Concept

Certain projects have requirements that are to be implemented using UX-Bridge. In these cases, before implementation, a few questions have to be considered and addressed when defining the concept.

When developing a concept, two distinct situations can arise:
If a FirstSpirit project is developed from scratch, then the focus of development is on the optimum implementation of the specialized requirements. At the same time, the concept should be flexible enough to make it easy to implement and integrate future requirements.
If, on the other hand, UX-Bridge is to be integrated into a pre-existing project, the main question is how best to integrate UX-Bridge into the existing concepts and mechanisms.

In the following, some points will be considered which become relevant in many projects.

2.1. Generation and deployment concept

In many projects, the website is updated through periodic generation and deployment schedules. These schedules carry out a complete or partial alignment process. Under certain circumstances, these schedules can also be run manually. If, in this scenario, UX-Bridge is to be used, it is often sufficient to expand the existing schedules by adding the UX-Bridge-specific tasks. Details on this can be found in Section Create and configure schedule.

During generation, a message is sent to the UX-Bus for every page reference generated within the UX-Bridge generation task. Within the project, it should thus be ensured that, within this task, only the necessary page references are generated. If in a project, for example, only the news (maintained via a content source) is to be delivered via UX-Bridge, then only the necessary content projection pages are to be generated in the UX-Bridge generation task. In order to carry out this limitation, you can, for example, use a partial generation. Alternatively, full generation can also be used. However, uxbSkipMessage should then be used in order to cancel the generation of page references in the UX-Bridge presentation channel that are not to generate any messages (see Online Documentation for FirstSpiritTemplate developmentTemplate syntaxSystem objects#globalPreview-specificCancel a preview/generation).

In other projects, a majority of changes are released via workflows, which subsequently carry out generation and deployment of the participating objects. In most cases a script identifies the changed objects, which are then defined as start nodes of a partial generation. Objects are deleted using delete workflows. UX-Bridge can be easily integrated into these scenarios as well (see Section Workflow coupling).

If the time in which the objects must be available on the website is of high importance, then the use of a workflow-oriented approach is often the better choice. The fewer objects that have to be created during a generation process, the faster the objects will be available on the website. DeltaGeneration in FirstSpirit 5 provides a simplified mechanism for such scenarios (Developer APIDelta Generation or Access-APIGenerate Task). FirstSpirit 4 already offers this capability through the revision API.

All told, UX-Bridge can be incorporated into the existing generation and deployment concept or into one to be newly created, and for this purpose, does not include a standalone (separate) solution.

2.2. Data model and adapter

If UX-Bridge is to be integrated into an existing FirstSpirit project, then the data model is often preset in FirstSpirit. With heavily structured content through the database schema; with weakly structured content through forms of the page and section templates. Here, we recommend reviewing whether the web applications to be created (which later are to access the UX-Bridge data) can work with this data model, or whether a different data model makes more sense. This could be the case if the web application requires a much simpler or a much more complicated data model.

In the latter case, the FirstSpirit data is, in other words, just a part of the data model of the web application. In the first case, a subset of the data model stored in FirstSpirit is sufficient for many web applications. You also must consider whether the data model of the web application is to be denormalized for performance reasons.

If you have decided to use deviating data models, then you should clarify the step in which the transformation from one data model to another is to be carried out. The answer is certainly project-specific, and therefore only the possible variants will be described here.

An evaluation has to be carried out in the context of the project:

  1. The transformation is handled in the CMS syntax in the templates, which generate the XML necessary for _UX-Bridge (see Section Data exchange format – FS templating vs. WebApp development). The adapters themselves do not carry out any large transformations, but instead write the objects to the content repository in the manner defined by the data exchange format.
  2. The data exchange format corresponds to the data model in FirstSpirit 1:1. The transformation in the data model for the content repository is carried out within the adapter.
  3. It is a hybrid approach, which carries out transformations in both components. This depends on where implementation is easier.

The following considerations can help during the evaluation:

  1. If the data is to be written to multiple content repositories, then it is often sensible to generally retain the data exchange format and carry out any necessary/logical transformations for writing to the content repository within the adapter.
  2. If the data is to be written to multiple content repositories, then for every content repository, a unique adapter can be implemented which contains only the logic necessary for this repository. Alternatively, an adapter can write the data to both content repositories. This is useful, for example, if the data is supposed to be written within a transaction bracket.
  3. Does an adapter handle only one particular type of object, or are multiple object types bundled into one adapter? Object types in this context refer to different types of content. Say, for example, you would like to make all products and all news from a FirstSpirit data source available via UX-Bridge. Here, for example, we need to determine whether the two types of objects are to be transferred to the same content repository and/or data model or not.
  4. For decoupling and maintenance purposes, it is generally important to consider whether it would be better to use multiple (but lean) adapters or one adapter that contains all the logic.
  5. The advantage of a general data exchange format is that only one message has to be sent via the UX-Bus, which, however, then has to be processed by multiple adapters and can be written to multiple content repositories. In addition, no adaptations to the data exchange format may be necessary if new content repositories and web applications are to be connected in the course of the project.
  6. If UX-Bridge is to communicate with a system that can already receive JMS messages, then it may make sense to carry out a transformation directly on the UX-Bus. In this way, no adapter has to be implemented, which in turn would generate JMS messages only. In such cases, the routing can be expanded on the UX-Bus by adding corresponding transformation instructions.

2.3. News distribution/routing

As mentioned in the previous section, for every page reference within the generation, a message is generated and sent to the UX-Bus. Part of the UX-Bus is a routing component which receives the message and forwards or routes it to another so-called end point. Details on the standard configuration can be found in Section Routing.

This configuration can be adapted for the project-specific requirements. In this case, a simple domain-specific language (DSL) is available with the Apache Camel Spring XML syntax with which all current enterprise integration patterns (see http://camel.apache.org/enterprise-integration-patterns) can be implemented. With this powerful integration framework, the UX-Bus can be used as an information hub for the website and all participating systems.

Here are some examples which can be implemented on the UX-Bus through a routing:

  1. A content router is configured which sends certain messages only to certain end points/adapters.
  2. New end points can be configured which serve as the interface to web applications or back-end systems. Through these, an adapter can, for example, direct a web application to empty its cache because new data has been written to the content repository.
  3. Existing third-party applications can send messages to the UX-Bus, which are then processed by the web application, adapters or FirstSpirit.

In the standard configuration of UX-Bridge, a routing method is already configured which is sufficient for standard scenarios. Project-specific adaptations are necessary only for more complex scenarios (see Section Data model and adapter).

3. Quick Walkthrough

This chapter describes the steps for implementing UX-Bridge in a project. The Quick Walkthrough is intended for experienced FirstSpirit template developers and web application developers. Detailed, step-by-step instructions can be found in the Tutorials.

3.1. FirstSpirit

3.1.1. Installation

The first step in development within the context of UX-Bridge is to install the components.

To do this, first install the provided module in the server settings of FirstSpirit server. Doing this will start the UX-Bridge service, and the UX-Bridge components will be available in the projects of the server.

In addition to the FirstSpirit module, installation of the UX-Bus is also required. For local development, the installation in standalone operation is recommended.

For further details refer the UX-Bridge Installation Manual.

3.1.2. Data exchange format – FS templating vs. WebApp development

The architecture of UX-Bridge permits the development of solutions based on UX-Bridge to be divided into two roles:

  • A template developer creates the necessary templates, workflows and schedules.
  • A (web) application developer develops the adapter for the content repository and the (web) application.

If the roles are performed by different people, then a common data exchange format should be defined during the conceptual design process. In this case the intended data format is the one that is generated by the templates and is sent as a message to the adapter via the UX-Bus.

The data exchange format thus forms the interface between the components and thus also between the two roles. From the point of view of the template developer, this is the end product of their work. For the (web) application developer, this represents the input for the adapter. Here, only an outer container is prescribed by UX-Bridge. The remainder can be freely defined (see Section Creating and filling a presentation channel).

A carefully selected data exchange format can significantly reduce the implementation effort.

The following questions can help during the selection:

  • Does the data, and thus the data model, already exist in FirstSpirit? If yes, then you are subject to certain limitations. If no, it is advisable to match the data exchange format as closely as possible to that of the data in the content repository or of the web application.
  • For performance reasons, it may make sense to write the data to the content repository in denormalized form. Possible inconsistencies can be corrected through a full deployment, because the data usually continues to be available in normalized form in FirstSpirit.
  • Is the data to be written to more than one content repository? If yes, it is to be determined whether a data exchange format is sufficient and/or the adapter(s) can take over the transformation and persistence in the content repository. In some cases, it may even be more efficient to generate two data exchange formats through FirstSpirit, which then can be adopted in the respective content repository without a transformation step.

3.1.3. Creating and filling a presentation channel

To use UX-Bridge, a new template set (UXB) must first be created under project settingsTemplate sets. As a presentation channel, XML is to be configured as the Unicode to XML conversion rule and xml is to be configured as the target file extension.

create new Template Set
Figure 1. create new Template Set


The Unicode to XML conversion rule serves to transform special characters and control characters into corresponding XML entities, which otherwise would be interpreted in XML as a part of the markup language or would appear as invalid characters.

In order to send messages to the UX-Bus, the corresponding template that is to generate the messages contains fields that have been defined in the data exchange format and which are to be output in XML format.

presentation channel in SiteArchitect
Figure 2. presentation channel in SiteArchitect


Only the following structure is predefined:

<uxb_entity uuid = String destinations = String  language = String command = String objectType = String>
 <uxb_content>
   […] defined data exchange format […]
 </uxb_content>
</uxb_entity>

Property

Description

Example

Required field

Uuid

Unique identifier of the object, for example, fs_id

1234

Yes

destinations

Destination(s) of the message

(live repository, comma-separated)

postgres, mongodb

Yes

language

Language of the message

DE (German)

No

command

Command to be executed by the adapter (e.g. create/ delete)

Add

No

objectType

Object type evaluated by the adapter (e.g. News, Products)

News

No

The language, command and objectType attributes are optional, but have proven helpful with the adapters implemented by e-Spirit.

3.1.4. Create and configure schedule

In order to convert the data from FirstSpirit into messages that can be further processed by UX-Bus, it is mandatory for a schedule to be created or an existing schedule extended so that it generates XML. This is then forwarded to the UXB-Service, which generates a message from it and sends this to the UX-Bus. The schedule can be started later via a Workflow. This section will now describe two schedules that will probably be needed in every project that uses UX-Bridge.

Partial generation

In order to publish new content quickly on the website, it is useful to create a schedule or to expand on one in such a manner that it carries out the steps described in the following while only generating and publishing the new content.

A typical generation schedule which uses UX-Bridge is divided into multiple actions as described in the following:

schedule entry planning
Figure 3. schedule entry planning


First, all of the complete static pages that are needed for display on the website (e.g. news overview pages) should be generated in a generation action. This generation action takes place in the deployment schedules commonly used to date and does not have to be adapted.

In the next step, the content generated in the previous step should then be transferred to the web server as usual (in the example, via rsync). In this step, too, no adaptations are usually necessary.

Next, in order to use UX-Bridge, a new, additional action needs to be added which activates the generation process for UX-Bridge by calling a script:

UX-Bridge Generate. 

#! executable-class
com.espirit.moddev.uxbridge.inline.UxbInlineUtil

In the subsequent generation action, the exact page should then be generated (e.g., the news detail page) that generates the XML to be forwarded to the UXB-Service. It is important to ensure that the UXB template set has been enabled in the advanced properties.

activating the presentation channel
Figure 4. activating the presentation channel


Delta generation in FirstSpirit 5 now adapts this generation action automatically so that it no longer generates all pages, but rather just the page desired. In addition, for example, a workflow script that initiates the generation process can transfer the generated page to the schedule.

The last action, UX-Bridge Statistics Report, is optional and enables the cycle times to be measured for the messages in the UX-Bus until deployment on the website.

INFO  22.08.2012 09:59:54.631 (de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl): starting task 'UX-Bridge Statistics Report' - schedule entry 'UX-Bridge-Test (News)' (id=5142)
INFO  22.08.2012 10:00:04.645 (com.espirit.moddev.uxbridge.service.UxbServiceImpl): Time for #uxb/pressreleasesdetails/UXB/EN/256 (postgres): 242ms
INFO  22.08.2012 10:00:04.645 (com.espirit.moddev.uxbridge.service.UxbServiceImpl): Time for #uxb/pressreleasesdetails/UXB/DE/256 (postgres): 224ms
INFO  22.08.2012 10:00:04.645 (com.espirit.moddev.uxbridge.service.UxbServiceImpl): 2/2 deployed successfully (overall: 233ms, postgres: 233ms).
INFO  22.08.2012 10:00:04.646 (de.espirit.firstspirit.server.scheduler.ScheduleManagerImpl): finished task 'UX-Bridge Statistics Report' - schedule entry 'UX-Bridge-Test (News)' (id=5142)

The following script call is necessary for this:

UX-Bridge Statistics Report. 

#! executable-class
com.espirit.moddev.uxbridge.inline.UxbResultHandler

The service waits the maximum amount of time defined in the UX-Bridge service configuration until the adapters' responses are evaluated. If there is no response within this time frame, then the message is classified as having a delivery error. Since response times can vary depending on message and system, this value can be configured.

Please note that every generation action invoked between UX-Bridge Activate Generation and UX-Bridge Statistics Report will perform a UX-Bridge generation. To add a further file system generation without calling the `UX-Bridge Statistics Report´ script beforehand, it will be necessary to add a preceding script to deactivate the UX-Bridge generation:

UX-Bridge Deactivate Generation. 

#! executable-class
com.espirit.moddev.uxbridge.inline.UxbDeactivate

Complete alignment process

It makes sense to create an additional schedule which carries out a complete alignment process in order to maintain the most current state of the data inventory in the content repository. For this purpose, the data deleted in FirstSpirit must also be deleted in the external repository.

The following procedure is recommended:

  1. Complete generation of the static pages in FirstSpirit
  2. Run the UX-Bridge schedule in order to write all data to the content repository (also refer to the previous section). This data is to be given an up-to-date time stamp, which is saved in content repository in the lastmodified field.
  3. Run a script that calls up the cleanup method with a timestamp as a parameter describing the latest project revision of the schedule. The cleanup method then deletes all data saved in the lastmodified field that are older than the ones copied over. Those are the data that were already deleted in the FirstSpirit project and thus, in step 2, have no new timestamp saved in lastmodified.

cleanup-Skript. 

import com.espirit.moddev.uxbridge.api.v1.service.UxbService;
uxbService = context.getConnection().getService(UxbService.class);
uxbService.removeUxbEntriesByTime(context.getStartTime().getTime(), "news", "postgres,mongodb");

Historic Data

To generate historic data using UX-Bridge a new script action is to be added before the action UX-Bridge - Activate Generation.

Script. 

import java.util.Date;
Date d = new Date(114, 5, 24);
context.setStartTime(d);

The date that is set will be used in the following action (UX-Bridge - Activate Generation).

Recognizing the schedule start/end in the adapter

To make sure the adapter is able to respond at the start and/or end of a schedule in certain circumstances, the UX-Bridge can send a separate message automatically at both the start and end of generation (see UX-Bridge Installation Manual).

The format of the start message is as follows:

<uxb_entity projectName="PROJEKT_NAME" status="start" schedulerId="AUFTRAGS_ID" createTime="ZEITPUNKT_DER_NACHRICHT" projectId="PROJEKT_ID" startTime="STARTZEITPUNKT_DES_AUFTRAGS" />

During full generation, the command="startMaintenanceMode" attribute is also added.

The format of the end message is as follows:

<uxb_entity projectName="PROJEKT_NAME" status="end" schedulerId="AUFTRAGS_ID" createTime="ZEITPUNKT_DER_NACHRICHT" projectId="PROJEKT_ID" startTime="STARTZEITPUNKT_DES_AUFTRAGS" />

During full generation, the command="stopMaintenanceMode" attribute is also added.

If the maximal number of allowed errors or timeouts are exceeded in the adapter during a full generation, the command attribute has the value keepMaintenanceModeUp.

Adding root attributes

The root node of every message sent through the UX-Bridge comes with a set of predefined root attributes like the project name or the id of the schedule entry. They can be extended by further custom attributes by adding the following script Configure UXB Message Header to the beginning of the schedule:

Script - Configure UXB Message Header. 

import java.util.HashMap;
attributeMap = new HashMap();
attributeMap.put("customAttribute1", "customAttributeValue1");
attributeMap.put("customAttribute2", "customAttributeValue2");
context.setProperty("uxbMessageRootAttributes", attributeMap);

The script creates a HashMap containing each custom attribute as a key/value-pair. The HashMap will then be added to the schedule context, so that the UXBService can include the additional attributes during generation of the messages.

3.1.5. Skipping pages during generation

Pages can be skipped when generating messages by setting the uxbSkipMessage page variable.

uxbSkipMessage. 

$CMS_SET(#global.pageContext["uxbSkipMessage"], true)$

The use of "stopGenerate" (see Online Documentation for FirstSpiritTemplate developmentTemplate syntaxSystem objects#globalPreview-specificCancel a preview/generation) is not supported and in this case will result in invalid XML, resulting in error messages in the log.

3.1.6. Reading out expanded project information

The UX-Bridge can send a message with expanded project information at the start of generation (see UX-Bridge Installation Manual). This information currently includes the defined project languages and resolutions.

Example. 

<uxb_entity projectName="UXB" objectType="projectInfo" schedulerId="92460" createTime="1371037891875" projectId="88785"  startTime="1375346932923">
 <project key="UXB">
  <id>88785</id>
  <name>UXB</name>
  <languages>
   <language key="DE">
    <name>Deutsch</name>
    <abbreviation>DE</abbreviation>
    <isMasterLanguage>true</isMasterLanguage>
   </language>
  </languages>
  <resolutions>
   <resolution key="ORIGINAL">
    <name>Originalauflösung</name>
    <uid>ORIGINAL</uid>
    <height>0</height>
    <width>0</width>
    <isOriginal>true</isOriginal>
   </resolution>
  </resolutions>
 </project>
</uxb_entity>

3.1.7. Workflow coupling

The publication of content via UX-Bridge can be started directly via scripts and schedules or indirectly via workflows.

Release workflow

In order to publish content, an existing workflow simply has to be expanded by adding a workflow script that starts a schedule which, alongside generation and deployment, also generates XML messages and forwards them to the UXB-Service (see Partial generation).

Delete workflow

In order to delete content, an existing delete workflow has to be expanded by adding a workflow script that generates an XML message, which is forwarded to the UXB-Service.

Invoking the UXB-Service is written in the script as follows, where msg (string) corresponds to the XML message:

UXB-Service. 

UxbService uxbService = context.getConnection().getService(UxbService.class);
uxbService.removeUxbEntry(msg);

The XML message follows the example below:

<uxb_entity uuid=STRING language=STRING destinations=STRING objectType=STRING command=STRING />

Property

Description

Example

Required field

Uuid

Unique identifier of the object, for example, fs_id

12345

Yes

Destinations

Target(s) of the message (comma-separated)

postgres

Yes

Command

Command to be executed by the adapter

delete

Yes

Language

Language of the message

DE (German)

No

objectType

Object type evaluated by the adapter (e.g. News, Products)

news

No

3.2. Adapters

Adapters are also used to read out the data from the JMS messages and to write them to the selected repositories. In the tutorial, two adapters are implemented as web applications as an example, but other implementations (e.g. standalone Java) are also possible.

3.2.1. Feedback

FirstSpirit expects a response from the adapter in the form of an XML document after a message has been written to a repository. The response is expected in the case of both successful and failed processing. The XML document is structured as follows:

<uxb_entity command=STRING createTime= STRING destinations=STRING finishTime=STRING language=STRING path=STRING schedulerId=STRING startTime=STRING status=STRING uuid=STRING ><uxb_error>STRING </uxb_error></uxb_entity>

Property

Description

Example

Required field

destinations

The target repository to which the object has been or was to be written.

postgres

Yes

startTime

Timestamp for the start of the action (appended to the XML document by FirstSpirit)

1314567899516

Yes

finishTime

Timestamp for completion of the command

1314567899516

Yes

path

FirstSpirit internal path (appended to the XML document by FirstSpirit during the action)

the/Path/to/

Yes

status

Status of the action.

Possible values:

"OK" if successful,

"FAIL" if the action fails

OK

Yes

uuid

Unique identifier of the object, for example, fs_id

123456

Yes

schedulerId

Unique ID for the schedule (appended to the XML document by FirstSpirit during the action)

123456

Yes

command

Command executed by the adapter

delete

No

language

Language of the message

DE (German)

No

createTime

Timestamp for the creation of the action (appended to the XML document by FirstSpirit during the action)

1314567899516

No

uxb_error

The container element for the error message present in the event of an error

com.mongodb. MongoException

No

3.3. WebApplication

Through the open architecture of UX-Bridge and the fact that the type and number of repositories is not preassigned, the technology and the framework for developing the WebApplication can be freely selected. It is useful to base the selection of technology and the framework both on the application and the knowledge/company standards that are in place.

3.4. Routing

UX-Bridge uses Apache Camel to route messages. Java Message Service (JMS) is used for the transportation and message protocol. The two participating components of the FirstSpirit server and adapters assume both the role of a producer, which generates messages and makes them available in an end point, and the role of a consumer, which retrieves the messages from an end point and processes them further. The UX-Bus, in this scenario, simply takes over the routing of messages between the participating end points.