1. Introduction

Content as a Service provides the editorial data of FirstSpirit via a uniform JSON-based data format that can be consumed by any endpoint. The data is updated transparently when it is changed or released in the FirstSpirit editorial system. With a cloud installation of the Content as a Service platform the data can be efficiently accessed worldwide. In FirstSpirit, only the form configurations and the data maintained by the editorial system are accessed. Templating in FirstSpirit is completely eliminated and is shifted to the front end or front end services (for more information, see FSXA documentation).

The FirstSpirit developer can thus concentrate completely on modeling the domain-oriented objects and creating the corresponding input components. The following section describes the conventions for data format and URLs, as well as the optimal delivery of content for different consumers.

In the remainder of this document the abbreviation CaaS will be used instead of Content as a Service.

2. Introduction FirstSpirit module

The CaaS Connect module is the link between FirstSpirit editorial environment and the CaaS platform that acts as the content delivery. This connection is largely invisible to the editorial team and only requires attention in the event of a technical problem. The editors can always view the current state of data directly via the CaaS platform. As soon as a change has been made in FirstSpirit, the synchronization is triggered in the background, and the database immediately reflects the change.

There are traditional and fragment projects in FirstSpirit. Technical pages - and thus also the elements that are transferred to the CaaS platform - correspond to the page references in traditional projects and to the fragments in fragment projects. For the sake of simplicity we speak of pages in both cases. The CaaS Connect module automatically distinguishes between traditional and fragment projects.

2.1. CaaS URLs

The module creates documents in the CaaS platform for the following FirstSpirit elements:

  • Page references

  • Media

  • Global content pages

  • Project settings page

  • Datasets

Each element will be represented as multiple documents. For each language of the FirstSpirit project and for both the preview and release state, a document is created. Each of these documents can be identified and referenced with a unique URL.

2.1.1. URL schema

The CaaS URL for a document consists of four parts: tenant id, project identifier, collection identifier, and document identifier. The tenant id is the maintained name or abbreviation of the tenant. It is set in the CaaS Connect Service by the server administrator. The project identifier is the GID of the FirstSpirit project. The collection identifier allows you to differentiate between release and preview state. It contains either the value release.content or preview.content. The document identifier is composed of the FirstSpirit GID of the FirstSpirit element and the language. Together with the base URL of the CaaS endpoint, this results in the fully qualified URL of a document.

CaaS URLs for project settings do not use their FirstSpirit GID as document identifier but instead use the identifier projectsettings.

Example:

The tenant id is defaultTenant.

The FirstSpirit project GID is e54cb80e-1f9c-4e8d-84b8-022f473202eb.

The fully qualified CaaS URL of a sample page with the GID f6910b22-6ae8-4ce1-af45-c7b364b3117a in preview and for the English language looks like this

CaaS endpoint: http://localhost:8080

CaaS URL: defaultTenant/e54cb80e-1f9c-4e8d-84b8-022f473202eb.preview.files/f6910b22-6ae8-4ce1-af45-c7b364b3117a.en_GB

The configuration dialog of the project component shows an overview of all collection URLs of the project.

2.2. Data format

Unlike traditional FirstSpirit projects and earlier module versions, CaaS version 3 or later does not support either output channels or FirstSpirit templating. Instead, a JSON document is generated for each page of the project, based on the toJson standard of FirstSpirit. An adaptation of the format is not possible. This restriction allows to deliver a complete, standardized data format via CaaS platform, so that all consuming endpoints can work with the same data format. Since the standard data format is very comprehensive, the platform offers filtering and aggregation capabilities to reduce data volumes for mobile end points, for example. For more information about CaaS platform, see platform documentation.

Since the CaaS URLs derive the document identifier from a unique FirstSpirit ID (among other things), it is necessary to use a filter when querying a document using its name. To query a CaaS page (a FirstSpirit page reference) with the name services a GET request is executed with the following URL:

For more information on queries for the CaaS platform, see platform documentation.

2.2.1. CaaS JSON format

The standard JSON format of FirstSpirit serves as the basis for the CaaS JSON format and is extended by the CaaS Connect module both with CaaS specific JSON format configuration, as well as with some attributes that simplify its usage. The CaaS specific format configurations include reducing the output of datasets to references, the indirect referencing of records of a content projection, and enabling the output of the FirstSpirit metadata.

Dataset URLs: FirstSpirit does not address individual datasets and instead works with content projections or selects and embeds datasets in pages. In CaaS projects, individual datasets are identified by a unique URL and can be queried with it. Therefore, pages do not embed datasets, but contain references to the stored datasets in the CaaS platform.

JSON snippet example
{
   "fsType" : "FS_DATASET",
   "name" : "st_button_link",
   "value" : {
      "fsType" : "DatasetReference",
      "target" : {
         "entityType" : "product",
         "fsType" : "Dataset",
         "identifier" : "fae0687b-c365-4851-919e-566f4d587201",
         "schema" : "products"
      }
   }
}

Dataset routes: The route attribute contains the relative route of a dataset. This route is only calculated if a preview page has been selected for the underlying table template. The preview page should contain a content-projection for the rendered table, and the setting "number of entries per page" should be set to "1" (see Content Projection). This is the only way different datasets will have different routes.

JSON snippet example
{
    "route" : "/Company/Locations/Locations.html"
}

Fragment metadata: The attribute fragmentMetaData contains the attributes id (fragment ID) and type (fragment type).

JSON snippet example
{
    "fragmentMetaData": {
        "id": "378d5ec9_58f1_4dec_83bc_724dc93de5c2",
        "type": "news"
    }
}

Locale: The locale attribute contains the attributes identifier (abbreviation of the language), country (associated country) and language (associated language).

JSON snippet example
{
    "locale": {
        "identifier": "EN",
        "country": "GB",
        "language": "en"
    }
}

Media references: References to media point to the associated document of the medium. The URLs to the actual binary data of the medium are inside the referenced document.

JSON snippet example of a media reference
{
   "st_header_image": {
        "fsType": "FS_REFERENCE",
        "name": "st_header_image",
        "value": {
            "fsType": "Media",
            "name": "header_home",
            "identifier": "b7fd2e2c-a022-44d6-ae2d-f0b9fd48f9a2",
            "uid": "header_home",
            "uidType": "MEDIASTORE_LEAF",
            "mediaType": "PICTURE",
            "url": "https://some-caas-api/some-tenant/631e4786-0dc3-4db6-bad8-adaad685944a.preview.content/b7fd2e2c-a022-44d6-ae2d-f0b9fd48f9a2.de_DE"
        }
    }
}

Content2Section: The JSON data of content projections or its sections contain references to its records when using the standard JSON configuration of FirstSpirit. However, the CaaS Connect module uses a specific configuration for the JSON format so that the records are not referenced directly, but indirectly via a query object. For this purpose, the query object contains identifiable attributes of the content projection.

JSON snippet example
{
   "displayName" : "blog",
   "entityType" : "blog",
   "filterParams" : {},
   "fsType" : "Content2Section",
   "maxPageCount" : 0,
   "name" : "blog",
   "ordering" : [
      {
         "ascending" : false,
         "attribute" : "fs_id"
      }
   ],
   "query" : null,
   "recordCountPerPage" : 1,
   "schema" : "global",
   "template" : {
      "displayName" : "Blog entry",
      "fsType" : "TableTemplate",
      "identifier" : "e657e0f0-0fd3-456f-b5ab-560a879ca748",
      "name" : "Blog entry",
      "uid" : "global.blog",
      "uidType" : "TEMPLATESTORE_SCHEMA"
   }
}

2.3. Preview and release state

An essential distinction between release and preview data states is made by both FirstSpirit and the CaaS Connect module. The platform manages both states of data, which are distinguishable by different CaaS URLs (see URL schema). A synchronization of both data states is always based on certain actions that the editors perform in FirstSpirit. Release actions are the only actions that update the release state, all other changes only affect the preview state.

2.4. Manual data reconciliation

For certain scenarios, it may be necessary to perform a full reconciliation between FirstSpirit and CaaS:

  • Populating the CaaS with existing project data to start a project.

  • Reconciling data in the event of an error to manually restore data consistency between systems.

This manual full reconciliation of the data is enabled via schedules, which are automatically created with the installation of the CaaS Connect module. After module installation, the project contains two schedules Caas Connect Release Generation and Caas Connect Preview Generation, which can perform a full synchronization for the release or preview state. These schedules are automatically updated when the CaaS Connect module is updated.

schedules
Figure 1. configuration dialog of the project component of the CaaS module

Changes to the configuration of these schedules are not persistent, as the schedules are reset to their original state by the automatic update. If the configuration of these schedules needs to be changed, these changes have to be applied to a copy of the respective schedule.

2.4.1. Overwrite the complete data set

If changes in the FirstSpirit project are expected to be visible in the CaaS data but instead are not reflected in the CaaS, the schedules can be used to overwrite the full data set of the CaaS. For this purpose it is necessary to create a copy of the respective schedule and to adjust its configuration so that the replicationMode parameter of the script task contains the value FULL (in contrast to the default DELTA). Running this task will then overwrite the complete data set in CaaS.

full replication
Figure 2. Job configuration dialog

2.4.2. Hidden sections

In FirstSpirit it is possible to hide sections. The data of these sections is treated differently depending on the preview or release state.

Sections hidden in FirstSpirit are always part of the preview data, but never included in the release data. The preview data also contains an additional JSON attribute named displayed per section. This indicates whether the section is displayed or hidden and thus whether it is transferred to the CaaS as part of the release data.

Accordingly, the following possibilities for sections arise:

Visibility Part of preview state Value of displayed in preview state Part of release state

Displayed

Yes

true

Yes

Hidden

Yes

false

No

Please note that the displayed attribute is only present in the preview state. In the release state, only visible sections are present and therefore this attribute is not used there.

2.5. Push notifications (change streams)

It is often convenient to be notified about changes in the CaaS platform. For this purpose the CaaS platform offers change streams. This feature allows a websocket connection to be established to the CaaS platform, through which events about the various changes are published.

By default, a crud change stream is provided for each collection created by CaaS Connect. It publishes all insert, replace, update and delete events. It is accessible at <collectionUrl>/_streams/crud. The exact definition of the change streams can be accessed at any time in the collection metadata.

More information on using change streams is available in the CaaS platform documentation.

2.6. Indexes

To query documents from CaaS platform, filters can be used that specify criteria for the documents to be returned. To speed up filter queries on frequently used document attributes, various indexes are set up on some collections created by CaaS Connect. Information on existing indices can be retrieved under <collectionUrl>/_indexes/.

The following indexes are set up in the collections preview.content and release.content:

  • idx_identifier_lang_country with the attribute combination identifier, locale.language and locale.country

  • idx_entity_lang_country with the attribute combination entityType, locale.language and locale.country

  • idx_fstype_lang_country with attribute combination fsType, locale.language and locale.country

  • idx_route with the document attribute route

The order of the index fields is relevant. Please use the database documentation to ensure that your filter queries are designed in such a way that the indexes are used.

More information on indices is available in the CaaS platform documentation.

3. Hosting of media files

The CaaS Connect module supports several methods to host the binary data of media:

  • Upload to S3 and access via a CloudFront distribution (CloudFront CDN)

    • This is the default in the e-Spirit Cloud.

  • Upload to S3 and access via a S3 REST API

  • Upload to CaaS and access via the CaaS REST API.

    • This is the default when no S3 configuration is provided.

The options have different use cases and are therefore explained individually below.

The number of media files that are transferred to the target system is identical for all hosting options. For language-independent media, exactly one binary file is transferred. For language-dependent media, one binary file is transferred for each language. Additionally, for images the binary data of each resolution is transferred.

3.1. Controlling which resolutions are transmitted using the CaaS tag

By optionally assigning the CaaS tag, it is possible to control which resolutions are subsequently transferred to a file bucket or S3 bucket during a generation.

add resolution tag
Figure 3. Configuration dialog of a new tag

In the following example, only three resolutions are transmitted to the CaaS.

resolution list
Figure 4. listing of the project resolutions

If no CaaS tag is configured in ServerManager for the given project, corresponding binary data will be transferred by default for all resolutions present in the project.

More information about tagging resolutions can be found in the FirstSpirit documentation for administrators.

Please note that previously transferred binary data is not removed from CaaS by subsequent changes to the resolution tag.

Additionally a change in the CaaS tags will not automatically update the CaaS data, you have to execute the synchronization manually.

3.2. Upload to S3

This option includes the upload of the media files into a S3 bucket. The binary files are stored within the bucket under the configured path prefix (default is prefix /). The path is also composed of the GID of the project and the path segment /preview, depending on whether the media is in the release or preview state. The rest of the path and the filename of a media file are derived from the Advanced URL of the media.

Example of a S3 object path:

/631e4786-0dc3-4db6-bad8-adaad685944a/preview/FirstSpirit/images/thisisatest.png.

When uploading media files to S3, the files can be accessed either through a CloudFront distribution or directly through the associated S3 REST API endpoint. The following chapters describe these two access options.

3.2.1. Access via CloudFront Distribution

When using a CloudFront distribution to access the media files, the URL to a file is composed of the domain of the CloudFront distribution and the path of the S3 object (described above).

For example, the following URL points to the original resolution of an image of the preview state, which is accessible through a CloudFront distribution that is configured to use the S3 bucket for the domain my-cloudfront-domain.tld.

The following code snippet shows an example of the content of a media document, including the URLs that point to the binary data for each resolution of the image.

If the CaaS-Tag is set in the resolutions settings, only these will also be listed in the resolutionsMetaData object.

Example: JSON of a media document in the preview state
{
    "_id": "3b3fe5b6-9ec1-4e45-84c8-bc6a39d0aa61.en_US",
    "fsType": "Media",
    "name": "thisisatest",
    "displayName": "thisisatest",
    "identifier": "3b3fe5b6-9ec1-4e45-84c8-bc6a39d0aa61",
    "uid": "thisisatest",
    "uidType": "MEDIASTORE_LEAF",
    "fileName": "thisisatest",
    "languageDependent": false,
    "mediaType": "PICTURE",
    "description": null,
    "resolutionsMetaData": {
        "ORIGINAL": {
            "fileSize": 16358,
            "extension": "png",
            "mimeType": "image/png",
            "width": 578,
            "height": 340,
            "url": "https://my-cloudfront-domain.tld/631e4786-0dc3-4db6-bad8-adaad685944a/preview/FirstSpirit/Bilder/thisisatest.png"
        },
        "Teaser": {
            "fileSize": 1192,
            "extension": "png",
            "mimeType": "image/png",
            "width": 111,
            "height": 65,
            "url": "https://my-cloudfront-domain.tld/631e4786-0dc3-4db6-bad8-adaad685944a/preview/FirstSpirit/Bilder/thisisatest_Teaser.png"
        }
    },
    "metaFormData": {},
    "changeInfo": {
        "date": "2021-05-10T10:41Z",
        "revision": 21077
    },
    "locale": {
        "identifier": "EN",
        "country": "US",
        "language": "en"
    },
    "_etag": {
        "$oid": "60990def3542845893b034da"
    }
}

CloudFront Caching

When changing the binary data of media, its files are not immediately invalidated in the cache of the CloudFront distribution. Instead, an automatic update takes place after a maximum of 15 minutes, provided that the CloudFront distribution is configured to take into account the Cache-Control header from an origins HTTP responses. In the FirstSpirit Cloud this is active by default.

For more information regarding the Cache-Control header, see the S3 object metadata section.

3.2.2. Access via S3 REST API

When accessing media files via the S3 REST API, the URL to a media file is composed of the S3 REST API (virtual hosted endpoint of the bucket) and the path of the S3 object (described above).

For example, the following URL points to the original resolution of an image of the preview state in the S3 bucket my-bucket located in the region eu-central-1.

The following code snippet shows an example of the content of a media document, including the URLs that point to the media data for each resolution of the image.

Example: JSON of a media document in the preview state
{
    "_id": "3b3fe5b6-9ec1-4e45-84c8-bc6a39d0aa61.en_US",
    "fsType": "Media",
    "name": "thisisatest",
    ...
    "resolutionsMetaData": {
        "ORIGINAL": {
            "fileSize": 16358,
            "extension": "png",
            "mimeType": "image/png",
            "width": 578,
            "height": 340,
            "url": "https://my-bucket.s3.eu-central-1.amazonaws.com/631e4786-0dc3-4db6-bad8-adaad685944a/preview/FirstSpirit/Bilder/thisisatest.png"
        },
        "Teaser": {
            ...
            "url": "https://my-bucket.s3.eu-central-1.amazonaws.com/631e4786-0dc3-4db6-bad8-adaad685944a/preview/FirstSpirit/Bilder/thisisatest_Teaser.png"
        }
    },
    ...
}

3.2.3. S3 object metadata

During the upload of media files to the S3 bucket, different metadata attributes are set for the S3 object:

  • Content-Type

    • This attribute is used as the Content-Type response header for requests to the S3 object.

    • It contains the MIME type of the media file (e.g. image/png).

  • Cache-Control

    • This attribute is used as the Cache-Control response header for requests to the S3 object.

    • For the release state it contains the value public, max-age=300, s-maxage=900 by default.

    • For the preview state it contains the value private, no-cache.

3.3. Upload to CaaS and access via CaaS REST API

In this mode, the binary data of media is transferred to CaaS file buckets. The binaries of the media are transferred to either the CaaS file bucket

  • [Project-GID].release.files or

  • [Project-GID].preview.files

depending on whether they are in release or preview state. The file buckets are located in the CaaS database corresponding to the tenant ID. The remaining path and file name of a media file is composed of the FirstSpirit-GID of the medium, as well as language and/or resolution (if the medium depends on language and/or resolution).

The binary data can be accessed via the CaaS REST API. The URL to a media file is composed of the base URL of the CaaS REST API, the path (described above) and the suffix /binary.

The following code snippet shows an example of the content of a media document, including the URLs that point to the media data for each resolution of the image.

Example: JSON of a media document in the preview state
{
    "_id": "3b3fe5b6-9ec1-4e45-84c8-bc6a39d0aa61.en_US",
    "fsType": "Media",
    ...
    "resolutionsMetaData": {
        "ORIGINAL": {
            "fileSize": 16358,
            "extension": "png",
            "mimeType": "image/png",
            "width": 578,
            "height": 340,
            "url": "https://my-caas-url.tld/defaultTenant/e54cb80e-1f9c-4e8d-84b8-022f473202eb.preview.files/f6910b22-6ae8-4ce1-af45-c7b364b3117a.ORIGINAL/binary"
        },
        "Teaser": {
            ...
            "url": "https://my-caas-url.tld/defaultTenant/e54cb80e-1f9c-4e8d-84b8-022f473202eb.preview.files/f6910b22-6ae8-4ce1-af45-c7b364b3117a.Teaser/binary"
        }
    },
    ...
}

The CaaS is a product of e-Spirit AG, Dortmund, Germany.

Only a license agreed upon with e-Spirit AG is valid with respect to the user for using the module.

5. Help

The Technical Support of the e-Spirit AG provides expert technical support covering any topic related to the FirstSpiritâ„¢ product. You can get and find more help concerning relevant topics in our community.

6. Disclaimer

This document is provided for information purposes only. e-Spirit may change the contents hereof without notice. This document is not warranted to be error-free, nor subject to any other warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or fitness for a particular purpose. e-Spirit specifically disclaims any liability with respect to this document and no contractual obligations are formed either directly or indirectly by this document. The technologies, functionality, services, and processes described herein are subject to change without notice.