CaaS Platform

All authentication settings for the communication with or within the CaaS platform are specified in the credentials block of the custom-values.yaml. So here you will find usernames and default passwords as well as the CaaS Master API Key. It is strongly recommended adjusting the default passwords and the CaaS Master API Key.

The configuration of the repository includes two parameters:

openssl rand -base64 756

An adjustment of the parameters imageRegistry and imageCredentials is necessary to configure the used Docker registry.

Ingress-Definitions control the incoming traffic to the respective component. However, the definitions contained in the chart are not created in the standard configuration. The parameters restApi.ingress.enabled and restApi.ingressPreview.enabled allow the ingress configuration for the REST Interface.

If the setting options are not sufficient for the specific application, the Ingress can also be generated independently. In this case the corresponding parameter must be set to the value enabled: false. The following code example provides an orientation for the definition.

The scaling of the REST Interface is done with the help of a Horizontal Pod Autoscaler. Its activation as well as configuration must be done in the custom-values.yaml file to overwrite the default values defined in the values.yaml file.

The Horizontal Pod Autoscaler allows to scale down or up the REST Interface depending on the current CPU load. The parameter targetCPUUtilizationPercentage specifies the percentage value from which scaling should take place. At the same time the parameters minReplicas and maxReplicas define the minimum and maximum number of possible REST Interfacen instances.

We distinguish horizontal scaling from vertical scaling here. Horizontal scaling means additional instances that handle the traffic. Vertical scaling means allocating more CPU/RAM to existing instances.

Horizontal scaling

Unlike REST Interface, horizontally scaling the Mongo database is only possible manually. Therefore, it cannot be performed automatically using a Horizontal Pod Autoscaler.

Scaling the Mongo database is done using the replicas parameter. This parameter must be entered in the custom-values.yaml file to override the default value defined in the values.yaml file.

Vertical scaling

Vertical scaling is done using a Vertical Pod Autoscalers. Vertical Pod Autoscalers are Custom Resources in Kubernetes, so first you need to ensure support in your cluster.

After that, you can configure the following parameters in your custom-values.yaml:

Applying the configuration

The updated custom-values.yaml file must be applied after the configuration changes for the REST Interface or Mongo database with the following command.

The deployment of the ServiceMonitors provided by the CaaS platform for the REST Interface and the mongo database, is controlled via the custom-values.yaml file of the Helm-Charts.

Typically, Prometheus is configured to consider only ServiceMonitors with specific labels. The labels can therefore be configured in the custom-values.yaml file and are valid for all ServiceMonitors of the CaaS Helm chart. Furthermore, the parameter scrapeInterval allows a definition of the frequency with which the respective metrics are retrieved.

The MongoDB metrics are provided via a sidecar container and retrieved with the help of a separate database user. You can configure the database user in the credentials block of the custom-values.yaml. The sidecar container is stored with the following standard configuration:

The deployment of the alerts provided by the CaaS platform is controlled via the custom-values.yaml file of the Helm-Charts.

Prometheus is typically configured to consider only alerts with specific labels. The labels can therefore be configured in the custom-values.yaml file and apply to all alerts in the CaaS Helm chart:

The deployment of the Grafana dashboards provided by the CaaS platform is controlled via the custom-values.yaml file of the Helm-Charts.

Typically, the Grafana Sidecar Container is configured to consider only configmaps with specific labels and in a defined namespace. The labels of the configmap and the namespace in which it is deployed can therefore be configured in the custom-values.yaml file:

Each request to the REST Interface must contain an HTTP Authorization header containing the API Key as Bearer token: Authorization: Bearer <key>. The value of key is expected to be the value of the key attribute of the corresponding API Key.

See the Validation of API Keys section below for more information.

It is possible to generate a short-lived (up to 24 hours) security token for an API Key. The token contains the same permissions as the API Key which it was generated for. There are two ways to generate and use these tokens:

If multiple authentication mechanisms are used at the same time in a request, only the first one will be evaluated. The order is as follows:

The interface returns all results in HAL format. This means that they are not simply raw data, such as traditionally unstructured content in JSON format.

The HAL format offers the advantage of simple but powerful structuring. In addition to the required content, the results contain additional meta-information on the structure of this content.

Example

In this example a filtered query was sent. Without knowing the exact content, its structure can be read directly from the meta information. At this point, the REST Interface returns three results from a set of five documents corresponding to the filter criteria and displays them on a single page.

The results of REST Interface are always delivered paginated. To control the page size and requested page, the HTTP query parameters pagesize and page can be used for GET requests. The default value for the pagesize parameter is set to 20 in the CaaS platform and the maximum is set to 100. These values can be changed in your custom-values.yaml in case of on-premises installation. For more information, see the RESTHeart documentation.

Filters are always used when documents are not to be determined by their ID but by their content. In this way, both single and multiple documents can be retrieved.

For example, the query of all English language documents from the products collection has the following structure:

https://REST-HOST:PORT/Database/products?filter={fs_language: "EN"}

CaaS documents can reference other CaaS documents. When fetching documents the referenced content will often times also be needed. In that case, the query parameter resolveRef can be used to avoid sequential requests.

The following two JSON document examples demonstrate this:

In the first document, the JSON at formData.ps_audio.value.url contains an absolute URL to another document in CaaS. The following request example shows how that reference can be resolved in the same request:

The value of the query parameter must specify the path in the JSON to the URL. The response then contains an additional attribute _resolvedRefs:

The resolveRef parameter can also be used for queries on collections. In this case, the references in all returned documents are resolved. The documents of the resolved references are collected in a new document, which is added to the array of the response.

Global API Keys are cross-database and are managed in the apikeys collection of the caas_admin database. Unlike local API Keys, they allow permissions to be defined for multiple or even all databases.

Local API Keys are defined per database and are managed accordingly in the apikeys collection of any database. Unlike global API Keys, local API Keys can only define permissions for resources within the same database.

The authorization of an API Key is performed on the basis of all of its permission entries. Its permission entries are defined in the permissions attribute.

The url attribute of a permission is used to check whether access should be granted. The value is compared with the URL path of an incoming request. What type of comparison is executed depends on the mode of the permission.

There are three different modes:

Local and global API Key distinction

The following table includes examples that illustrates the authorization distinction that is made for local and global API keys when using the permission mode PREFIX or REGEX.

The following endpoints are available for managing API Keys:

The database is based on the type of API Key.

The following snippet shows the example creation of a local API Key.

In this example, a new ApiKey is created via curl, which has the appropriate permissions set in the (via the url attribute) to the specified collection.

Each API Key is validated against a stored JSON schema when created and updated. The JSON schema secures the basic structure of API Keys and can be queried at /<database>/_schemas/apikeys.

Further validations ensure that no two API Keys can be created with the same key. Likewise, an API Key must not contain a URL more than once.

If an API Key does not satisfy the requirements, the corresponding request is rejected with HTTP status 400.

If the JSON schema has not been successfully stored in the database before, requests are answered with HTTP status 500.

If you have CaaS Connect in use, predefined indices are already created that support some frequently used filter queries. The exact definitions can be found at https://REST-HOST:PORT/Database/Collection/_indexes/.

If the predefined indices do not cover your use cases and you observe long response times or even request timeouts, you can create your own indexes. The REST Interface can be used to manage the desired indexes. The procedure is described in the RESTHeart documentation.

To create or update a GraphQL app, send a PUT request including an app definition to the following URL:
https://REST-HOST:PORT/<tenant>/gql-apps/<tenant>___<name>

A GraphQL app can retrieve data using MongoDB queries or aggregations. See Object Mappings for more information. An example using an aggregation to fetch data is provided in the appendix.

Creating the definition will provision an endpoint at the following URL:
https://REST-HOST:PORT/graphql/<tenant>___<name>

For more information on how to execute GraphQL queries see chapter GraphQL API.

There are two ways for creating a GraphQL app:

Using JSON body ("application/json")

The app definition with the sections descriptor, schema and mapping is passed in the request body (content type application/json) . The descriptor.uri parameter must match the last path segment of the URL (i.e. <tenant>___<name>) and is not optional in the CaaS - contrary to what is stated in the RESTHeart documentation.

Using file upload ("multipart/form-data")

The app definition and schema can also be created/updated using a multipart upload (content type multipart/form-data). This allows storing the app definition and schema as separate files and more importantly, the schema does not need to be JSON encoded when creating/updating the app.

To upload the app definition and schema, each must be present in the request as individual parts:

To delete a GraphQL app, a DELETE request is made to the following URL:
https://REST-HOST:PORT/<tenant>/gql-apps/<tenant>___<name>

Under certain conditions, such as after restoring individual collections from a backup, it might happen that the previously created GraphQL apps are out of sync. In such a case, all tenant-specific GraphQL apps must be resynchronized.

To trigger the resynchronization of all existing GraphQL apps of all tenants, send a POST request against the /_logic/sync-gql-apps endpoint.

This chapter describes a use case for a GraphQL application as an example. For this we will outline the individual steps that belong to the creation of a GraphQL application and its later use.

Create the GraphQL app definition

In the example scenario, a GraphQL application is created that can be used to query data records located in the CaaS. The data sets used here are the products from the example project of the fictitious company “Smart Living”. Image references and product categories in the data sets are resolved directly.

The entire command to create the GraphQL app definition for this example scenario looks like this.

Schema of the GraphQL app definition

The schema used for the example contains the following definitions.

Lines 1, 9 and 14 of the schema are the starting point for the type definitions of the objects used in the GraphQL app. In addition, each GraphQL schema contains a query type (line 22) that defines what data can be queried by a GraphQL app. More details about schemas in GraphQL can be found in the GraphQL documentation.

In line 23 we define a query with the name products, which returns a collection of [Product]. To specify the languages in which we need this data, we add the _language variable. As most of our customers are German or English, we also add a default value of ["DE", "EN"]. This marks the variable as optional.

Mapping the GraphQL app definition

The GraphQL app definition mapping represents the connection between the schema and the data in the database. Each type described in the schema generally requires an explicit entry, so this part of a GraphQL app definition is usually the longest. There may be situations where the fields in the type should be named exactly as the keys of the data. In this special case, no explicit entry in the mapping is necessary. For details about the mapping in GraphQL app definition, see the corresponding chapter in RESTHeart documentation.
The following example is an excerpt from creating a GraphQL app definition and clarifies some use cases.

The first use case considered is the so-called field to field mapping. In this type of mapping, a field in the type is assigned a corresponding attribute of the data. An example of this can be seen in line 3, where the field Category.name from the schema refers to attribute displayName from the data.

The second use case is the field to query mapping. Here a field in the type is mapped to the result of a data query. An example of such a mapping can be found in line 45ff: the field Query.products is mapped by the data found in the REST Interface under /mycorp-dev/d8db6f24-0bf8-4f48-be47-5e41d8d427fd.preview.content and correspond to the filters entityType": "product" and "locale.identifier": { "$in": { "$arg":"_language" } }. This means that exactly those products are queried which are located in the defined source, represent an entity of “product” and use one of the language abbreviations passed in the “_language” argument.

Another example of a “field to query mapping” can be found starting at line 29. In this mapping definition, the product categories, which are maintained in separate records, are identified using a foreign key relationship. The complete entry from line 29-42 states that the product.categories field will list all product categories that are under /mycorp-dev/d8db6f24-0bf8-4f48-be47-5e41d8d427fd.preview.content, whose identifier is stored in the formData.tt_categories.value.identifier field, and whose locale.identifier exactly matches what is in the product record as locale.identifier. Since a product can reference multiple categories in the dataset under formData.tt_categories.value.identifier, the key $in is used here.

Using the GraphQL app

Requests can now be made to a GraphQL application using this app definition.

This request example shows how to call the GraphQL app using curl. The app is always available at /graphql/<descriptor.uri>. Through this query, product data is retrieved depending on the variable $lang. The variable is passed as a value to the _language argument defined in the schema. Since a default value for _language is included in the schema, specifying a value is optional in this scenario. Further details on query arguments and variables can be found in the GraphQL documentation.

Aggregations can be used to create complex queries to the CaaS. For example, it’s possible to dynamically add a computed attribute to the returned documents. Much more complex aggregations can be created. A full list of possible aggregation stages and operators can be found here.

This tutorial serves as an introduction to creating and using GraphQL apps. To complete it you need to be familiar with the REST Interface in terms of authentication and querying documents.

GraphQL apps allow you to query your CaaS documents via your own GraphQL API endpoints. These endpoints also make the document data accessible in a specific format/schema that you define.

In the following steps we will be creating such a GraphQL app and use it to query document data. We’ll be using curl to send HTTP requests, but you can use other alternative tools.

The errors

indicate the presence of orphaned file chunks in the MongoDB data. The orphaned data can be removed with the following mongo shell script: