Generation

Creating and delivering content in FirstSpirit is based on the “pre-generation” principle. Wherever possible, the content is not first produced on the live system at the time of the user query (live rendering), but is instead already on the FirstSpirit server (i.e. in a less time-sensitive offline mode) for what is called “generation”.

By default, a schedule entry is created with each project. The schedule entry is used to carry out full generation (“generate full” schedule) and partial generation (“generate partly” schedule). As with any schedule entry, these schedule entries an be managed via the project properties (“schedule management”, see Schedule management (→Documentation for Administrators)). These schedule entries can also be started from SiteArchitect with the relevant configuration set (“Project / Generate project” or “Generate partial project”, see Project (→Documentation FirstSpirit SiteArchitect)) if the corresponding permissions are present (see below).

In addition, individual generation schedules can also be set up in schedule management. Thus, specified start nodes can be defined for individual, project-specific generation (partial generation) or even a generation schedule that only includes the released changes to the last deployed project status (delta generation).

All generation options configured for a project can be started in SiteArchitect via the “Run schedule” menu item (as long as the editor has the permissions necessary for this).

Full generation

The purpose of a full generation is to generate the entire project. This process can be very time intensive, depending on the scope of the project. Therefore, using partial generation or delta generation is recommended for regular, smaller generation actions.

Generated content: All released

• page references from the site store are included as well as
• all released media that are referenced in the output of these page references.

Media that are not referenced in the project are not included in a full generation.

Partial generation

The purpose of a partial generation is to generate certain subsection of a project quickly and accurately. Depending on the configuration, for instance, only certain start nodes, only certain media objects and/or only changed project content are included.

Generated content: The following are included:

• all released page references from the Site Store, starting from the specified start nodes,
• all released media that are referenced in the output of these page references, and
• all released media from the Media Store, starting from the specified start nodes.

Media generation

In principle, this is a simple partial generation where only the start nodes from the media store are defined.

The purpose of a media generation is to enable faster deployment of individual partial sections. Saving the processor intensive media generation into separate partial generation schedule entries improves performance, especially when executing multiple small partial generation schedules, since for each generation only the desired media are included. However, this option is also suitable for deploying media in a remote project, for instance (“remote media”, see also the Remote access page).

Generated content: All released media from the Media Store are included, starting from the specified start nodes.

Delta generation

Typical partial generations use a simple strategy to reduce the scope of the generation. Not all of the (released) project content is generated; only the partial sections of a project that are currently being edited are generated instead. The disadvantages of these generation strategies are obvious. All content below the defined start nodes are always generated, regardless of whether they have actually been changed or not.

Delta generation utilizes a more efficient generation strategy. It no longer involves generating complete projects or complete subtrees of a project, but rather only the released content that has actually changed since a previous revision state. The purpose behind delta generation is to avoid unnecessary and time-intensive generation and redeployment of unchanged content.

Delta generation only supports the generation of the release state. In other words, it is only possible for projects that use the release function (the “Use release” option in the project settings must be enabled).

How delta generation works:

• The relevant previous revision is read out and saved in the property
• All objects in the project that have changed are determined (changes since the revision indicated by the #deltaGeneration.last_execution variable).
• All of the new nodes to be generated (i.e., structure folders, page references, etc.) based on these changed objects and taking the defined dependency rules into account are determined.
• These nodes are added to the next (=subsequent to the script) or explicitly specified generation task.

The changes to the earlier state of a project are determined by the FirstSpirit Access API. To do this, a script (action:“script”) must be activated before the actual generation (action: “generate”).

Access API documentation

• DeploymentUtil
• DeltaGeneration
• DeltaGeneration.ChangeSet
• DeltaGeneration.DeletedPageInfo

Determining the revision

The information identifying the revision starting from which changes should be taken into account is stored persistently in the #deltaGeneration.last_execution schedule variable – its name is defined in the constant

DeltaGeneration.DELTA_GENERATION_LAST_EXECUTION

If the corresponding variable is not (yet) available, full generation is carried out automatically.

The #deltaGeneration.last_execution variable is read out automatically when DeploymentUtil.createDeltaGeneration(context) is called. It is copied to a property of the same name and then (for subsequent executions of the schedule) set to the start time of the schedule. If required, the value of the variable can also be adapted using a user-defined script task (see below).

In the event of an error, the #deltaGeneration.last_execution variable is not reset automatically.

The variable might need to be reset to its previous value if deployment fails and / or is not executed due to generation errors. The schedule variable must be reset to its previous value via an additional script. For this purpose, the old value saved automatically in the property of the same name can be accessed:

context.setVariable(DeltaGeneration.DELTA_GENERATION_LAST_EXECUTION, context.getProperty(DeltaGeneration.DELTA_GENERATION_LAST_EXECUTION));

Otherwise, only the changes between the last (interrupted and thus not deployed) execution of the schedule and the current schedule would be taken into account in the next execution of the schedule. The changes from the defective execution would not be taken into account.

The schedule variable (and thus the value of the determined revision) is only available within the schedule. Other generation schedules are not taken into account automatically.

If there is more than one generation schedule (e.g., full, partial, and delta), changes that have already been deployed by another schedule through an intermediate generation are also taken into account (regenerated unnecessarily) in the context of a delta generation schedule. If this is not desired, the time starting from which the changes should be determined can be adapted by manipulating the DeltaGeneration.DELTA_GENERATION_LAST_EXECUTION schedule variable accordingly before calling DeploymentUtil.createDeltaGeneration(context).

Determining the changed objects

“Dependency rules” are used to define which changes in a project will trigger the generation of which nodes. The dependency rules that are activated as standard are defined in the DeltaGeneration.DEFAULT_DEPENDENCY_RULES constant.

Start nodes that have been entered are not considered to be restrictions: Any start nodes that have already been defined in the corresponding generation task have no effect on the determination of changes (in the delta). In particular, they do not restrict determination to the corresponding subtrees.

Determining dependences

There will often be dependences between different elements within a project. If these dependences are also to be taken into account during a delta generation, this requires dependency rules.

All dependency rules always take changes to the release state of an element into account. A rule will only apply to simple changes in stores that do not support release (e.g., Template Store).

The following dependency rules are available:

• PROPAGATE_PAGE_CHANGES: if a page changes, all pages with references to it will also be considered to have changed.
• PROPAGATE_SECTION_CHANGES: if a section changes, all pages containing references to this section will be considered to have changed.
• PROPAGATE_MEDIA_CHANGES_TO_PAGESTORE: if a medium (image or file) changes, all pages in the Page Store containing references to this medium will be considered to have changed.
• PROPAGATE_MEDIA_CHANGES_TO_SITESTORE: if a medium (image or file) changes, all page references or menu levels in the Site Store containing direct references to this medium will be considered to have changed.
• PROPAGETE_TEMPLATE_CHANGES: if a template changes, all pages in the Page Store that are based on this template will be considered to have changed.
• PROPAGATE_MEDIA_CHANGES_TO_CONTENTSTORE: if a medium (image or file) changes, all datasets in the Data Store containing references to this medium will be considered to have changed.
• PROPAGATE_MEDIA_CHANGES_TO_TEMPLATESTORE: if a medium (image or file) changes, all templates in the Template Store containing references to this medium will be considered to have changed.
• PROPAGATE_PAGE_CHANGES_CASCADING: if a page changes, pages that link to this page directly or indirectly (via its page reference) will be considered to have changed too.
• PROPAGATE_CONTENT_PRODUCER_MOVE: if a page reference moves, it includes all pages that link to this page.
• PROPAGATE_GCA_CHANGES: if a global content area (GCA) changes, all pages that display this global content will be considered to have changed.
• UPDATE_PAGEREF_WHEN_ENTITY_HAS_CHANGED: controls the behavior of delta generation of detail pages for content projections. (see the following paragraph Dependencies associated with content projections)
• CHANGED_MEDIA: all changed media are generated via a corresponding media generation process.

Dependencies associated with content projections

The UPDATE_PAGEREF_WHEN_ENTITY_HAS_CHANGED dependency rule controls the delta generation behavior of detail pages for content projections. Detail pages are page references that represent individual datasets, i.e. for which the value 1 is entered in the field “Number of entries per page” on the “Content” tab.

• If this rule ist set (or if no DependencyRules are set), the entire page reference that is used to generate this dataset is also regarded as changed when this dataset is changed. As a result, all detail pages of the corresponding page references will be regenerated during a delta generation. In particular, detail pages that do not contain the changed dataset but are based on the same table are also regenerated.
• If this rule is explicitly not set, only those detail pages will be regenerated during a delta generation that represent individual changed datasets.

Note: This rule has no influence on the handling of overview pages. These are generally always regenerated when a dataset has been changed in the underlying table.

Example

Detailed instructions on configuring a delta generation and a sample script for determining changes in a project can be found in the Community:

• Delta generation - part 1
• Delta generation - part 2

Generation directories

By default, generated project content is stored by FirstSpirit Server in the default generation directories (fs5staging, fs5preview and fs5webedit) or in local project generation directories such as the following:

for content (without media): fs5staging / project ID / schedule ID / language channel_template set / project structure

for media: fs5staging / project ID / schedule ID / media / language channel / project structure

• Project ID: When a project is created, this ID is assigned to it automatically by the system. It can be found in the project overview of the ServerManager, for instance.
• Schedule ID: The system assigns this ID automatically to a schedule when the schedule is created. The default schedules for full or partial generation are included in every newly created project. The schedule ID can be found in the overview of schedules in the project properties (“schedule management”, see Schedule management (→Documentation for Administrators)) or in a schedule itself, for instance.
• Language channel_template set: A subdirectory is created for each project language (see “Project properties / Languages”, see Languages (→Documentation for Administrators)). The abbreviated code created in the project properties is used as the identifier. If several template sets are present (see “Project properties / Template sets”, see Template sets (→Documentation for Administrators)), a directory is created for each template set in each language. When this is done, an abbreviated code is appended to the language code for the particular template set, separated by an underscore; for example: de_1.
  However, this method greatly depends on the selected URL creator: default URLs (de, de_1, en, en_1), Infix URLs (abc.de.html, 1/abc.de.html, abc.en.html, 1/abc.en.html), AdvancedURLs (MeinTitel.html, MeinTitel-1.html, MyTitle.html, MyTitle-1.html). In addition, a distinction is made between language-independent and language-dependent media: media/de/abc.jpg, media/en/abc.jpg vs. media/bcd.jpg
  For more information, see Generating default URLs and AdvancedURLs.
• Project structure: The hierarchical structure of the Site Store is recreated under the language/template set directory: a directory is created for each released folder in the Site Store (“menu level”, see Settings at menu level (→Documentation FirstSpirit SiteArchitect)); these directories contain the released page reference(s). The hierarchical structure of the Media Store is recreated under the “media” folder: a directory is created for each release folder in the Media Store; these directories contain released media. In the case of language-dependent media, each language receives its own directory where its respective media are placed.

Example
For media: .../fs5staging/11599/11640/media/de/medien_ordner/bild.jpg
For content: .../fs5staging/11599/11640/de_1/menueebene_1/seite.html

The generation directories can be adapted to specific projects. The structures depend on the individual project configuration and may vary from the structures shown here.

Access to these generation directories is protected, which means that when calling this content, user authentication is required, as long as it has been configured.

Permissions

In order to carry out full generation actions, server or project administrator permissions are required as standard. Otherwise, the “Generate project” and “Generate partial project” submenus are disabled under the “Project” menu in SiteArchitect.

For an editor without administration permissions to be permitted to start a full and/or partial generation, he (or alternatively, a group) must be selected in the schedule properties of the “Interactive execution” area (see Execute generation (→Documentation for Administrators)).