Archive old project states

By using the schedule action “Archive old project states” (FirstSpirit ServerManager / Project properties / Schedule management / Add / Actions) it is possible to swap out files which are not longer required (automatically and regularly at a defined point in time):

Advantages:

• The project size is reduced.
• Loading times are reduced.
• The FirstSpirit Server's overall performance is improved.

Repositories: Archiving and versioning of project data

FirstSpirit uses repositories to archive and maintain version histories of project data. Each project has a repository in the server directory data\projects\. For each action made in SiteArchitect, data is written to the repository. This applies to actions that create new elements as well as actions that delete elements. In addition, deleted elements are not removed from the repository. Since new data is always being added, the repository will continue to grow and will always require more hard disk space.

Archive files: Move project data from the repositories

The “Archive old project states” is used to carry out archival of the selected project so that data which are no longer required are moved out of the project and into a repository, reducing load times and increasing the performance of the FirstSpirit server. Data are moved from the repositories to archive files. Archive data that are no longer required can subsequently be deleted in order to free up hard disk space.

Each project has its own archive folder in the server directory archive and the data to be archived are moved to an archive file (tar.gz format) in the corresponding project folder. However, they can be viewed later using the Archive function in the server and project properties and can be installed as needed.

For more information on the FirstSpirit archival concept, see Project archiving.

Use the archive function with caution. Depending on the configuration, the version history may no longer be complete after archival. Thus, even older revisions may no longer be restored properly. During an export, only the currently available project status is exported without any existing archives or archived project statuses after archiving.

Monitoring: Storage space monitoring during archiving

During archival, the available storage space of the volume in which the repository directory is located will be monitored. If the available storage space falls below the following thresholds, the archival run will be aborted:

• option hdd.limit (configuration file fs-server.conf)
• the available storage space is less than half of what was available at the time archival was started

If archival is aborted due to these conditions, the following message will be written to the server log:

repository iteration interrupted, file system usage limit reached - lastId=[ID]

The log entry mentions the ID of the last archived element of this archival run. When the archival schedule task is started again, archival will continue from this point.

Configure task: Project archiving

The archiving criteria can be specified using the following options:

Version history

The user can specify in this area the period of time when archival should take place or the period of time the version history should be preserved.

Maintain version history at least 120 day(s) (partial archiving): this option is used to specify the period of time for which the entire version history with all revisions should be preserved. If, for instance, 120 days is set, archiving only takes into account revisions that are older than 120 days at the start of the archive schedule entry. All changes that are more recent than 120 days at the start of the archive schedule entry can also be completed without a gap even after archival.

Do not maintain version history (complete archiving): if this option is selected, the entire version history is taken into account when the schedule entry is executed. All data that are no longer required (see Minimum project archiving requirement for more information) are moved to the archive file.

Objects

The user can specify in this area what type of data is to be archived.

Content, media and data content: if this option is selected, all content from a project's page, media and data stores is archived (that is, everything except templates).

Note on archiving database contents:

Please note that there are size limitations when archiving database content: If the amount of all database entries is larger than 90% of 8 GB the archive process for the database entries will be cancelled. In this case a WARN will be logged (in the file fs-server.log), e.g. WARN 09.08.2010 12:50:42.080 {seID=369117} (de.espirit.or.impl.AbstractSessionHandler): cleanup for entityType=\'cases\' aborted, tar entry size limit reached! [schema=P222005_222001] Execute further archiving schedules for archiving the remaining database entries.

Note on archiving “automatically calculated resolutions”(*):

• If the “Content, media, and data content” option is activated, all automatically calculated resolutions that are no longer required are subsequently removed from the project and are no longer carried over to the archive file. These are calculated media/pictures from the image cache (MEDIA_STORE_CACHED_PICTURES) whose resolutions are no longer available in the project (i.e. which were previously removed via Project properties – Resolutions – Delete).
  The automatically calculated resolutions that are no longer required are removed regardless of the additional settings in place for the project archiving process (e.g., for “Version history” and “Options”).
  
  (*) What are automatically calculated resolutions?
  Media/Pictures can exist in various resolutions in the project. f necessary, the system automatically creates image data in the correct resolution, when a medium is being requested in a certain resolution for the first time. Such data is saved on the server in an image cache (MEDIA_STORE_CACHED_PICTURES). This method speeds up generation times, as generating large numbers of media in many different resolutions is very time-consuming. However, the image cache increases the project's data volume. 

Templates: if this option is selected, templates are archived. This option can only be selected in conjunction with “Content, media and data content”.

System data: system data are information generated by the system for each action in SiteArchitect (e.g. creating or deleting objects, releases, etc.) (see also Version history and Revisions). If this option is selected, the data of closed workflows and internal revision media are archived in addition to system data that are not longer used. Revisions for which user data no longer exist are deleted completely (i.e. all internal files belonging to this revision). This also includes the UI files. Archived revisions can later be viewed using the “Archive” function on the “Revisions” tab.

If closed schedules are also to be archived, the boxes “Content, media and data content”and“System data” must be checked. In this case, all files are archived that are associated with a schedule entry that was closed at a particular point in time within the archival period.

Options

The user can specify in this area whether only objects or also version history entries that are no longer needed are to be archived.

archive objects marked as deleted only: if this option is selected, only objects that were deleted are archived.

archive deleted objects and version history no longer required: if this option is selected, in addition to deleted objects, version history entries that are no longer required are archived as well. This means that the version history of objects is reduced. In any case, however, the full version history from the period of partial archiving (as long as this option is selected), the revisions from the last release state (if present) and the current edited state are preserved.

Waiting time per archiving step: when archiving a high volume of data, this value can be used to set a pause (in milliseconds), which is added between each step in the archival process. This reduces the load on the server during the archival process.

Runtime

Since archiving a high volume of data can take a long time, a maximum runtime limit for archiving a project can be set. Users can specify the amount of time allowed for archival in the “Runtime” area.

Limit maximum archiving runtime per run to 60 minutes: the user can set the maximum number of minutes for archival before the process is stopped. The default setting is 60 minutes. The next time the action is started (manually or automatically), archiving begins where it left off when last stopped. An archive file is created specifically for each of these “partial archives”.

Do not limit archiving runtime: if this option is selected, archiving will continue without any time limit until it is completed.

Creating and configuring an archival schedule action via API

Starting point FirstSpirit Access API: interface ProjectCleanupTask (package: firstspirit.access.schedule).