Persistency Management guide [9.4]

Persistency management is the term used in Blueriq to address everything that has to do with persisting (or saving, storing) information and later on retrieving, displaying, updating and deleting this information.

In many systems, persistency management is all about databases and SQL-statements. In Blueriq, it is not. We deliberately made the technical way in which data is stored and retrieved not the business engineer's concern, let alone the end user's concern. In stead, Blueriq provides an intuitive canvas where a business engineer can design what information to persist when. Furthermore, easy to use services to create, read, update and delete information and a service to display information objects are provided. What happens behind the scenes is, as mentioned before, not of interest to the business engineer.

This guide helps you to understand the big picture and the concepts. Details can be found in the help files of the individual components.

Experienced Business engineers might use the Dossier Manager (Dossier plugin) for Persistency Management. The functionality described in this design guide is aimed to replace this Dossier Manager.

Terminology

The following terms play a role within persistency management in Blueriq and will be discussed in this design guide:

Aggregate definition
Entities, attributes and relations
Create, read, update and delete aggregates
Lists of aggregates
Standard metadata
Custom metadata
Versioning
Authorization
References

Aggregates

Organizations usually offer more than one product or service to their customers. All these products and services have their own cases, so multiple cases can be running concerning the same customer. To get a good overview of a customer there is a need for shared storage of information, which is shared among the several cases. To accomplish this, the concept aggregate is introduce in Blueriq. See the example below.

Shown above are three different cases (in grey) that are all part of a single business model of a fictitious insurance company. The first case concerns a medical insurance, the second a dental insurance and the third a home insurance. These cases use and share information (in blue). The medical and dental case both use the medical information and information about persons, whereas the home insurance makes use of the person information and information about houses. These information objects are called aggregates in Blueriq and will typically exist of one or more entities with their endogenous relations.

The reason that different aggregates can be created has to do with design, reuse as well as performance. Housing information is usually not needed deciding about a dental insurance. On the other hand, it is not wanted that a medical insurance case and a dental insurance case each use their own medical information, as they both concern the medical information of the same customer. Finally, loading unnecessary information has a negative impact on performance.

Aggregate definition

Aggregates are used to combine information that belongs together, and should be stored and loaded together. In Blueriq, information that belongs together is modeled in an entity, or as multiple entities with relations between them. An Aggregate is the selection of parts of your domain that can be stored and retrieved together in a single action, and consists of one or more entities. When an entity is added to an aggregate definition, all entities that are connected to the entity are part of the aggregate as well. At the moment it is not possible to create a precise aggregate definition, excluding for instance a few attributes and relations.

Shown below is the design of the aggregate Person information, in which the aggregate is blue, and the entities that are defined within the aggregate are pink.

Since it suffices to declare the main entities in a domain model that are to be part of the aggregate, the design of the aggregate Person only consists of the entities Person and Employment. However, the entity Person has relations with the entities Job and Household, so whenever a person is stored, his or her jobs are also stored, as well as the person's household situation. See also Aggregate design.

When storing such an aggregate, metadata is stored as well. This includes standard metadata, such as when the aggregate has been created and who created it, but also custom metadata that the business engineer can define on the aggregate definition by using expressions. This custom metadata can be seen as identifying attributes for an aggregate, but are not necessarily unique. In the example above, the Person aggregate could have the following custom metadata attributes: FirstName, LastName, SocialSecurityNumber, CurrentJob, HouseholdSize. As you see, custom metadata for an aggregate can originate from different entities, or even be a complex expressions, for example NumberOfJobs is calculated as COLLECT Job FROM Person.HasJobs. It is even possible to define custom metadata that does not originate from entities of the aggregate definition at all, since custom metadata consists of an expression.

Only the (custom) metadata is available in lists, as this information is stored in an efficient manner. When a new aggregate entry is created or updated, the (custom) metadata is updated as well.

Working with aggregates

In order for an end user to work with aggregates in cases, it must be possible that these aggregates are displayed in a list. This could enable the knowledge worker to look for a known person in the system if this person contacts the call center. This list functionality is very similar to the list functionality that is already available for instances, tasks and cases and is implemented as a Blueriq container.

To let the business engineer create meaningful aggregate lists, custom metadata attributes can be added to the aggregate definition and then used as column in the aggregate list. Below is an example of such a list, which illustrates sorting, filtering, custom metadata and paging. For more information on the aggregate list see AQ_Aggregate_List.

By letting the business engineer create different aggregate lists for different purposes, the end user will work with only the aggregates that are relevant for him. Since only metadata from these aggregates is retrieved from the database while displaying them - and not all data completely - this will boost performance.

Aggregates can be created, read, updated and deleted. For these specific actions, services are available in Blueriq.

Aggregate create service: AQ_Aggregate_Create
Aggregate read service: AQ_Aggregate_Read
Aggregate update service: AQ_Aggregate_Update
Aggregate delete service: AQ_Aggregate_Delete

Versioning

As shown in the aggregate list in the previous chapter, it is possible to specify versioning for aggregates. This means that when creating an aggregate, a version number is stored along with it. When updating an aggregate, the version to update has to be specified and the update itself will result in an increased version number. When reading an aggregate, a specific version can be specified. The same goes for deleting an aggregate. It is advised to use a control entity to store the aggregate IDs as well as the version IDs.

There are three versioning strategies when designing aggregates, which can all be modeled in Blueriq:

No versioning
With this strategy each create action will result in a new aggregate instance. Updating an aggregate instance means overwriting the original aggregate instance and deleting an aggregate instance will actually delete the aggregate instance from the database. Reading a non-versioned aggregate instance is trivial: it will read "the" aggregate instance, since there is only one.
Versioning all
With this strategy each create action will result in a new aggregate instance as well, but updating an aggregate instance means that a new version of the instance is stored alongside all previous versions. When deleting such a versioned aggregate instance, it is necessary to specifiy which specific version(s) to delete. Reading an aggregate instance also means that it has to be specified which specific version(s) to read.
Beware that the number of versions for an aggregate instance can grow out of control with this strategy. Also, think about the design of your aggregate list when using full versioning.
Versioning latest
This strategy is based on the previous strategy, but with some limitations for the end user. Versioning is still in place, so updating an aggregate instance will create a new version of the instance, but in this strategy reading will always return the latest aggregate instance version. Deleting an aggregate instance with this strategy means deleting all versions. Also be aware of the fact that the number of versions for an aggregate instance can grow out of control.

References between aggregates

In many situations you want that different aggregate entries refer to another. For example should the case aggregate refer to the person that this case is about.

We assume that there is no relation between the case entity and the Person entity as the Person would be stored together with the case if this is the case.

You should model an attribute at the case entity which stores the aggregate Id of the Person aggregate. When loading the case into memory, first the case aggregate is loaded. This retrieves the person aggregate Id, which can be loaded afterwards by a second service. When initially creating the aggregates the order has to be reversed. For updating, the ordering does not matter, as the Id does not change.

Authorization

There currently is no authorization on aggregate level. This means that you are responsible for prohibiting the end user to load certain aggregates by choosing adequate authorization on a page or at container level.

Migration

When your application changes over time, the migration of data becomes an important topic. Aggregates help you with this, but cannot solve all your problems. For complex scenarios it is advised to create your own database scripts for migration purposes. Database migration scripts generally do everything in a lazy manner, so that not the complete database is updating when the server starts.

A new attribute is added

When a new attribute is added to the domain, all already stored aggregates do not have this attribute. In this situation, it is possible to load existing aggregates, and the value for this new attribute is simply set to unknown.

An attribute is deleted

When loading an existing aggregate that still has a deleted attribute, then this attribute is ignored. Upon the next update action, the old entry is overwritten and the information of the deleted attribute is lost.

An attribute is renamed

Renaming an attribute is regarded as deleting the old attribute and adding a new one. Existing information in the old attribute is lost.

Metadata

For metadata the same principles hold as for domain attributes. A special case is a change in the expression of a custom metadata attribute. Here also the lazy principles applies. When the expression for a custom metadata field changes, the value in the database is updated the next time an update action is performed for the aggregate entry.