Process run information about all activity instances and process data values that are part of a business process instance is stored in the Audit Trail. There are situations in which this detailed information is not necessarily needed for all process types. Pure BPM-driven solutions are mainly requiring a full and detailed Audit Trail on an abstract level or business view. There are also many use cases in which Stardust is used more as a routing and transformation engine, whereas the persistence of all process run information is even less important. For these use cases the performance like latency or throughput is more critical. To reduce the amount of data stored in the database and increase the overall runtime performance of the engine, Stardust provides the option to use transient processes.
Transient processes keep instances of the process in-memory only during the course of execution. No audit trail is persisted at all during or after the execution of the process instance. The corresponding objects are created during process execution and are available in the context of the activity threads running for that process instance.
Almost no remote calls to the database and no persisting of data is necessary, thus transient processes are executed faster and with lower latency.
If there is system downtime during process execution, the instances of a transient process do not leave a trace in the audit trail. However in case of unhandled faults, the information is written to the audit trail.
To summarize, using project transient processes provides the following performance enhancements:
There is almost no contention like Java synchronization or database locks, when running transient process instances due to:
The process run information of transient processes is only kept in an internal in-memory storage, e.g. Hazelcast, during execution of the process instance and will be discarded when the process instance is completed.
For details on how to use transient processes refer to chapter Tuning for low Latency and high Throughput in the Operation Guide.
In usual processing of process instances several activity threads can be running in parallel in the scope of one process hierarchy. This is typically the case if several parallel transitions originate from one gateway of type AND -split. This could result in a very large number of parallel running threads. To avoid such a possible overhead, all activity threads in a transient process instance are executed sequentially. As in the normal (persistence Immediate ) case, the first transition modeled for that AND split will be executed first and in the same transaction. All other activity threads will be invoked via Messaging. The corresponding messages will be processed:
Due to technical restrictions, several events can occur during the lifetime of a process instance that are incompatible with transient processing. These events will force the engine to persist the audit trail for this process instance in order to avoid permanent loss of data. This is the case when the process instance cannot be further processed without asynchronous actions of an external party, e.g. a participant or external system. An example for such a required interaction is the processing of an interactive activity or the triggering of a recovery. The current state of the process instance audit trail will be persisted and any subsequent processing of the process instance will be done in persistence mode Immediate for the following cases:
Currently, asynchronous subprocesses are not supported for transient processing. Whenever the workflow of a transient or deferred process instance reaches a subprocess activity that invokes an asynchronous sub-process, the audit trail persistence is set to Immediate for both the parent as well as the sub-process.
Whenever a process instance changes persistence mode from Transient to Immediate or Deferred for one of the reasons listed above, this change in persistence mode will propagate and be applied to the entire process hierarchy from that point of execution onwards.
In case the property Carnot.Engine.ErrorHandling.ApplicationExceptionPropagation is set to value always and an ApplicationException occurs, the whole transient process instance graph is purged from the in-memory storage and written to the audit trail database such that a subsequent recovery action may recover the process instance. This also applies if the property is set to onRollback and an activity thread rolls back due to an uncaught exception and there are other activity threads already completed, e.g. committed against the internal cache.
As long as a process instance is executed transiently, pull events, i.e. events triggered by the event daemon, are not processed. Instead a warning is logged whenever a transient process instance having a pull trigger event bound is written to the in-memory storage. This can be the case for example if its transaction is committed. The according log message looks as follows:
Event binding 'Event binding [oid = 1, type = 2, model oid = 2, object oid = 1, handler oid = 2]' applies to a transient process instance and will therefore not be processed by the event daemon.
Note that the usage of any audit trail storage mode other than Immediate requires a second level cache configured in the runtime, e.g. Hazelcast. Currently, the following platforms are covered:
Retrieving a process instance or its related data via the WorkflowService or the AdministrationService will always return a snapshot of the currently executed process instance. After completion the snapshot is completely gone (unless its deferred). Since a transient process instance by definition is executed without waiting in a defined state, e.g. a suspended activity instance, there are no means to modify the state of the process instance safely. In general, to perform a safe modification you have to first determine the current state and afterwards modify it based on the retrieved state. However in case of transient processes as soon as the state has been retrieved it might have already changed. Therefore the outcome of any modifying operations is undefined.
All QueryService methods operate against the persisted Audit Trail database exclusively. Hence any transient process instances do not influence any QueryService method results. Transient processes will never change any query results, even after their completion.
This use case describes the execution of a transient process instance, for example a process instance that executes without leaving a trace in the database.
The precondition for this use case is one of the following:
The transient process instance has been completed without leaving a trace in the database.