Daemons

Daemons are tasks that run continuously in the background of the Stardust Engine. They play an important role in setting automatic triggers and events. The system provides two daemons by default, the System daemon and the Event daemon. Apart from them, daemons are created for every pull trigger defined in the model, like for example mail trigger or timer trigger.

The daemon execution is triggered at fixed intervals by a timer service. This timer service will start a short-living thread for each daemon type that is supposed to be started. Stopping the daemon stops the timer, so that no more executions are scheduled. Starting the daemons means that a corresponding timer is created that schedules execution jobs. A locking system guarantees that there are no overlapping job executions of the same daemon, i.e. a second execution will not start if the previous one has not yet finished. Server stopping will automatically stop the timers.

The periodicity of the timer service can be individually configured in the Stardust properties file. For details refer to section Daemon Properties. The total loop duration is the maximum (not the sum) of the duration of the loop tasks and the configured periodicity.

A running daemon is automatically restarted after a server restart.

Behavior in Case of Exception

If an execution fails with an exception, the next execution is triggered by the system timer event not being affected by the former exception. An exception is errors based on database connection problems.

The following functionalities are available to start or stop daemons necessary to run Stardust:

Daemons communicate with the Stardust Administration Perspective or with the console via timestamps in the audit trail database. If you intend to start and stop daemons from multiple machines, the machine system time must be the same on all machines.

Daemon Types

The following predefined daemons are currently provided:

Mail Daemon

On the technical side, the mail daemon periodically polls the mail servers of all mail triggers defined for process definitions of a workflow model. If a new mail matching the specification of the mail trigger is found, a process instance is created.

The process instance is created asynchronously to the daemon execution. All the data contained in the trigger mail, and possibly necessary for executing the activities in the process, are passed as workflow data to the process instance.

Timer Trigger Daemon

The timer daemon periodically checks if timer-based triggers defined for the running process instances have to be fired. If a timer-based trigger matching the current timestamp is found, a new process instance is created asynchronously to the daemon execution.

Event Daemon

The event daemon checks whether timer events were fired and need processing. The daemon periodically checks all event handlers bound and executes the event actions corresponding to matching handlers.

System Daemon

The system daemon periodically checks if daemon actions are defined and executes those actions. Please note that currently only one action is available, which is responsible for password expiration notification. In case this action is started and password rules are set, it is checking password expiration and sending a notification mail according to the password rules settings. Per default it runs once per day.

Prioritization Daemon

The prioritization daemon:

Daemon Log Entries

When encountering an Exception, except in cases where it would not even be able to write to the database, because for example of connection problems, log entries are written by daemons. These log entries are stored in the audit trail, in the table daemon_log. Please refer to the chapter The Repository Model of the Operation Guide for details on this table.

Acknowledgment

The acknowledge mechanism determines to wait at least one timer periodicity before a guaranteed "is running" is received.

Daemon Properties

Daemons are configured via a set of parameters, which, depending on the operating mode, can be provided either in the deployment descriptor of the DaemonListener or in the carnot.properties.

Periodicity

The most important property for daemons is the periodicity:

<daemon name>.Periodicity

where <daemon name> includes the following:

The periodicity controls how often the daemon should perform its task. By setting the property you can specify a time interval expressed in seconds between the executions to achieve more accuracy. The default value is 5 seconds.

Behavior due to large periodicity

If the periodicity of daemons is increased by using these properties then the following behavior is expected:

  1. An Acknowledgment status Response Required might occur, but the daemons are performing the required actions nonetheless.
  2. If the action point falls between the custom duration settings, it will be performed only when the daemons poll next time. For example: If a re-submission is supposed to happen at a specific time and the daemon is polling before that time and after that time according to the custom setting, the re-submission will occur at the last polling time due to the long periodicity setting.

Timeout

The default value for daemon timeout period is set to 5 seconds. You can adapt this value in the carnot.properties file by setting the Carnot.Engine.Tuning.FindDaemonLogQueryTimeout parameter. The value "0" means that no timeout is performed.

Transactional Behavior of Event Bindings

All the events that need to be handled by the event daemon or timer trigger are stored in the event_binding table. When the thread for the event daemon starts, it reads all the entries in this table, which have a target timestamp that expired. For each event action, a separate transaction is started. Thus, if the transaction rolls back, the daemon thread itself will remain running and process the next event. The rolled back event will be handled during the next execution of this daemon type. The event itself is handled in a separate thread, whereby the following two types of exceptions are caught:

Unrecoverable exceptions will remove the event binding from the database table, mark a log entry, and subsequently commit (so that the daemon thread can continue with the next event). The runtime exceptions will first start a retry mechanism. After five times it gives up and mark this event as disabled. It does so by adding 1024 to the type field, which else holds 1 for process instances and 2 for activity instances (1025 means disabled process instance).

Daemons in a Clustered Environment

In a clustered environment the behavior of daemons depends on the server configuration. With standard configurations, no failover is provided and it is not guaranteed that there is only one instance running. It is guaranteed however that a specific event or action is performed by exactly one instance, i.e. different instances will execute different sets of actions, possibly in parallel.

Note that, depending on the platform used, there might be cluster able timer bean implementations providing failover and uniqueness on the level of the application server. Please check with your specific user platform.

EJB Deployments

If all configured as clusters in EBJ mode, the EJB timer guarantees that only one timer event will be executed for all cluster nodes.

Spring Deployments

A heartbeat is executed on every Stardust instance. Every instance will check the daemon_log table, if a daemon execution is required, based on the last execution time in the database and the next execution time calculated by the periodicity. If a new run is required, a lock on the audit trail is acquired which will be available to only one engine. All other engines will not acquire the lock and only see the updated last execution time. Thus they wait another time. If no new execution is required as last modification time + periodicity > current time, the heartbeat sleeps and tries again during the next run.