CHAPTER 3

 

THE METEOR2 WORKFLOW MODEL AND ARCHITECTURE

 

 

The objective of the METEOR project [KS95] at Bellcore was to facilitate and support the automation of enterprise-wide operations (workflows) that execute on a heterogeneous, autonomous and distributed operating environment. To achieve this objective, the METEOR model [KS95] offers a well-defined model and an intermediate language for specifying the workflows and the tasks, a compiler for the language, and a runtime enactment system that provides functionality to monitor the progress of the workflow execution, enforce the inter-task dependencies and manage the execution of each individual task.

The METEOR2 [SKM+96, WS97] workflow model extends the METEOR in terms of both the logical model and the runtime architecture to support large-scale multi-system workflow applications in heterogeneous and distributed operating environments. This chapter gives an overview on the basic concepts of the METEOR2 workflow model followed by a discussion on the overall architecture of the METEOR2 workflow management systems.

 

3.1 THE METEOR2 WORKFLOW MODEL

 

A workflow in METEOR2 can be modeled as a collection of tasks and a set of inter-task dependencies. In this section, we shall review the basic concepts of the METEOR2 workflow model, including the concepts of the METEOR2 task model, the different types of inter-task dependencies to construct a workflow and the workflow intermediate language used to specify a workflow.

 

3.1.1 THE METEOR2 TASK MODEL

 

The concepts of task, processing entity and interface are primary to the METEOR2 task model.

A workflow comprises multiple tasks. Different tasks may behave distinguishably according to the semantic of tasks, but the runtime system of a workflow need not know the execution details of a task. Only certain states of a task, such as starting of the task and the completion of the task, need to be monitored at runtime in order to enforce the correct execution of the task and the workflow. Therefore, the various heterogeneous tasks in the real world can be categorized into different categories based on the common structure of their runtime states.

In the METEOR2 model, task structures are modeled by using directed graphs [MSK+96]. The nodes in the graph represent the externally visible states of a task, while the arcs correspond to a task’s permissible internal state transitions. An internal transition can be either controllable or uncontrollable. An internal transition is said to be controllable if an enable arc outside of the task can trigger the transition; otherwise, the internal transition is said to be uncontrollable.

One of the characteristics of the METEOR2 workflow model is that it provides well-defined task structures to model heterogeneous workflow tasks in the real world. Tasks in the METEOR2 task model could be simple tasks or compound tasks. A simple task is an atomic activity that represents an indivisible logical unit in a workflow, while a collection of related simple tasks can be organized hierarchically to form a compound task. Both simple tasks and compound tasks can be transactional and non-transactional in nature [KS95].

Figure 3.1 The METEOR2 Task Model

 

A transactional task (see Figure 3.1) is a task that minimally obeys the atomicity property of a transaction and maximally supports the ACID transactional semantics [GR93]. The task structure contains four external visible states: initial, executing, committed and aborted. State transitions are uncontrollable outside of the task scope. Hence, a transactional task can be considered as a "black box" that once the task starts to execute, the workflow run-time system will only be aware of the task termination status after execution, either failed or committed. A good example of this kind of task is a banking transaction task. A transaction update is committed and saved to the account if and only if the transaction is completed successfully. Otherwise, the task should be aborted and none of the changes during the transaction will take affect in the account.

A non-transactional task (see Figure 3.1) is a task that does not support the atomicity or any of the ACID transactional properties [GR93]. The externally visible states of a non-transactional task are initial, executing, failed and done. Human activities are usually considered to be the non-transactional tasks.

Workflow tasks can also be categorized into user tasks and application tasks. User tasks are manual tasks that involve human actions or human-computer interactions. Application tasks typically comprise of scripts, computer programs or some other automated processes that need not involve human interactions. Here, application tasks are automated processes. The importance to distinguish user tasks and application tasks is for the distinct recovery strategies in the case of task errors. This would be discussed in details in chapter 4.

 

3.1.2 INTER-TASK DEPENDENCIES

 

As we discussed above, tasks contain external controllable states. The controllable state transitions of a task are dependent on the run-time states of other relevant tasks. Inter-task dependencies specify how tasks in a workflow are coordinated for execution in a semantically correct order [KS95]. Therefore, tasks in a workflow influence each other in a way that the execution outcome of a task could be seen as an event to trigger the state transitions of other tasks as well as its own.

In the METEOR2 model, two general types of inter-task dependencies are identified: control dependencies and data dependencies. Control dependencies specify how controllable transitions of tasks depend on the current observable states of other tasks, while data dependencies, which may be optionally associated with control dependencies, specify how the data constraints are associated with task executions (e.g., the activation of a task depends on the output data values of other tasks).

The control-flow of tasks in a workflow can be modeled as a directed graph in METEOR2. The nodes of a graph represent tasks and the arcs represent the dependencies. The Map Designer [Mur95, Lin97] is a tool that provides a Graphical User Interface to let workflow designers visually define both the control dependencies and data dependencies of a workflow. The output of the Map Designer is an internal representation of the workflow design using the Workflow Intermediate Language (WIL). We will discuss the WIL in more details in the next section. Inter-task dependencies are enforced by the workflow scheduler during the runtime.

 

3.1.3 WORKFLOW INTERMEDIATE LANGUAGE

 

METEOR2 uses the well-defined Workflow Intermediate Language to conceptually specify workflows [Das97]. The WIL is the continuation and enhancement work following the Workflow Specification Language (WFSL) and Task Specification Language (TSL) described in the original [KS95].

The current WIL is capable of specifying the inter-task dependencies (both control dependencies and data dependencies) of a workflow, defining input/output data objects passed among different heterogeneous tasks in a workflow and allowing workflow correctness issues (e.g., error handling and recovery information for each task) to be declared and associated with the workflow specification. [Lin97, Zhe97] has a detailed description about the Workflow Intermediate Language.

 

3.2 THE METEOR2 WFMS ARCHITECTURE

Figure 3.2 The METEOR2 WFMS Architecture

 

A WFMS is a set of tools that provide support for process definition, workflow enactment, and administration and monitoring of workflow processes [Hol94]. The METEOR2 WFMS is based on the METEOR2 workflow model that provides comprehensive and powerful tool sets for visually modeling processes and building workflows on a user-friendly GUI environment and a code generator that can automatically generates ready-to-run reliable workflow enactment systems using multi-paradigm workflow technology (either web-only or web-CORBA as the workflow infrastructure). In this section, we give a brief overview on the current METEOR2 WFMS architecture.

 

3.2.1 THE OVERALL ARCHITECTURE

 

Figure 3.2 depicts the overall METEOR2 WFMS architecture. The overall architecture of the METEOR2 WFMS consists of two major parts: the build-time components and the run-time components.

 

3.2.2 BUILD-TIME COMPONENTS

 

The build-time components are a collection of design-time tools that help users map the common organizational processes into our METEOR2 workflow process model, which can be further processed by a respective code generator to produce a reliable workflow application. The current build-time components contain the Workflow Designer, the Workflow Repository and the Run-time Code Generator.

 

3.2.3 RUN-TIME COMPONENTS

 

The run-time components of the METEOR2 WFMS contain the Enactment Systems with different Workflow Engines and a set of Monitoring and Administration Utilities.