Reliable Distributed Data Management for Workflow-based Applications (PhD Thesis, finished)
Workflows provide an easy to use programming model for the construction of arbitrary complex data processing pipelines that are used in various application domains of business or science. When it comes to high performance workflow execution, the distribution (outscaling) of integral parts of the workflow across an environment of computational nodes is a key concept. However, such an approach brings along a set of challenges such as scalability and reliability. Above all, if the execution of workflows is to span mobile (resource limited) devices as well, such as in the case of emergency management scenarios, reliability becomes a crucial requirement. These challenges cannot be met by centralized solutions as they imply bottlenecks and single points of failure. Hence, the middleware responsible of managing the workflows has to be distributed, as well. Thereby, the efficient replication of workflow metadata within the distributed middleware, is a crucial prerequisite and should have only marginal effects on the middleware adaptivity and elasticity behaviour if the challenges of distributed workflow management are to be met. Our goal is to provide decentralized middleware for the reliable execution of distributed workflows. Moreover, the provided reliabilty should be scalable with the number of workflow instances as well. Thereby, the idea is to provide scalable services at the level of the node as well as the level of node clusters. Locally, modular component based ligtweight architectures are to be provided that are easily customizable and deployable to any kind of node environments. Globally, system services for reliable management of workflow metadata are to be provided. Such services should comprise scalable strong conistency semantics of the managed data as well as self-healing fault detection and recovery mechanisms.
Staff
Funding Agencies
Swiss National Science Foundation, Project SOSOA