CaltechAUTHORS
  A Caltech Library Service

Mapping Abstract Complex Workflows onto Grid Environments

Deelman, Ewa and Blythe, James and Gil, Yolanda and Kesselman, Carl and Mehta, Gaurang and Vahi, Karan and Blackburn, Kent and Lazzarini, Albert and Arbree, Adam and Cavanaugh, Richard and Koranda, Scott (2003) Mapping Abstract Complex Workflows onto Grid Environments. Journal of Grid Computing, 1 (1). pp. 25-39. ISSN 1570-7873. https://resolver.caltech.edu/CaltechAUTHORS:20200318-132250303

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200318-132250303

Abstract

In this paper we address the problem of automatically generating job workflows for the Grid. These workflows describe the execution of a complex application built from individual application components. In our work we have developed two workflow generators: the first (the Concrete Workflow Generator CWG) maps an abstract workflow defined in terms of application-level components to the set of available Grid resources. The second generator (Abstract and Concrete Workflow Generator, ACWG) takes a wider perspective and not only performs the abstract to concrete mapping but also enables the construction of the abstract workflow based on the available components. This system operates in the application domain and chooses application components based on the application metadata attributes. We describe our current ACWG based on AI planning technologies and outline how these technologies can play a crucial role in developing complex application workflows in Grid environments. Although our work is preliminary, CWG has already been used to map high energy physics applications onto the Grid. In one particular experiment, a set of production runs lasted 7 days and resulted in the generation of 167,500 events by 678 jobs. Additionally, ACWG was used to map gravitational physics workflows, with hundreds of nodes onto the available resources, resulting in 975 tasks, 1365 data transfers and 975 output files produced.


Item Type:Article
Related URLs:
URLURL TypeDescription
https://doi.org/10.1023/a:1024000426962DOIArticle
https://rdcu.be/b26yQPublisherFree ReadCube access
ORCID:
AuthorORCID
Blackburn, Kent0000-0002-3838-2986
Additional Information:© 2003 Kluwer Academic Publishers. Issue Date: March 2003. We gratefully acknowledge many helpful and stimulating discussions on these topics with our colleagues Ian Foster, Michael Wilde, Yong Zhao and Jens Voeckler. The Chimera system mentioned in this paper has been jointly developed by the University of Southern California Information Sciences Institute, the University of Chicago and Argonne National Laboratory. We would like to thank the following LIGO scientists for their contribution to the development of the grid-enabled LIGO software: Stuart Anderson, Marsha Barnes, Philip Charlton, Phil Ehrens, Ed Maros, Greg Mendell, Mary Lei and Isaac Salzman. This research was supported in part by the National Science Foundation under grants ITR-0086044(GriPhyN) and EAR-0122464 (SCEC/ITR). LIGO Laboratory operates under NSF cooperative agreement PHY-0107417. This paper has been assigned LIGO document number LIGO-P030006-00-E.
Group:LIGO
Funders:
Funding AgencyGrant Number
NSFITR-0086044
NSFEAR-0122464
NSFPHY-0107417
Subject Keywords:complex applications, planning, reliability, workflow management
Other Numbering System:
Other Numbering System NameOther Numbering System ID
LIGO DocumentP030006-00-E
Issue or Number:1
Record Number:CaltechAUTHORS:20200318-132250303
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20200318-132250303
Official Citation:Deelman, E., Blythe, J., Gil, Y. et al. Mapping Abstract Complex Workflows onto Grid Environments. Journal of Grid Computing 1, 25–39 (2003). https://doi.org/10.1023/A:1024000426962
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:101975
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:18 Mar 2020 20:43
Last Modified:18 Mar 2020 20:43

Repository Staff Only: item control page