Deelman, Ewa and Blythe, James and Gil, Yolanda and Kesselman, Carl and Mehta, Gaurang and Vahi, Karan and Blackburn, Kent and Lazzarini, Albert and Arbree, Adam and Cavanaugh, Richard and Koranda, Scott (2003) Mapping Abstract Complex Workflows onto Grid Environments. Journal of Grid Computing, 1 (1). pp. 25-39. ISSN 1570-7873. doi:10.1023/a:1024000426962. https://resolver.caltech.edu/CaltechAUTHORS:20200318-132250303
Full text is not posted in this repository. Consult Related URLs below.
Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20200318-132250303
Abstract
In this paper we address the problem of automatically generating job workflows for the Grid. These workflows describe the execution of a complex application built from individual application components. In our work we have developed two workflow generators: the first (the Concrete Workflow Generator CWG) maps an abstract workflow defined in terms of application-level components to the set of available Grid resources. The second generator (Abstract and Concrete Workflow Generator, ACWG) takes a wider perspective and not only performs the abstract to concrete mapping but also enables the construction of the abstract workflow based on the available components. This system operates in the application domain and chooses application components based on the application metadata attributes. We describe our current ACWG based on AI planning technologies and outline how these technologies can play a crucial role in developing complex application workflows in Grid environments. Although our work is preliminary, CWG has already been used to map high energy physics applications onto the Grid. In one particular experiment, a set of production runs lasted 7 days and resulted in the generation of 167,500 events by 678 jobs. Additionally, ACWG was used to map gravitational physics workflows, with hundreds of nodes onto the available resources, resulting in 975 tasks, 1365 data transfers and 975 output files produced.
| Item Type: | Article | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Related URLs: |
| |||||||||
| ORCID: |
| |||||||||
| Additional Information: | © 2003 Kluwer Academic Publishers. Issue Date: March 2003. We gratefully acknowledge many helpful and stimulating discussions on these topics with our colleagues Ian Foster, Michael Wilde, Yong Zhao and Jens Voeckler. The Chimera system mentioned in this paper has been jointly developed by the University of Southern California Information Sciences Institute, the University of Chicago and Argonne National Laboratory. We would like to thank the following LIGO scientists for their contribution to the development of the grid-enabled LIGO software: Stuart Anderson, Marsha Barnes, Philip Charlton, Phil Ehrens, Ed Maros, Greg Mendell, Mary Lei and Isaac Salzman. This research was supported in part by the National Science Foundation under grants ITR-0086044(GriPhyN) and EAR-0122464 (SCEC/ITR). LIGO Laboratory operates under NSF cooperative agreement PHY-0107417. This paper has been assigned LIGO document number LIGO-P030006-00-E. | |||||||||
| Group: | LIGO | |||||||||
| Funders: |
| |||||||||
| Subject Keywords: | complex applications, planning, reliability, workflow management | |||||||||
| Other Numbering System: |
| |||||||||
| Issue or Number: | 1 | |||||||||
| DOI: | 10.1023/a:1024000426962 | |||||||||
| Record Number: | CaltechAUTHORS:20200318-132250303 | |||||||||
| Persistent URL: | https://resolver.caltech.edu/CaltechAUTHORS:20200318-132250303 | |||||||||
| Official Citation: | Deelman, E., Blythe, J., Gil, Y. et al. Mapping Abstract Complex Workflows onto Grid Environments. Journal of Grid Computing 1, 25–39 (2003). https://doi.org/10.1023/A:1024000426962 | |||||||||
| Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. | |||||||||
| ID Code: | 101975 | |||||||||
| Collection: | CaltechAUTHORS | |||||||||
| Deposited By: | Tony Diaz | |||||||||
| Deposited On: | 18 Mar 2020 20:43 | |||||||||
| Last Modified: | 16 Nov 2021 18:07 |
Repository Staff Only: item control page
