CaltechAUTHORS
  A Caltech Library Service

An Evaluation of the Cost and Performance of Scientific Workflows on Amazon EC2

Juve, Gideon and Deelman, Ewa and Berriman, G. Bruce and Berman, Benjamin P. and Maechling, Philip (2012) An Evaluation of the Cost and Performance of Scientific Workflows on Amazon EC2. Journal of Grid Computing, 10 (1). pp. 5-21. ISSN 1570-7873. doi:10.1007/s10723-012-9207-6. https://resolver.caltech.edu/CaltechAUTHORS:20120511-131415294

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20120511-131415294

Abstract

Workflows are used to orchestrate data-intensive applications in many different scientific domains. Workflow applications typically communicate data between processing steps using intermediate files. When tasks are distributed, these files are either transferred from one computational node to another, or accessed through a shared storage system. As a result, the efficient management of data is a key factor in achieving good performance for workflow applications in distributed environments. In this paper we investigate some of the ways in which data can be managed for workflows in the cloud. We ran experiments using three typical workflow applications on Amazon’s EC2 cloud computing platform. We discuss the various storage and file systems we used, describe the issues and problems we encountered deploying them on EC2, and analyze the resulting performance and cost of the workflows.


Item Type:Article
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1007/s10723-012-9207-6DOIArticle
https://rdcu.be/bVvQzPublisherFree ReadCube access
ORCID:
AuthorORCID
Berriman, G. Bruce0000-0001-8388-534X
Additional Information:© 2012 Springer Science+Business Media B.V. Received: 15 August 2011; Accepted: 1 March 2012; Published online: 18 March 2012. This work was supported by the National Science Foundation under the IntelData (IIS-0905032) and Pegasus (OCI-0722019) grants. This research made use of Montage, funded by the National Aeronautics and Space Administration’s Earth Science Technology Office, Computation Technologies Project, under Cooperative Agreement Number NCC5-626 between NASA and the California Institute of Technology.
Group:Infrared Processing and Analysis Center (IPAC)
Funders:
Funding AgencyGrant Number
NSFIIS-0905032
NSFOCI-0722019
NASANCC5-626
Subject Keywords:Cloud computing; Scientific workflows
Issue or Number:1
DOI:10.1007/s10723-012-9207-6
Record Number:CaltechAUTHORS:20120511-131415294
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20120511-131415294
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:31438
Collection:CaltechAUTHORS
Deposited By: Tony Diaz
Deposited On:14 May 2012 15:52
Last Modified:09 Nov 2021 19:53

Repository Staff Only: item control page