CaltechAUTHORS
  A Caltech Library Service

Implementation of collective I/O in the Intel Paragon parallel file system: initial experiences

Bordawekar, Rajesh (1997) Implementation of collective I/O in the Intel Paragon parallel file system: initial experiences. In: ICS '97 Proceedings of the 11th international conference on Supercomputing. ACM , New York, NY, pp. 20-27. ISBN 0-89791-902-5. https://resolver.caltech.edu/CaltechAUTHORS:20160810-111243309

Full text is not posted in this repository. Consult Related URLs below.

Use this Persistent URL to link to this item: https://resolver.caltech.edu/CaltechAUTHORS:20160810-111243309

Abstract

A majority of parallel applications achieve parallelism by partitioning data over multiple processors. Accessing distributed data structures such as arrays from files often requires each processor to make a large number of small noncontiguous data requests. This problem can be addressed by replacing small non-contiguous requests by large collective requests. This approach, known as collective I/O, has been found to work extremely well in practice. This paper describes implementation and evaluation of a collective I/O prototype in the Parallel File System (PFS) of the Intel Paragon Operating System (Paragon OS). We evaluate the collective I/O performance using its comparison with the PFS MRECORD and KUNIX I/O modes. It is observed that collective I/O provides significant performance improvement over accesses in the H-UNIX mode. However, in many cases, various implementation overheads cause collective I/O to provide lower performance than the H-RECORD I/O mode.


Item Type:Book Section
Related URLs:
URLURL TypeDescription
http://dx.doi.org/10.1145/263580.263586DOIPaper
http://dl.acm.org/citation.cfm?doid=263580.263586PublisherPaper
Additional Information:© 1997 ACM. We would like to thank the CACR staff for both computing and administrative support. We would like to thank David Kotz for reviewing the technical report and for answering numerous questions. We would like to thank Tina Pauna for reviewing earlier manuscripts. Finally, we would like to thank anonymous reviewers for their comments. This work was supported in part by the Scalable I/O Initiative, contract number DABT63-94-C-0049 from the Defense Advanced Research Projects Agency(DARPA), administered by the US Army at Fort Huachuca, and Caltech’s Center for Advanced Computing Research. Access to Caltech’s Intel Paragon was provided by CACR.
Funders:
Funding AgencyGrant Number
Defense Advanced Research Projects Agency (DARPA)DABT63-94-C-0049
Caltech’s Center for Advanced Computing ResearchUNSPECIFIED
Record Number:CaltechAUTHORS:20160810-111243309
Persistent URL:https://resolver.caltech.edu/CaltechAUTHORS:20160810-111243309
Official Citation:Rajesh Bordawekar. 1997. Implementation of collective I/O in the Intel Paragon parallel file system: initial experiences. In Proceedings of the 11th international conference on Supercomputing (ICS '97). ACM, New York, NY, USA, 20-27. DOI=http://dx.doi.org/10.1145/263580.263586
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:69539
Collection:CaltechAUTHORS
Deposited By: Kristin Buxton
Deposited On:10 Aug 2016 18:19
Last Modified:03 Oct 2019 10:23

Repository Staff Only: item control page