Amin, A. and Bockelman, B. and Letts, J. and Levshina, T. and Martin, T. and Pi, H. and Sfiligoi, I. and Thomas, M. and Wüerthwein, F. (2011) High Throughput WAN Data Transfer with Hadoop-based Storage. Journal of Physics Conference Series, 331 . Art. No. 052016. ISSN 1742-6588 http://resolver.caltech.edu/CaltechAUTHORS:20120410-110725216
Full text not available from this repository.
Use this Persistent URL to link to this item: http://resolver.caltech.edu/CaltechAUTHORS:20120410-110725216
Abstract
Hadoop distributed file system (HDFS) is becoming more popular in recent years as a key building block of integrated grid storage solution in the field of scientific computing. Wide Area Network (WAN) data transfer is one of the important data operations for large high energy physics experiments to manage, share and process datasets of PetaBytes scale in a highly distributed grid computing environment. In this paper, we present the experience of high throughput WAN data transfer with HDFS-based Storage Element. Two protocols, GridFTP and fast data transfer (FDT), are used to characterize the network performance of WAN data transfer.
| Item Type: | Article |
|---|---|
| Additional Information: | © 2011 Institute of Physics. Published under licence by IOP Publishing Ltd. |
| Subject Keywords: | Accelerators, beams and electromagnetism; Electronics and devices; Nuclear physics; Instrumentation and measurement; Particle physics and field theory |
| Classification Code: | PACS: 84.40.Ua; 07.05.Kf; 07.05.Wr; 29.50.+v; 29.85.-c; 07.05.Bx; 29.40.Gx |
| Record Number: | CaltechAUTHORS:20120410-110725216 |
| Persistent URL: | http://resolver.caltech.edu/CaltechAUTHORS:20120410-110725216 |
| Related URLs: | |
| Official Citation: | High Throughput WAN Data Transfer with Hadoop-based Storage A Amin et al 2011 J. Phys.: Conf. Ser. 331 052016 |
| Usage Policy: | No commercial reproduction, distribution, display or performance rights in this work are provided. |
| ID Code: | 30048 |
| Collection: | CaltechAUTHORS |
| Deposited By: | Tony Diaz |
| Deposited On: | 10 Apr 2012 21:07 |
| Last Modified: | 10 Apr 2012 21:07 |
Repository Staff Only: item control page
