Processing of GASKAP-Hi pilot survey data using a commercial supercomputer

Creators: Kemp, I. P.^{1, 2}; Pingel, N. M.³; Worth, R.⁴; Wake, J.⁴; Mitchell, D. A.⁵; Midgely, S. D.⁶; Tingay, S. J.¹; Dempsey, J.^{7, 8}; Dénes, H.⁹; Dickey, J. M.¹⁰; Gibson, S. J.¹¹; Jameson, K. E.¹²; Lynn, C.⁷; Ma, Y. K.⁷; Marchal, A.⁷; McClure-Griffiths, N. M.⁷; Stanimirović, S.³; van Loon, J. Th.¹³

1. International Centre for Radio Astronomy Research
2. CSIRO Space and Astronomy, 26 Dick Perry Avenue, Kensington, 6151, WA, Australia
3. University of Wisconsin–Madison
4. DUG Technology, 76 Kings Park Road, West Perth, 6005, WA, Australia
5. CSIRO Space and Astronomy, Cnr Vimiera Pembroke Roads, Marsfield, 2122, NSW, Australia
6. Defence Science and Technology Group
7. Australian National University
8. Commonwealth Scientific and Industrial Research Organisation
9. School of Physical Sciences and Nanotechnology, Yachay Tech University, Hacienda San José S/N, 100119, Urcuquí, Ecuador
10. University of Tasmania
11. Western Kentucky University
12. California Institute of Technology
13. Keele University

Abstract

Modern radio telescopes generate large amounts of data, with the next generation Very Large Array (ngVLA) and the Square Kilometre Array (SKA) expected to feed up to 292 GB of visibilities per second to the science data processor (SDP). However, the continued exponential growth in the power of the world's largest supercomputers suggests that for the foreseeable future there will be sufficient capacity available to provide for astronomers' needs in processing 'science ready' products from the new generation of telescopes, with commercial platforms becoming an option for overflow capacity. The purpose of the current work is to trial the use of commercial high performance computing (HPC) for a large scale processing task in astronomy, in this case processing data from the GASKAP-Hi pilot surveys. We delineate a four-step process which can be followed by other researchers wishing to port an existing workflow from a public facility to a commercial provider. We used the process to provide reference images for an ongoing upgrade to ASKAPSoft (the ASKAP SDP software), and to provide science images for the GASKAP collaboration, using the joint deconvolution capability of WSClean. We document the approach to optimising the pipeline to minimise cost and elapsed time at the commercial provider, and give a resource estimate for processing future full survey data. Finally we document advantages, disadvantages, and lessons learned from the project, which will aid other researchers aiming to use commercial supercomputing for radio astronomy imaging. We found the key advantage to be immediate access and high availability, and the main disadvantage to be the need for improved HPC knowledge to take best advantage of the facility.

Copyright and License

Acknowledgement

The authors thank CSIRO, Australia, DUG Technology, and Curtin University, Australia for supporting this work financially and in kind.

This scientific work uses data obtained from Inyarrimanha Ilgari Bundara/the Murchison Radio-astronomy Observatory. We acknowledge the Wajarri Yamaji People as the Traditional Owners and native title holders of the Observatory site. CSIRO’s ASKAP radio telescope is part of the Australia Telescope National Facility (https://ror.org/05qajvd42). Operation of ASKAP is funded by the Australian Government with support from the National Collaborative Research Infrastructure Strategy. ASKAP uses the resources of the Pawsey Supercomputing Research Centre. Establishment of ASKAP, Inyarrimanha Ilgari Bundara, the CSIRO Murchison Radio-astronomy Observatory and the Pawsey Supercomputing Research Centre are initiatives of the Australian Government, with support from the Government of Western Australia, Australia and the Science and Industry Endowment Fund, Australia . This paper includes archived data obtained through the CSIRO ASKAP Science Data Archive, CASDA (http://data.csiro.au).

Contributions

I.P. Kemp: Writing – review & editing, Writing – original draft, Visualization, Validation, Software, Project administration, Methodology, Investigation. N.M. Pingel: Writing – review & editing, Supervision, Software, Methodology. R. Worth: Writing – review & editing, Software, Investigation. J. Wake: Writing – review & editing, Software, Methodology. D.A. Mitchell: Writing – review & editing, Visualization, Supervision, Software, Resources, Methodology. S.D. Midgely: Writing – review & editing, Supervision, Resources. S.J. Tingay: Writing – review & editing, Supervision, Resources, Conceptualization. J. Dempsey: Conceptualization. H. Dénes: Writing – review & editing, Validation, Investigation. J.M. Dickey: Writing – review & editing, Validation, Investigation. S.J. Gibson: Validation. K.E. Jameson: Project administration. C. Lynn: Validation. Y.K. Ma: Writing – review & editing, Validation. A. Marchal: Writing – review & editing, Validation. N.M. McClure-Griffiths: Writing – review & editing, Supervision, Resources, Funding acquisition, Conceptualization. S. Stanimirović: Writing – review & editing, Validation. J. Th. van Loon: Writing – review & editing, Validation.

Conflict of Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Ian Kemp reports financial support was provided by DUG Technology. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Data Availability

Data is under embargo and available to members of the GASKAP collaboration.

Files

1-s2.0-S2213133724001161-main.pdf

	All versions	This version
Views	0	0
Downloads	0	0
Data volume	0 Bytes	0 Bytes