A Caltech Library Service

Results of the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC)

Hložek, R. and Ponder, K. A. and Malz, A. I. and Dai, M. and Narayan, G. and Ishida, E. E. O. and Allam, T., Jr. and Bahmanyar, A. and Biswas, R. and Galbany, L. and Jha, S. W. and Jones, D. O. and Kessler, R. and Lochner, M. and Mahabal, A. A. and Mandel, K. S. and Martínez-Galarza, J. R. and McEwen, J. D. and Muthukrishna, D. and Peiris, H. V. and Peters, C. M. and Setzer, C. N. (2020) Results of the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC). . (Unpublished)

[img] PDF - Submitted Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Next-generation surveys like the Legacy Survey of Space and Time (LSST) on the Vera C. Rubin Observatory will generate orders of magnitude more discoveries of transients and variable stars than previous surveys. To prepare for this data deluge, we developed the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC), a competition which aimed to catalyze the development of robust classifiers under LSST-like conditions of a non-representative training set for a large photometric test set of imbalanced classes. Over 1,000 teams participated in PLAsTiCC, which was hosted in the Kaggle data science competition platform between Sep 28, 2018 and Dec 17, 2018, ultimately identifying three winners in February 2019. Participants produced classifiers employing a diverse set of machine learning techniques including hybrid combinations and ensemble averages of a range of approaches, among them boosted decision trees, neural networks, and multi-layer perceptrons. The strong performance of the top three classifiers on Type Ia supernovae and kilonovae represent a major improvement over the current state-of-the-art within astronomy. This paper summarizes the most promising methods and evaluates their results in detail, highlighting future directions both for classifier development and simulation needs for a next generation PLAsTiCC data set.

Item Type:Report or Paper (Discussion Paper)
Related URLs:
URLURL TypeDescription Paper
Hložek, R.0000-0002-0965-7864
Ponder, K. A.0000-0002-8207-3304
Malz, A. I.0000-0002-8676-1622
Narayan, G.0000-0001-6022-0484
Ishida, E. E. O.0000-0002-0406-076X
Allam, T., Jr.0000-0002-8117-9254
Biswas, R.0000-0002-5741-7195
Galbany, L.0000-0002-1296-6887
Jha, S. W.0000-0001-8738-6011
Jones, D. O.0000-0002-6230-0151
Kessler, R.0000-0003-3221-0419
Lochner, M.0000-0003-2221-8281
Mahabal, A. A.0000-0003-2242-0244
Mandel, K. S.0000-0001-9846-4417
Martínez-Galarza, J. R.0000-0002-5069-0324
McEwen, J. D.0000-0002-5852-8890
Muthukrishna, D.0000-0002-5788-9280
Peiris, H. V.0000-0002-2519-584X
Peters, C. M.0000-0003-4639-0211
Setzer, C. N.0000-0002-7439-2735
Additional Information:Attribution 4.0 International (CC BY 4.0) This paper has undergone internal review in the LSST Dark Energy Science Collaboration. Author contributions are listed below. R. Hložek: data curation, formal analysis, funding acquisition, investigation, project administration, software, supervision, validation, visualization, writing - editing, lead writing - original draft K. Ponder: software development and implementation for validation, testing, visualization and analysis of results, writing - editing, writing - original draft A.I. Malz: conceptualization, formal analysis, investigation, methodology, visualization, writing - editing, writing - original draft M. Dai: software development and implementation for validation, testing, analysis of results, writing - editing G. Narayan: simulation, software, example kit, validation, visualization, writing - editing, writing - original draft E.E.O. Ishida: software development and implementation for validation, testing, Kaggle liason, analysis of results, writing - editing T. Allam Jr: investigation, development of software for simulation validation A. Bahmanyar: investigation, implementation of metric methodology and software R. Biswas: conceptualization, methodology, software for validation and testing, writing - editing L. Galbany: implementation of validation software, writing - editing S.W. Jha: conceptualization, simulation validation D. Jones: conceptualization, simulation validation, writing - editing R. Kessler: simulation development and validation testing, data generation, writing - editing M. Lochner: conceptualization, simulation validation, writing - editing A.A. Mahabal: conceptualization, simulation validation K.S. Mandel: conceptualization, simulation validation, writing - editing J.R. Martínez-Galarza: conceptualization, simulation validation, writing - editing J.D. McEwen: conceptualization, simulation validation D. Muthukrishna: conceptualization, simulation validation, results analysis and testing, writing - editing H.V. Peiris: conceptualization, supervision, funding acquisition, writing - editing C.M. Peters: conceptualization, simulation validation C.N. Setzer: conceptualization, simulation validation The Photometric LSST Astronomical Time Series Classification Challenge data set generation relied on numerous members of the astronomical community to provide models of astronomical transients and variables. Photometric LSST Astronomical Time Series Classification Challenge is made possible through an intercollaboration agreement between LSST's Dark Energy Science Collaboration and the Transient and Variable Stars Science Collaboration. These models are described in Kessler et al. (2019a). We are grateful to Federica Bianco and staff at NYU for hosting the PLAsTiCC team meeting. The awards for PLAsTiCC contest were generously supported by Kaggle. We thank staff� at Kaggle, particularly Elizabeth Park, Maggie Demkin and Sohier Dane, for their help making the contest a success. We thank Federica Bianco, Francisco F�örster Burón, Chad Schafer and Melissa Graham for providing internal review of this document for the DESC and TVS collaborations, and thank Seth Digel for comments on this draft. This work was supported by an LSST Corporation Enabling Science grant. The financial assistance of the National Research Foundation (NRF) towards this research is hereby acknowledged. Opinions expressed and conclusions arrived at, are those of the authors and are not necessarily to be attributed to the NRF. UK authors acknowledge funding from the Science and Technology Funding Council. Canadian co-authors acknowledge support from the Natural Sciences and Engineering Research Council of Canada. The Dunlap Institute is funded through an endowment established by the David Dunlap family and the University of Toronto. The authors at the University of Toronto acknowledge that the land on which the University of Toronto is built is the traditional territory of the Haudenosaunee, and most recently, the territory of the Mississaugas of the New Credit First Nation. They are grateful to have the opportunity to work in the community, on this territory. R. H. acknowledges support from CIFAR, and the Azrieli and Alfred. P. Sloan foundations. K. A .P. acknowledges support from the Berkeley Center for Cosmological Physics; the Director, Office of Science, Office of High Energy Physics of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231; and U.S. Department of Energy Office of Science under Contract No.DE-AC02-76SF00515. E. E. O. I. acknowledges support from CNRS 2017 MOMENTUM grant. A. I. M acknowledges support from the Max Planck Society and the Alexander von Humboldt Foundation in the framework of the Max Planck-Humboldt Research Award endowed by the Federal Ministry of Education and Research. L. G. was funded by the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 839090. D. O. J acknowledges support provided by NASA Hubble Fellowship grant HST-HF2-51462.001 awarded by the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc., for NASA, under contract NAS5-26555. D.O.J. also acknowledges support from the Gordon & Betty Moore Foundation. This work has been partially supported by the Spanish grant PGC2018-095317-B-C21 within the European Funds for Regional Development (FEDER). C. N. S., R. B. and H. V. P. were partially supported by the research environment grant "Gravitational Radiation and Electromagnetic Astrophysical Transients (GREAT)" funded by the Swedish Research Council (VR) under Dnr 2016-06012. RB and HVP were additionally supported by the research project grant "Understanding the Dynamic Universe" funded by the Knut and Alice Wallenberg Foundation under Dnr KAW 2018.0067. M. D. acknowledges support from the Horizon Fellowship at the Johns Hopkins University. At Rutgers University (M. D., S. W. J.), this research was supported by NSF award AST-1615455 and DOE award DE-SC0011636. The DESC acknowledges ongoing support from the Institut National de Physique Nucléaire et de Physique des Particules in France; the Science & Technology Facilities Council in the United Kingdom; and the Department of Energy, the National Science Foundation, and the LSST Corporation in the United States. DESC uses resources of the IN2P3 Computing Center (CC-IN2P3-Lyon/Villeurbanne - France) funded by the Centre National de la Recherche Scienti�que; the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231; STFC DiRAC HPC Facilities, funded by UK BIS National E-infrastructure capital grants; and the UK particle physics grid, supported by the GridPP Collaboration. This work was performed in part under DOE Contract DE-AC02-76SF00515.
Funding AgencyGrant Number
National Research Foundation (South Africa)UNSPECIFIED
Science and Technology Facilities Council (STFC)UNSPECIFIED
Natural Sciences and Engineering Research Council of Canada (NSERC)UNSPECIFIED
David Dunlap FamilyUNSPECIFIED
University of TorontoUNSPECIFIED
Canadian Institute for Advanced Research (CIFAR)UNSPECIFIED
Azrieli FoundationUNSPECIFIED
Alfred P. Sloan FoundationUNSPECIFIED
Berkeley Center for Cosmological PhysicsUNSPECIFIED
Department of Energy (DOE)DE-AC02-05CH11231
Department of Energy (DOE)DE-AC02-76SF00515
Centre National de la Recherche Scientifique (CNRS)UNSPECIFIED
Max Planck SocietyUNSPECIFIED
Alexander von Humboldt FoundationUNSPECIFIED
Bundesministerium für Bildung und Forschung (BMBF)UNSPECIFIED
Marie Curie Fellowship839090
NASA Hubble FellowshipHST-HF2-51462.001
Gordon and Betty Moore FoundationUNSPECIFIED
Fondo Europeo de Desarrollo Regional (FEDER)PGC2018-095317-B-C21
Swedish Research Council2016-06012
Knut and Alice Wallenberg FoundationKAW 2018.0067
Johns Hopkins UniversityUNSPECIFIED
Department of Energy (DOE)DE-SC0011636
Institut National de Physique Nucléaire et de Physique des Particules (IN2P3)UNSPECIFIED
GridPP CollaborationUNSPECIFIED
Record Number:CaltechAUTHORS:20210205-093243756
Persistent URL:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:107935
Deposited By: George Porter
Deposited On:05 Feb 2021 22:53
Last Modified:05 Feb 2021 22:53

Repository Staff Only: item control page