A Caltech Library Service

Improving data quality monitoring via a partnership of technologies and resources between the CMS experiment at CERN and industry

Azzolini, Virginia and Andrews, Michael and Cerminara, Gianluca and Dev, Nabarun and Jessop, Colin and Marinelli, Nancy and Mudholkar, Tanmay and Pierini, Maurizio and Pol, Adrian and Vlimant, Jean-Roch (2019) Improving data quality monitoring via a partnership of technologies and resources between the CMS experiment at CERN and industry. EPJ Web of Conferences, 214 . Art. No. 01007. ISSN 2100-014X. doi:10.1051/epjconf/201921401007.

PDF - Published Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


The Compact Muon Solenoid (CMS) experiment dedicates significant effort to assess the quality of its data, online and offline. A real-time data quality monitoring system is in place to spot and diagnose problems as promptly as possible to avoid data loss. The a posteriori evaluation of processed data is designed to categorize it in terms of their usability for physics analysis. These activities produce data quality metadata. The data quality evaluation relies on a visual inspection of the monitoring features. This practice has a cost in term of human resources and is naturally subject to human arbitration. Potential limitations are linked to the ability to spot a problem within the overwhelming number of quantities to monitor, or to the lack of understanding of detector evolving conditions. In view of Run 3, CMS aims at integrating deep learning technique in the online workflow to promptly recognize and identify anomalies and improve data quality metadata precision. The CMS experiment engaged in a partnership with IBM with the objective to support, through automatization, the online operations and to generate benchmarking technological results. The research goals, agreed within the CERN Openlab framework, how they matured in a demonstration applic tion and how they are achieved, through a collaborative contribution of technologies and resources, are presented.

Item Type:Article
Related URLs:
URLURL TypeDescription
Dev, Nabarun0000-0003-2792-0491
Jessop, Colin0000-0002-6885-3611
Mudholkar, Tanmay0000-0002-9352-8140
Pierini, Maurizio0000-0003-1939-4268
Pol, Adrian0000-0002-9034-0230
Vlimant, Jean-Roch0000-0002-9705-101X
Additional Information:© The Authors, published by EDP Sciences, 2019. This is an Open Access article distributed under the terms of the Creative Commons Attribution License 4.0, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Published online 17 September 2019. Authors would like to thank the CMS collaboration for believing in this project, for dedicating some woman/manpower to it and for allowing the use of the data analyzed. We acknowledge the support of the CERN openlab project in creating the best conditions of communications and in providing an exclusive technical support infrastructure. The participation to the CHEP 2018 conference has been possible thanks to the funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement n. 772369), authors are grateful for the endorsement.
Funding AgencyGrant Number
European Research Council (ERC)772369
Record Number:CaltechAUTHORS:20201016-150841613
Persistent URL:
Official Citation:Improving data quality monitoring via a partnership of technologies and resources between the CMS experiment at CERN and industry Virginia Azzolin, Michael Andrews, Gianluca Cerminara, Nabarun Dev, Colin Jessop, Nancy Marinelli, Tanmay Mudholkar, Maurizio Pierini, Adrian Pol and Jean-Roch Vlimant EPJ Web Conf., 214 (2019) 01007 DOI:
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:106125
Deposited By: George Porter
Deposited On:16 Oct 2020 22:31
Last Modified:16 Nov 2021 18:50

Repository Staff Only: item control page