Improving cosmological reach of a gravitational wave observatory using Deep Loop Shaping

Creators: Buchli, Jonas¹; Tracey, Brendan¹; Andric, Tomislav^{2, 3}; Wipf, Christopher⁴; Chiu, Yu Him Justin¹; Lochbrunner, Matthias¹; Donner, Craig¹; Adhikari, Rana X.⁴; Harms, Jan^{2, 3}; Barr, Iain¹; Hafner, Roland¹; Huber, Andrea¹; Abdolmaleki, Abbas¹; Beattie, Charlie¹; Betzwieser, Joseph⁴; Cabi, Serkan¹; Degrave, Jonas¹; Dong, Yuzhu¹; Fritz, Leslie¹; Gupta, Anchal⁴; Groth, Oliver¹; Huang, Sandy¹; Norman, Tamara¹; Openshaw, Hannah¹; Rollins, Jameson⁴; Thornton, Greg¹; van den Driessche, George¹; Wulfmeier, Markus¹; Kohli, Pushmeet¹; Riedmiller, Martin¹; The LIGO Instrument Team; Abbott, R.⁴; Abouelfettouh, I.; Adhikari, R. X.⁴; Ananyeva, A.⁴; Appert, S.; Apple, S. K.; Arai, K.⁴; Aritomi, N.; Aston, S. M.; Ball, M.; Ballmer, S. W.; Barker, D.; Barsotti, L.; Berger, B. K.; Betzwieser, J.; Bhattacharjee, D.; Billingsley, G.⁴; Biscans, S.; Blair, C. D.; Bode, N.; Bonilla, E.; Bossilkov, V.; Branch, A.; Brooks, A. F.⁴; Brown, D. D.; Bryant, J.; Cahillane, C.; Cao, H.; Capote, E.; Clara, F.; Collins, J.; Compton, C. M.; Cottingham, R.; Coyne, D. C.⁴; Crouch, R.; Csizmazia, J.; Cumming, A.; Dartez, L. P.; Davis, D.⁴; Demos, N.; Dohmen, E.; Driggers, J. C.; Dwyer, S. E.; Effer, A.; Ejlli, A.; Etzel, T.⁴; Evans, M.; Feicht, J.⁴; Frey, R.; Frischhertz, W.; Fritschel, P.; Frolov, V. V.; Fuentes-Garcia, M.⁴; Fulda, P.; Fyffe, M.; Ganapathy, D.; Gateley, B.; Gayer, T.; Giaime, J. A.; Giardina, K. D.; Glanzer, J.⁴; Goetz, E.; Goetz, R.; Goodwin-Jones, A. W.⁴; Gras, S.; Gray, C.; Griffith, D.⁴; Grote, H.; Guidry, T.; Gurs, J.; Hall, E. D.; Hanks, J.; Hanson, J.; Heintze, M. C.; Helmling-Cornell, A. F.; Holland, N. A.; Hoyland, D.; Huang, H. Y.; Inoue, Y.; James, A. L.⁴; Jennings, A.; Jia, W.; Jones, D. H.; Kabagoz, H. B.; Karat, S.⁴; Karki, S.; Kasprzack, M.⁴; Kawabe, K.; Kijbunchoo, N.; King, P. J.; Kissel, J. S.; Komori, K.; Kontos, A.; Kumar, Rahul; Kuns, K.; Landry, M.; Lantz, B.; Laxen, M.; Lee, K.; Lesovsky, M.⁴; Villarreal, F. Llamas; Lormand, M.; Loughlin, H. A.; Macas, R.; MacInnis, M.; Makarem, C. N.⁴; Mannix, B.; Mansell, G. L.; Martin, R. M.; Mason, K.; Matichard, F.; Mavalvala, N.; Maxwell, N.; McCarrol, G.; McCarthy, R.; Mc-Clelland, D. E.; McCormick, S.; McRae, T.; Mera, F.; Merilh, E. L.; Meylahn, F.; Mittleman, R.; Moraru, D.; Moreno, G.; Mullavey, A.; Nakano, M.⁴; Nelson, T. J. N.; Neunzert, A.; Notte, J.; Oberling, J.; O'Hanlon, T.; Osthelder, C.⁴; Ottaway, D. J.; Overmier, H.; Parker, W.; Patane, O.; Pele, A.⁴; Pham, H.; Pirello, M.; Pullin, J.; Quetschke, V.; Ramire, K. E.; Ransom, K.; Reyes, J.; Richardson, J. W.; Robinson, M.; Rollins, J. G.⁴; Romel, C. L.; Romie, J. H.; Ross, M. P.; Ryan, K.; Sadecki, T.; Sanchez, A.; Sanchez, E. J.⁴; Sanchez, L. E.⁴; Savage, R. L.; Schaetzl, D.⁴; Schiworski, M. G.; Schnabel, R.; Schofield, R. M. S.; Schwartz, E.; Sellers, D.; Shaffer, T.; Short, R. W.; Sigg, D.; Slagmolen, B. J. J.; Soike, C.; Soni, S.; Srivastava, V.; Sun, L.; Tanner, D. B.; Thomas, M.; Thomas, P.; Thorne, K. A.; Todd, M. R.; Torrie, C. I.⁴; Traylor, G.; Ubhi, A. S.; Vajente, G.⁴; Vanosky, J.; Vecchio, A.; Veitch, P. J.; Vibhute, A. M.; von Reis, E. R. G.; Warner, J.; Weaver, B.; Weiss, R.; Whittle, C.⁴; Willke, B.; Wipf, C. C.⁴; Wright, J. L.; Xu, V. A.; Yamamoto, H.⁴; Zhang, L.⁴; Zucker, M. E.⁴

1. Google DeepMind
2. Gran Sasso Science Institute
3. Gran Sasso National Laboratory
4. California Institute of Technology

Abstract

Improved low-frequency sensitivity of gravitational wave observatories would unlock study of intermediate-mass black hole mergers and binary black hole eccentricity and provide early warnings for multimessenger observations of binary neutron star mergers. Today's mirror stabilization control injects harmful noise, constituting a major obstacle to sensitivity improvements. We eliminated this noise through Deep Loop Shaping, a reinforcement learning method using frequency domain rewards. We proved our methodology on the LIGO Livingston Observatory (LLO). Our controller reduced control noise in the 10- to 30-hertz band by over 30x and up to 100x in subbands, surpassing the design goal motivated by the quantum limit. These results highlight the potential of Deep Loop Shaping to improve current and future gravitational wave observatories and, more broadly, instrumentation and control systems.

Copyright and License

© 2025 the authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original US government works. https://www.science.org/about/science-licenses-journal-article-reuse

Acknowledgement

We thank J. Dean for strategic help and inspiration at the start of the project.

Funding

The authors gratefully acknowledge the support of the US National Science Foundation (NSF) for the construction and operation of the LIGO Laboratory and Advanced LIGO as well as the Science and Technology Facilities Council of the UK and the Max Planck Society for support of the construction of Advanced LIGO. Additional support for Advanced LIGO was provided by the Australian Research Council. LIGO was constructed by the California Institute of Technology and Massachusetts Institute of Technology with funding from the NSF and operates under cooperative agreement no. PHY-18671764464. Advanced LIGO was built under grant no. PHY-18680823459.

Contributions

R.X.A., J.Bu., S.C., J.H., M.R., and B.T. conceived the project. R.X.A., J.Bu., C.D., A.H., J.H., and B.T. led the project. T.A., R.X.A., I.B., J.Bu., J.Be., C.D., G.v.d.D., J.D., A.G., J.H., M.L., B.T., G.T., and C.W. developed the physics simulations. T.A., I.B., J.Bu., Y.H.J.C., C.D., J.D., M.L., B.T., and C.W. integrated the physics simulations with the learning framework. A.A., J.Bu., R.H., S.H., M.L., M.W., and B.T. developed the learning framework and performed learning experiments. C.D., T.N., J.R., and C.W. developed the real-time neural network interface. R.X.A., J.Be., J.Bu., C.D., A.G., B.T., and C.W. integrated the real-time neural network with the control system and ran experiments on LLO and the California Institute of Technology 40m prototype. C.B., J.Bu., Y.H.J.C., C.D., Y.D., O.G., M.L., and C.W. developed data curation tools. R.X.A., I.B., J.Bu., Y.H.J.C., B.T., and C.W. developed and ran the data analysis. L.F., P.K., H.O., and M.R. consulted for the project. T.A., R.X.A., J.Bu., J.H., B.T., and C.W. wrote the manuscript. The LIGO Instrument Team maintains and runs the LIGO Observatory.

Data Availability

The learning algorithm used in the actor-critic RL method is MPO (18), a reference implementation of which is available under an open-source license (22). Additionally, the software libraries launchpad (23), dm env (24), Jax/Haiku (25), and reverb (26) were used, which are also open source. Simulations were implemented in Lightsaber (27) and advLigoRTS (19). The identified LLO model and experimental data are available at (28).

Supplemental Material

Supplementary Text; Figs. S1 to S16; Tables S1 to S3; References (29–69) (PDF)

Files

	All versions	This version
Views	0	0
Downloads	0	0
Data volume	0 Bytes	0 Bytes