- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review Applied

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 4.0 International license. This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

Export citation

Export citation

Choose format for download:

Download Citation
  • Open Access
  • Access by Xinjiang University

Enhanced qubit readout via reinforcement learning

Aniket Chatterjee*

Jonathan Schwinger

Yvonne Y. Gao†

  • *Contact author: aniket.chatterjee@chch.ox.ac.uk
  • †Contact author: yvonne.gao@nus.edu.sg
PDF

Phys. Rev. Applied 23, 054057 – Published 22 May, 2025

DOI: https://doi.org/10.1103/PhysRevApplied.23.054057

Abstract

Measurement is an essential component of robust and practical quantum computation. For superconducting qubits, the measurement process involves the effective manipulation of the joint qubit-resonator dynamics, and it should ideally provide the highest quality for qubit state discrimination with the shortest readout pulse and resonator reset time. Here, we harness model-free reinforcement learning (RL), together with a tailored training environment, to achieve this multifaceted optimization task. Using the IBM quantum device, we demonstrate that the pulse obtained by the RL agent not only successfully achieves state-of-the-art performance, with an assignment error of (4.6±0.4)×10−3, but also executes the readout and the subsequent resonator reset almost three times faster than the system’s default process. Furthermore, the learned waveforms are robust against realistic parameter drifts and follow an intuitive form, making them readily implementable on existing hardware with little computational overhead. Our results provide an effective readout strategy to boost the performance of superconducting quantum processors and demonstrate the value of RL in providing optimal and practical solutions for complex quantum information processing tasks.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (49)

  1. IBM Quantum, https://quantum.ibm.com/ (2021).
  2. V. V. Sivak, A. Eickbusch, B. Royer, S. Singh, I. Tsioutsios, S. Ganjam, A. Miano, B. L. Brock, A. Z. Ding, and L. Frunzio et al., Real-time quantum error correction beyond break-even, Nature 616, 50 (2023).
  3. N. Sundaresan, T. J. Yoder, Y. Kim, M. Li, E. H. Chen, G. Harper, T. Thorbeck, A. W. Cross, A. D. Córcoles, and M. Takita, Demonstrating multi-round subsystem quantum error correction using matching and maximum likelihood decoders, Nat. Commun. 14, 2852 (2023).
  4. A. D. Córcoles, M. Takita, K. Inoue, S. Lekuch, Z. K. Minev, J. M. Chow, and J. M. Gambetta, Exploiting dynamic quantum circuits in a quantum algorithm with superconducting qubits, Phys. Rev. Lett. 127, 100501 (2021).
  5. A. P. M. Place, L. V. H. Rodgers, P. Mundada, B. M. Smitham, M. Fitzpatrick, Z. Leng, A. Premkumar, J. Bryon, A. Vrajitoarea, and S. Sussman et al., New material platform for superconducting transmon qubits with coherence times exceeding 0.3 milliseconds, Nat. Commun. 12, 1779 (2021).
  6. C. Wang, X. Li, H. Xu, Z. Li, J. Wang, Z. Yang, Z. Mi, X. Liang, T. Su, and C. Yang et al., Towards practical quantum computers: Transmon qubit with a lifetime approaching 0.5 milliseconds, Npj Quantum Inf. 8, 3 (2022).
  7. J. Aumentado, Superconducting parametric amplifiers: The state of the art in Josephson parametric amplifiers, IEEE Microw. Mag. 21, 45 (2020).
  8. R. Vijay, D. H. Slichter, and I. Siddiqi, Observation of quantum jumps in a superconducting artificial atom, Phys. Rev. Lett. 106, 110502 (2011).
  9. B. Ho Eom, P. K. Day, H. G. LeDuc, J. Zmuidzinas, and A wideband, Low-noise superconducting amplifier with high dynamic range, Nat. Phys. 8, 623 (2012).
  10. M. D. Reed, B. R. Johnson, A. A. Houck, L. DiCarlo, J. M. Chow, D. I. Schuster, L. Frunzio, and R. J. Schoelkopf, Fast reset and suppressing spontaneous emission of a superconducting qubit, Appl. Phys. Lett. 96, 203110 (2010).
  11. T. Walter, P. Kurpiers, S. Gasparinetti, P. Magnard, A. Potočnik, Y. Salathé, M. Pechal, M. Mondal, M. Oppliger, C. Eichler, and A. Wallraff, Rapid high-fidelity single-shot dispersive readout of superconducting qubits, Phys. Rev. Appl. 7, 054020 (2017).
  12. Y. Sunada, S. Kono, J. Ilves, S. Tamate, T. Sugiyama, Y. Tabuchi, and Y. Nakamura, Fast readout and reset of a superconducting qubit coupled to a resonator with an intrinsic Purcell filter, Phys. Rev. Appl. 17, 044016 (2022).
  13. R. Dassonneville, T. Ramos, V. Milchakov, L. Planat, E. Dumur, F. Foroughi, J. Puertas, S. Leger, K. Bharadwaj, and J. Delaforce et al., Fast high-fidelity quantum nondemolition qubit readout via a nonperturbative cross-Kerr coupling, Phys. Rev. X 10, 011045 (2020).
  14. F. Swiadek, R. Shillito, P. Magnard, A. Remm, C. Hellings, N. Lacroix, Q. Ficheux, D. C. Zanuz, G. J. Norris, and A. Blais et al., Enhancing dispersive readout of superconducting qubits through dynamic control of the dispersive shift: Experiment and theory, PRX Quantum 5, 040326 (2024).
  15. J. Gambetta, W. A. Braff, A. Wallraff, S. M. Girvin, and R. J. Schoelkopf, Protocols for optimal readout of qubits using a continuous quantum nondemolition measurement, Phys. Rev. A 76, 012325 (2007).
  16. C. A. Ryan, B. R. Johnson, J. M. Gambetta, J. M. Chow, M. P. da Silva, O. E. Dial, and T. A. Ohki, Tomography via correlation of noisy measurement records, Phys. Rev. A 91, 022118 (2015).
  17. C. C. Bultink, M. A. Rol, T. E. O’Brien, X. Fu, B. C. S. Dikken, C. Dickel, R. F. L. Vermeulen, J. C. de Sterke, A. Bruno, R. N. Schouten, and L. DiCarlo, Active resonator reset in the nonlinear dispersive regime of circuit QED, Phys. Rev. Appl. 6, 034008 (2016).
  18. D. T. McClure, H. Paik, L. S. Bishop, M. Steffen, J. M. Chow, and J. M. Gambetta, Rapid driven reset of a qubit readout resonator, Phys. Rev. Appl. 5, 011001 (2016).
  19. A. Bengtsson, A. Opremcak, M. Khezri, D. Sank, A. Bourassa, K. J. Satzinger, S. Hong, C. Erickson, B. J. Lester, and K. C. Miao et al., Model-based optimization of superconducting qubit readout, Phys. Rev. Lett. 132, 100603 (2024).
  20. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (The MIT Press, Cambridge, MA, USA, 2018), 2nd ed.
  21. S. Kakade, A natural policy gradient, in Proceedings of the 15th International Conference on Neural Information Processing Systems: Natural and Synthetic, edited by T. Dietterich, S. Becker, and Z. Ghahramani (MIT Press, Cambridge, MA, USA, 2001), pp. 1531–1538.
  22. V. François-Lavet, P. Henderson, R. Islam, M. G. Bellemare, and J. Pineau, An introduction to deep reinforcement learning, Foundations and Trends® in Machine Learning 11, 219 (2018).
  23. J. Schulman, S. Levine, P. Moritz, M. I. Jordan, and P. Abbeel, Trust region policy optimization, arXiv:1502.05477.
  24. AnikenC, https://github.com/AnikenC/RL-meets-Qubit-Readout.git GitHub - anikenc/rl-meets-qubit-readout: Repository for demonstration of enhanced qubit readout via reinforcement learning (2024).
  25. A. Blais, A. L. Grimsmo, S. M. Girvin, and A. Wallraff, Circuit quantum electrodynamics, Rev. Mod. Phys. 93, 025005 (2021).
  26. M. F. Dumas, B. Groleau-Paré, A. McDonald, M. H. Muñoz Arias, C. Lledó, B. D’Anjou, and A. Blais, Measurement-induced transmon ionization, Phys. Rev. X 14, 041023 (2024).
  27. C. Hoffer, Superconducting qubit readout pulse optimization using deep reinforcement learning (2021).
  28. B. Lienhard, Machine learning assisted superconducting qubit readout (2021).
  29. J. A. Nelder and R. Mead, A simplex method for function minimization, Comput. J. 7, 308 (1965).
  30. J. Kennedy and R. Eberhart, Particle swarm optimization, in Proceedings of ICNN’95 - International Conference on Neural Networks (IEEE, 1995), Vol. 4, pp. 1942–1948.
  31. X. Cheng, X.-J. Lu, Y.-N. Liu, and S. Kuang, Comparison of differential evolution, particle swarm optimization, quantum-behaved particle swarm optimization, and quantum evolutionary algorithm for preparation of quantum states, Chin. Phys. B 32, 020202 (2023).
  32. Y. Baum, M. Amico, S. Howell, M. Hush, M. Liuzzi, P. Mundada, T. Merkh, A. R. R. Carvalho, and M. J. Biercuk, Experimental deep reinforcement learning for error-robust gate-set design on a superconducting quantum computer, PRX Quantum 2, 040324 (2021).
  33. J. Olle, R. Zen, M. Puviani, and F. Marquardt, Simultaneous discovery of quantum error correction codes and encoders with a noise-aware reinforcement learning agent, Npj Quantum Inf. 10, 126 (2024).
  34. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms, arXiv:1707.06347.
  35. V. Konda and J. Tsitsiklis, Actor-Critic Algorithms, in Advances in Neural Information Processing Systems, edited by S. Solla, T. Leen, and K. Müller (MIT Press, Cambridge, MA, USA, 1999), Vol. 12, https://proceedings.neurips.cc/paper_files/paper/1999/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf.
  36. H. Sowerby, Z. Zhou, and M. L. Littman, Designing rewards for fast learning, arXiv:2205.15400.
  37. E. Jeffrey, D. Sank, J. Y. Mutus, T. C. White, J. Kelly, R. Barends, Y. Chen, Z. Chen, B. Chiaro, and A. Dunsworth et al., Fast accurate state measurement with superconducting qubits, Phys. Rev. Lett. 112, 190504 (2014).
  38. T. Proctor, M. Revelle, E. Nielsen, K. Rudinger, D. Lobser, P. Maunz, R. Blume-Kohout, and K. Young, Detecting and tracking drift in quantum information processors, Nat. Commun. 11, 5396 (2020).
  39. S. Hazra, W. Dai, T. Connolly, P. D. Kurilovich, Z. Wang, L. Frunzio, and M. H. Devoret, Benchmarking the readout of a superconducting qubit for repeated measurements, Phys. Rev. Lett. 134, 100601 (2025).
  40. P. A. Spring, L. Milanovic, Y. Sunada, S. Wang, A. F. van Loo, S. Tamate, and Y. Nakamura, Fast multiplexed superconducting qubit readout with intrinsic Purcell filtering, arXiv:2409.04967 [quant-ph].
  41. J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, http://github.com/google/jax JAX: composable transformations of Python+NumPy programs (2018).
  42. C. Lu, J. Kuba, A. Letcher, L. Metz, C. Schroeder de Witt, and J. Foerster, Discovered policy optimisation, Adv. Neural Inf. Process. Syst. 35, 16455 (2022).
  43. S. Huang, R. F. J. Dossa, A. Raffin, A. Kanervisto, and W. Wang, in ICLR Blog Track (2022), https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/.
  44. A. Bou, M. Bettini, S. Dittert, V. Kumar, S. Sodhani, X. Yang, G. De Fabritiis, and V. Moens, TorchRL: A data-driven decision-making library for PyTorch, arXiv:2306.00577.
  45. J. B. Curtis, I. Boettcher, J. T. Young, M. F. Maghrebi, H. Carmichael, A. V. Gorshkov, and M. Foss-Feig, Critical theory for the breakdown of photon blockade, Phys. Rev. Res. 3, 023062 (2021).
  46. D. I. Schuster, A. Wallraff, A. Blais, L. Frunzio, R. S. Huang, J. Majer, S. M. Girvin, and R. J. Schoelkopf, ac Stark shift and dephasing of a superconducting qubit strongly coupled to a cavity field, Phys. Rev. Lett. 94 (2005).
  47. J. Gambetta, A. Blais, D. I. Schuster, A. Wallraff, L. Frunzio, J. Majer, M. H. Devoret, S. M. Girvin, and R. J. Schoelkopf, Qubit-photon interactions in a cavity: Measurement-induced dephasing and number splitting, Phys. Rev. A 74, 042318 (2006).
  48. P. Kidger, On Neural Differential Equations, Ph.D. thesis, school University of Oxford, 2021.
  49. P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, and J. Bright et al., SciPy 1.0: Fundamental algorithms for scientific computing in Python, Nat. Methods 17, 261 (2020).

Outline

Information

Phys. Rev. Applied 23, 054057– Published 22 May, 2025
  • Received 12 December 2024
  • Revised 3 March 2025
  • Accepted 24 April 2025
Crossmark - Check for updates button
Creative Commons Logo

Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation