- Physical Review Journals
All JournalsPhysics Magazine
Access Provided by Xinjiang University

Physical Review Applied

Export citation

Export citation

Choose format for download:

Download Citation
  • Access by Xinjiang University

Designing fast quantum gates using optimal control with a reinforcement-learning ansatz

Bijita Sarma1,* and Michael J. Hartmann1,2

  • *Contact author: bijita.sarma@fau.de
PDF

Phys. Rev. Applied 23, 014015 – Published 6 January, 2025

DOI: https://doi.org/10.1103/PhysRevApplied.23.014015

Abstract

Fast quantum gates are crucial not only for the contemporary era of noisy intermediate-scale quantum devices but also for the prospective development of practical fault-tolerant quantum computing. Leakage errors, which arise from data qubits jumping beyond the confines of the computational subspace, are the main challenges in realizing nonadiabatically driven, fast gates. In this work, we propose and illustrate the usefulness of reinforcement learning (RL) to generate fast two-qubit gates in practical multilevel superconducting qubits. In particular, we show that the RL controller offers great effectiveness in finding piecewise constant gate-pulse sequences that act on two transmon data qubits coupled by a tunable coupler to generate a controlled-Z (cz) gate with a gate time of 10 ns and an error rate of approximately 4×10−3. Using a gradient-based method to solve the same optimization problem often does not achieve high fidelity for such fast gates. However, we show that using the gate pulses discovered by RL as an ansatz for the gradient-based controller can substantially enhance fidelity compared to using RL alone. While for a 10-ns pulse, this improvement is marginal, the combined RL + gradient approach decreases the gate errors below 10−4 for a gate of length 20 ns.

View figure in article

Physics Subject Headings (PhySH)

Article Text

References (57)

  1. M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information: 10th Anniversary Edition (Cambridge University Press, Cambridge, England, UK, 2010).
  2. M. Cerezo, A. Arrasmith, R. Babbush, S. C. Benjamin, S. Endo, K. Fujii, J. R. McClean, K. Mitarai, X. Yuan, L. Cincio, and P. J. Coles, Variational quantum algorithms, Nat. Rev. Phys. 3, 625 (2021).
  3. J. Preskill, Quantum computing in the NISQ era and beyond, Quantum 2, 79 (2018).
  4. F. Arute, K. Arya, R. Babbush, D. Bacon, J. C. Bardin, R. Barends, R. Biswas, S. Boixo, F. G. S. L. Brandao, D. A. Buell, et al., Quantum supremacy using a programmable superconducting processor, Nature 574, 505 (2019).
  5. M. Kjaergaard, M. E. Schwartz, J. Braumüller, P. Krantz, J. I.-J. Wang, S. Gustavsson, and W. D. Oliver, Superconducting qubits: Current state of play, Annu. Rev. Condens. Matter Phys. 11, 369 (2020).
  6. S. Rosenblum, P. Reinhold, M. Mirrahimi, L. Jiang, L. Frunzio, and R. J. Schoelkopf, Fault-tolerant detection of a quantum error, Science 361, 266 (2018).
  7. L. DiCarlo, J. M. Chow, J. M. Gambetta, L. S. Bishop, B. R. Johnson, D. I. Schuster, J. Majer, A. Blais, L. Frunzio, S. M. Girvin, and R. J. Schoelkopf, Demonstration of two-qubit algorithms with a superconducting quantum processor, Nature 460, 240 (2009).
  8. C. K. Andersen, A. Remm, S. Lazar, S. Krinner, N. Lacroix, G. J. Norris, M. Gabureac, C. Eichler, and A. Wallraff, Repeated quantum error detection in a surface code, Nat. Phys. 16, 875 (2020).
  9. Y. Ma, Y. Xu, X. Mu, W. Cai, L. Hu, W. Wang, X. Pan, H. Wang, Y. P. Song, C.-L. Zou, and L. Sun, Error-transparent operations on a logical qubit protected by quantum error correction, Nat. Phys. 16, 827 (2020).
  10. S. Krinner, N. Lacroix, A. Remm, A. Di Paolo, E. Genois, C. Leroux, C. Hellings, S. Lazar, F. Swiadek, J. Herrmann, et al., Realizing repeated quantum error correction in a distance-three surface code, Nature 605, 669 (2022).
  11. E. Knill and R. Laflamme, Theory of quantum error-correcting codes, Phys. Rev. A 55, 900 (1997).
  12. A. L. Grimsmo and S. Puri, Quantum error correction with the Gottesman-Kitaev-Preskill code, PRX Quantum 2, 020101 (2021).
  13. R. Acharya, I. Aleiner, R. Allen, T. I. Andersen, M. Ansmann, F. Arute, K. Arya, A. Asfaw, J. Atalaya, R. Babbush, et al., Suppressing quantum errors by scaling a surface code logical qubit, Nature 614, 676 (2023).
  14. Z. Chen, J. Kelly, C. Quintana, R. Barends, B. Campbell, Y. Chen, B. Chiaro, A. Dunsworth, A. G. Fowler, E. Lucero, et al., Measuring and suppressing quantum state leakage in a superconducting qubit, Phys. Rev. Lett. 116, 020501 (2016).
  15. C. C. Bultink, T. E. O’Brien, R. Vollmer, N. Muthusubramanian, M. W. Beekman, M. A. Rol, X. Fu, B. Tarasinski, V. Ostroukh, B. Varbanov, A. Bruno, and L. DiCarlo, Protecting quantum entanglement from leakage and qubit errors via repetitive parity measurements, Sci. Adv. 6, eaay3050 (2020).
  16. R. Fazio, G. M. Palma, and J. Siewert, Fidelity and leakage of Josephson qubits, Phys. Rev. Lett. 83, 5385 (1999).
  17. K. S. Chou, T. Shemma, H. McCarrick, T.-C. Chien, J. D. Teoh, P. Winkel, A. Anderson, J. Chen, J. Curtis, S. J. de Graaf, et al., Demonstrating a superconducting dual-rail cavity qubit with erasure-detected logical measurements, ArXiv:2307.03169.
  18. F. Yan, P. Krantz, Y. Sung, M. Kjaergaard, D. L. Campbell, T. P. Orlando, S. Gustavsson, and W. D. Oliver, Tunable coupling scheme for implementing high-fidelity two-qubit gates, Phys. Rev. Appl. 10, 054062 (2018).
  19. L. Heunisch, C. Eichler, and M. J. Hartmann, Tunable coupler to fully decouple and maximally localize superconducting qubits, Phys. Rev. Appl. 20, 064037 (2023).
  20. P. Mundada, G. Zhang, T. Hazard, and A. Houck, Suppression of qubit crosstalk in a tunable coupling superconducting circuit, Phys. Rev. Appl. 12, 054023 (2019).
  21. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (MIT Press, Cambridge, MA, 2016).
  22. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al., Mastering the game of Go with deep neural networks and tree search, Nature 529, 484 (2016).
  23. D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, et al., Mastering the game of Go without human knowledge, Nature 550, 354 (2017).
  24. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA, 2018).
  25. V. Krotov, Global Methods in Optimal Control Theory (CRC Press, Boca Raton, FL, USA, 1995).
  26. T. Caneva, T. Calarco, and S. Montangero, Chopped random-basis quantum optimization, Phys. Rev. A 84, 022326 (2011).
  27. M. Krenn, J. Landgraf, T. Foesel, and F. Marquardt, Artificial intelligence and machine learning for quantum technologies, Phys. Rev. A 107, 010101 (2023).
  28. V. Gebhart, R. Santagati, A. A. Gentile, E. M. Gauger, D. Craig, N. Ares, L. Banchi, F. Marquardt, L. Pezzè, and C. Bonato, Learning quantum systems, Nat. Rev. Phys. 5, 141 (2023).
  29. M. Bukov, A. G. R. Day, D. Sels, P. Weinberg, A. Polkovnikov, and P. Mehta, Reinforcement learning in different phases of quantum control, Phys. Rev. X 8, 031086 (2018).
  30. T. Fösel, P. Tighineanu, T. Weiss, and F. Marquardt, Reinforcement learning with neural networks for quantum feedback, Phys. Rev. X 8, 031084 (2018).
  31. R. Porotti, D. Tamascelli, M. Restelli, and E. Prati, Coherent transport of quantum states by deep reinforcement learning, Commun. Phys. 2, 1 (2019).
  32. B. Sarma, S. Borah, A. Kani, and J. Twamley, Accelerated motional cooling with deep reinforcement learning, Phys. Rev. Res. 4, L042038 (2022).
  33. L. Ding, M. Hays, Y. Sung, B. Kannan, J. An, A. Di Paolo, A. H. Karamlou, T. M. Hazard, K. Azar, D. K. Kim, et al., High-fidelity, frequency-flexible two-qubit fluxonium gates with a transmon coupler, Phys. Rev. X 13, 031035 (2023).
  34. S. Borah, B. Sarma, M. Kewming, G. J. Milburn, and J. Twamley, Measurement-based feedback quantum control with deep reinforcement learning for a double-well nonlinear potential, Phys. Rev. Lett. 127, 190403 (2021).
  35. Z. T. Wang, Y. Ashida, and M. Ueda, Deep reinforcement learning control of quantum cartpoles, Phys. Rev. Lett. 125, 100401 (2020).
  36. S. Borah and B. Sarma, No-collapse accurate quantum feedback control via conditional state tomography, Phys. Rev. Lett. 131, 210803 (2023).
  37. V. V. Sivak, A. Eickbusch, B. Royer, S. Singh, I. Tsioutsios, S. Ganjam, A. Miano, B. L. Brock, A. Z. Ding, L. Frunzio, S. M. Girvin, R. J. Schoelkopf, and M. H. Devoret, Real-time quantum error correction beyond break-even, Nature 616, 50 (2023).
  38. K. Reuer, J. Landgraf, T. Fösel, J. O’Sullivan, L. Beltrán, A. Akin, G. J. Norris, A. Remm, M. Kerschbaum, J.-C. Besse, et al., Realizing a deep reinforcement learning agent for real-time quantum feedback, Nat. Commun. 14, 1 (2023).
  39. N. Khaneja, T. Reiss, C. Kehlet, T. Schulte-Herbrüggen, and S. J. Glaser, Optimal control of coupled spin dynamics: design of NMR pulse sequences by gradient ascent algorithms, J. Magn. Reson. 172, 296 (2005).
  40. G. Jäger, D. M. Reich, M. H. Goerz, C. P. Koch, and U. Hohenester, Optimal quantum control of Bose-Einstein condensates in magnetic microtraps: Comparison of gradient-ascent-pulse-engineering and Krotov optimization schemes, Phys. Rev. A 90, 033628 (2014).
  41. V. F. Krotov, in Advances in Nonlinear Dynamics and Control: A Report from Russia, SpringerLink (Birkhäuser Boston, 1993), p. 74.
  42. Y. Sung, L. Ding, J. Braumüller, A. Vepsäläinen, B. Kannan, M. Kjaergaard, A. Greene, G. O. Samach, C. McNally, D. Kim, et al., Realization of high-fidelity CZ and ZZ-free iSWAP gates with a tunable coupler, Phys. Rev. X 11, 021058 (2021).
  43. J. Stehlik, D. M. Zajac, D. L. Underwood, T. Phung, J. Blair, S. Carnevale, D. Klaus, G. A. Keefe, A. Carniol, M. Kumph, M. Steffen, and O. E. Dial, Tunable coupling architecture for fixed-frequency transmon superconducting qubits, Phys. Rev. Lett. 127, 080505 (2021).
  44. T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, ArXiv:1801.01290.
  45. G. T. Genov, S. Rochester, M. Auzinsh, F. Jelezko, and D. Budker, Robust two-state swap by stimulated Raman adiabatic passage, J. Phys. B: At. Mol. Opt. Phys. 56, 054001 (2023).
  46. B. Khani, J. M. Gambetta, F. Motzoi, and F. K. Wilhelm, Optimal generation of Fock states in a weakly nonlinear oscillator, Phys. Scr. 2009, 014021 (2009).
  47. M. Suchara, A. W. Cross, and J. M. Gambetta, Leakage suppression in the Toric code, Quantum Inf. Comput. 15, 997 (2015).
  48. P. Aliferis and B. M. Terhal, Fault-tolerant quantum computation for local leakage faults, Quantum Inf. Comput. 7, 139 (2007).
  49. M. Werninghaus, D. J. Egger, F. Roy, S. Machnes, F. K. Wilhelm, and S. Filipp, Leakage reduction in fast superconducting qubit gates via optimal control, npj Quantum Inf. 7, 1 (2021).
  50. M. McEwen, D. Kafri, Z. Chen, J. Atalaya, K. J. Satzinger, C. Quintana, P. V. Klimov, D. Sank, C. Gidney, A. G. Fowler, et al., Removing leakage-induced correlated errors in superconducting quantum error correction, Nat. Commun. 12, 1 (2021).
  51. K. C. Miao, M. McEwen, J. Atalaya, D. Kafri, L. P. Pryadko, A. Bengtsson, A. Opremcak, K. J. Satzinger, Z. Chen, P. V. Klimov, et al., Overcoming leakage in quantum error correction, Nat. Phys. 19, 1780 (2023).
  52. E. Hyyppä, A. Vepsäläinen, M. Papič, C. F. Chan, S. Inel, A. Landra, W. Liu, J. Luus, F. Marxer, C. Ockeloen-Korppi, et al., Reducing leakage of single-qubit gates for superconducting quantum processors using analytical control pulse envelopes, PRX Quantum 5, 030353 (2024).
  53. L. Chen, S. P. Fors, Z. Yan, A. Ali, T. Abad, A. Osman, E. Moschandreou, B. Lienhard, S. Kosen, H.-X. Li, et al., Fast unconditional reset and leakage reduction in fixed-frequency transmon qubits, ArXiv:2409.16748.
  54. R. Barends, J. Kelly, A. Megrant, A. Veitia, D. Sank, E. Jeffrey, T. C. White, J. Mutus, A. G. Fowler, B. Campbell, et al., Superconducting quantum circuits at the surface code threshold for fault tolerance, Nature 508, 500 (2014).
  55. A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, Stable-baselines3: Reliable reinforcement learning implementations, J. Mach. Learn. Res. 22, 1 (2021).
  56. J. Achiam, Spinning Up in Deep Reinforcement Learning (2018), https://github.com/openai/spinningup.
  57. S. Oh, Errors due to finite rise and fall times of pulses in superconducting charge qubits, Phys. Rev. B 65, 144526 (2002).

Outline

Information

Phys. Rev. Applied 23, 014015– Published 6 January, 2025
  • Received 10 January 2024
  • Revised 22 November 2024
  • Accepted 25 November 2024
Crossmark - Check for updates button

© 2025 American Physical Society

Sign In to Your Journals Account

Filter

Filter

Article Lookup

Enter a citation