Quantum Reinforcement Learning: A Comprehensive Survey by Fraunhofer IIS Team

Researchers from the Fraunhofer Institute for Integrated Circuits in Germany are exploring the intersection of quantum computing and machine learning, specifically quantum reinforcement learning. This emerging field uses quantum resources to solve tasks, with a focus on noisy intermediate-scale quantum devices. The team is particularly interested in variational quantum circuits, which act as function approximators in a classical reinforcement learning setting. They are also investigating quantum reinforcement learning algorithms based on future fault-tolerant hardware, which could offer a provable quantum advantage.

Introduction to Quantum Reinforcement Learning

Quantum reinforcement learning (QRL) is a burgeoning field that combines quantum computing and machine learning. This article provides a comprehensive overview of the literature on QRL, with a particular focus on recent developments. The authors emphasize the use of noisy intermediate-scale quantum devices and variational quantum circuits in a classical reinforcement learning setting. They also discuss QRL algorithms based on future fault-tolerant hardware, some of which offer a provable quantum advantage. The article provides both a broad overview of the field and detailed summaries and reviews of selected literature.

Classical Reinforcement Learning

Reinforcement learning (RL) is a type of machine learning where an agent learns to make decisions by interacting with its environment. The agent observes the state of the environment, takes an action based on a policy, and receives a reward or penalty based on the outcome. The goal of RL is to learn a policy that maximizes the expected cumulative reward. The theoretical basis for RL is formed by Markov decision processes (MDPs) and the associated Bellman equation, which represents a consistency equation for the so-called value function. An optimal policy can be extracted from the optimal value function.

Quantum Computing Paradigm

Quantum computing (QC) is a new paradigm that leverages the principles of quantum mechanics to perform computations. In contrast to classical computing, which uses bits as the smallest unit of data, QC uses quantum bits or qubits. Qubits can exist in a superposition of states, allowing quantum computers to process a vast number of computations simultaneously. Variational quantum circuits (VQCs) are a type of quantum algorithm that can be used as function approximators in a classical reinforcement learning setting.

Quantum Reinforcement Learning Algorithms

The authors provide a detailed overview of various QRL algorithms. They start with quantum-inspired RL algorithms, which use quantum principles to enhance classical RL algorithms. They then delve into QRL algorithms that employ VQCs as function approximators. These algorithms replace a standard neural network function approximator with an appropriate VQC. The authors also discuss QRL algorithms based on future fault-tolerant hardware, which offer a provable quantum advantage.

Outlook on Quantum Reinforcement Learning

The authors conclude with their thoughts on the current state-of-the-art of QRL. They note that while there has been significant progress in the field, there are still many challenges to overcome. They express hope that their literature survey will be useful to researchers in the field and contribute to the further development of QRL.