Classical Machine Learning vs Quantum Machine Learning. QML Out-Gunned by Classical Models. Tested on 12 Models.

Researchers from Xanadu and Chalmers University of Technology have conducted a large-scale study testing 12 quantum machine learning models on six binary classification tasks. The study found that classical machine learning models generally outperformed quantum classifiers, contradicting claims made in several previous studies.

The researchers also found that removing entanglement from a quantum model often improved its performance. The study raises important questions for quantum model design and emphasizes the need for rigorous, evidence-based evaluations in the field. The researchers have developed an open-source package to facilitate further testing and evaluation of quantum machine learning models.

Is Quantum Machine Learning Outperforming Classical Models?

Quantum machine learning is a rapidly evolving field that seeks to leverage the potential of quantum computers to fundamentally transform our ability to learn from data. Just as classical machine learning has taken the world by storm. However, with the absence of noise-free quantum hardware, the evaluation of quantum machine learning models is primarily conducted through classical simulations. This approach, while necessary, is fraught with challenges due to the significant influence of experimental design on results, the small scales currently accessible, and narratives shaped by the commercialization of quantum technologies.

A team of researchers from Xanadu, a quantum technology company, and Chalmers University of Technology have developed an open-source package based on the PennyLane software framework to facilitate better decision-making in this field. They conducted a large-scale study that systematically tested 12 popular quantum machine learning models on six binary classification tasks, creating 160 individual datasets. The results of this study provide valuable insights into the performance of quantum models compared to classical ones and raise important questions for quantum model design.

How Do Quantum Models Compare to Classical Ones?

The researchers found that classical machine learning models outperformed quantum classifiers out of the box. This finding contradicts the claims made in about 40 of 55 relevant papers published on the preprint server arXiv until December 2023, which stated that a quantum model outperformed a classical one. The researchers also discovered that removing entanglement from a quantum model often resulted in a good or better performance. This suggests that quantumness may not be the crucial ingredient for the small learning tasks considered in this study.

The study also went beyond simplistic leaderboard comparisons, identifying five essential questions for quantum model design that emerged from the results. These questions will likely guide future research in the field and contribute to a more nuanced understanding of quantum machine learning’s potential and limitations.

What Are the Implications?

The findings of this study challenge the prevailing narrative that quantum machine learning is already outperforming classical machine learning in generic domains. While the field is progressing rapidly, the results suggest that we should be cautious in our optimism and rigorous evaluations. The researchers argue for an evidence-based evaluation of proposals in near-term quantum machine learning algorithm design, emphasizing the importance of scientific best practices in reducing bias.

The study also underscores the importance of benchmarking models via classical simulations despite the inherent challenges. This approach remains one of the main ways to judge ideas in quantum machine learning before noise-free hardware becomes available. The researchers’ open-source package and large-scale study provide valuable resources for others in the field to conduct rigorous evaluations.

What Does This Mean for the Future of Quantum Machine Learning?

The results of this study do not diminish the potential of quantum machine learning. Instead, they provide a more nuanced picture of the field’s current state and highlight important areas for future research. The researchers’ findings suggest that quantumness may not be the crucial ingredient for small learning tasks, raising intriguing questions about what aspects of quantum computing could provide advantages in machine learning.

The study also highlights the importance of rigorous, evidence-based evaluations in advancing the field. By challenging prevailing narratives and providing a robust framework for testing quantum machine learning models, the researchers are helping to ensure that the field’s progress is grounded in solid scientific evidence.

How Can We Improve Quantum Machine Learning Models?

The researchers’ findings suggest several avenues for improving quantum machine learning models. For instance, the fact that removing quantum entanglement from a quantum model often resulted in as good or better performance indicates that researchers may need to rethink the role of entanglement in these models. This could lead to new approaches to quantum model design that leverage other aspects of quantum computing.

The study also highlights the importance of rigorous testing and evaluation in improving quantum machine learning models. By systematically testing popular quantum machine learning (QML) models on a variety of tasks, the researchers were able to identify consistent signals across experiments that can guide future research. This approach could help researchers identify promising directions for future research and avoid investing resources in less promising avenues.