IQM researchers are directly addressing the challenge of proving quantum advantage in machine learning with a novel benchmarking approach using its Emerald chip, with 54 qubits, and Garnet chip, with 20 qubits. Their setup employs a “referee” quantum run to create labels optimized for quantum kernels, then independently tests a “challenger” quantum run against eleven classical machine learning models. Results at 15 qubits showed the quantum challenger correctly labelled 81% of test images, significantly outperforming the 51% rate of the best classical model.
Quantum Kernel Method for Fashion-MNIST Classification
IQM used its 54-qubit Emerald processor and 20-qubit Garnet chip to run experiments exploring the limits of quantum machine learning, demonstrating a progression towards increasingly complex hardware for these trials. The experiments employed a dataset of images depicting T-shirts and dresses sourced from Fashion-MNIST, a standard machine learning dataset. The core of the approach involves a “referee” and “challenger” setup, where one quantum run generates labels specifically optimized for quantum kernels, while the other independently assesses the model’s performance.
This methodology addresses a key challenge in the field; demonstrating genuine quantum advantage requires a fair comparison against established classical methods. The referee’s labels are designed to be easily predicted by quantum kernels but difficult for classical similarity measures, using a technique initially published by researchers in 2021.
The team then uses these labels to train and test the challenger quantum circuit, running it on a separate processor or qubit set to ensure independent verification. Eleven classical machine learning models were given the same training data and tasked with the same prediction challenge, providing a direct point of comparison. Esteban Payares, Nadia Milazzo, Ghada Ben Slama and Thomas Cope detailed their findings in the paper, Engineered Learning Problems on Noisy Quantum Processors, outlining the cross-device checks and error mitigation analysis that can be reused for future experiments.
IQM Emerald and Garnet: Referee-Challenger Experiment Setup
This choice allowed for a focused evaluation of the quantum machine learning approach without the confounding variables of highly detailed imagery. The core of the experimental design involved a two-run process, with the “referee” quantum circuit generating labels for images before the “challenger” circuit independently assessed them. This setup enabled a detailed comparison against classical methods, and the team reports that the method is reusable for future experiments.
The researchers also implemented an approach called APT, which approximately halved the effective error in the quantum features, bringing Emerald’s accuracy to within 2.5 points of an ideal model when using labels generated by noise-free simulation. This comparison is critical, as it allows the team to determine whether improvements are due to better quantum computation or simply more effective error mitigation.
While the team acknowledges this work does not yet demonstrate definitive quantum advantage, as the circuits remain simulatable on classical computers, the results offer a pathway toward fairer benchmarking, IQM says. The team ran experiments using up to 45 qubits on the IQM hardware, and the full findings detail where the quantum model’s performance advantage diminishes.
Error Mitigation Techniques Restore Quantum Performance Gaps
Three error mitigation techniques, readout error mitigation, zero-noise extrapolation (ZNE), and a novel approach called adaptive Pauli twirling (APT), were tested on 15-qubit circuits where initial performance gains had disappeared. Readout error mitigation proved ineffective, but both ZNE and APT successfully restored a statistically significant performance gap, improving results by 14 and 17 points respectively.
See today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals.
