A 0.339 accuracy improvement on real datasets resulted solely from tuning the bandwidth of a quantum feature map, exceeding any observed difference between quantum and classical kernels, according to new research. Ronak Bhalgami, an independent researcher, systematically compared quantum and classical kernel methods, finding that tuned quantum kernels are statistically indistinguishable from a tuned RBF on four real datasets.

This challenges previous benchmarks and suggests earlier positive results may have stemmed from flawed comparison methodologies. The work demonstrates that protocol controls, specifically, tuning the feature map’s bandwidth and recomputing the kernel within each fold, are critical for accurate evaluation.

Quantum Kernel Performance Mirrors Tuned RBF on Real Datasets

A bandwidth adjustment of the quantum feature map yielded a 0.120 accuracy improvement on synthetic data and a 0.339 on real data, exceeding every quantum-classical difference observed. This finding highlights the importance of classical optimization techniques, suggesting that meticulous tuning of classical kernel hyperparameters can often overshadow the benefits of employing a quantum approach. The research team systematically investigated the conditions under which quantum kernels might outperform their classical counterparts, focusing on controlled comparisons to isolate the true impact of quantum computation.

Researchers found that a tuned quantum kernel scored 0.956 on real data, demonstrating the substantial impact of protocol controls on performance. This highlights a critical methodological point: a comparison that fixes the quantum bandwidth while tuning the classical kernel is, in effect, measuring its own protocol rather than the kernels themselves.

Further analysis revealed a surprising trend regarding scalability; the researchers observed a -0.0045 accuracy decrease per qubit on synthetic data as dimensionality increased. This suggests that the potential quantum advantage diminishes as the problem size grows, a limitation that could hinder the application of quantum kernels to high-dimensional datasets. The team conducted a factorial design encompassing 324 cells, explicitly varying dimensionality, training size, class separability and data structure to map the influence of these factors on kernel performance.

By counting cost in hardware circuit evaluations, they aimed to provide a more accurate comparison of the resources required by each approach. The team’s findings emphasize that protocol variance alone can explain conflicting published conclusions, even without considering any inherent properties of the quantum kernels themselves. The study’s central methodological claim is that tuning the feature map’s bandwidth is paramount, and that failing to do so introduces a significant bias into any comparative analysis.

Bandwidth Tuning Dominates Kernel Accuracy Gains

A carefully tuned bandwidth within quantum feature maps delivers a substantial accuracy boost, 0.120 on synthetic data and 0.339 on real datasets, outperforming any observed advantage from using a quantum kernel itself. This finding emphasises that optimizing classical parameters can have a more significant impact on performance than the choice of quantum versus classical methods. The research team’s factorial design, encompassing 324 distinct conditions, systematically varied factors like dimensionality, training size and data structure alongside the four standard datasets used in the evaluation.

This highlights the importance of a sufficiently fine-grained search to accurately pinpoint optimal parameters. Further analysis of noise resilience showed that sampling error, shot noise, dominates gate fidelity errors by approximately an order of magnitude, indicating that the number of times a quantum circuit is run is a more critical constraint than the accuracy of the individual quantum operations.

However, the most striking result remains the dominance of bandwidth tuning. The observed accuracy gains from this single parameter adjustment exceeded all quantum-classical differences measured in the study. The researchers also tested a positive control dataset generated by the feature map itself, confirming a correct implementation by achieving high accuracy, and attributing any null results elsewhere to the method itself rather than a technical error.

The study’s findings demonstrate that the choice of bandwidth is paramount, and failure to properly tune this parameter can lead to misleading conclusions about the efficacy of quantum kernel methods. The team’s commitment to pre-committing hypotheses and analysis choices before examining results further strengthens the reliability of their findings. This rigorous approach, coupled with the comprehensive factorial design, provides a robust framework for comparing quantum and classical kernel methods and identifying the key factors that influence their performance.

Factorial Design Isolates Data Condition Covariates

The study treated the comparison between quantum and classical approaches as the primary object of investigation, establishing a rigorous framework for isolating the impact of specific data characteristics. The experimental design incorporated a four-tiered approach, beginning with a full factorial sweep across dimensionality, training set size, class separability and data structure, each tested with three independent data seeds. This generated the 324 cells used to transform “data conditions” from qualitative observations into quantifiable regression covariates.

Tier B validated these findings using four standard datasets, while subsequent tiers explored the relative impact of shot noise versus gate errors and the cost associated with different computational approaches. The analysis revealed a negative correlation between dimensionality and quantum advantage, with a decrease of 0.0045 accuracy per additional qubit. The advantage on synthetic data was confined almost entirely to data generated by the feature map itself.

Protocol Controls Address Benchmarking Discrepancies

A carefully controlled comparison of quantum and classical machine learning algorithms revealed that tuning the bandwidth of the quantum feature map improves accuracy by 0.120 on synthetic data and 0.339 on real data, exceeding any observed advantage of the quantum method itself. This finding highlights the importance of rigorous protocol controls in benchmarking quantum machine learning, as the largest gains stem from optimizing classical parameters rather than the quantum kernel itself.

The research, detailed in a new paper, directly addresses inconsistencies in previously published benchmarks by establishing a standardized methodology for comparison. The team’s protocol tackles three key issues that have plagued previous evaluations: asymmetry in hyperparameter tuning, data leakage through the encoding process, and the use of simulator time as a cost metric.

By tuning the bandwidth of both the quantum feature map and the classical kernel on the same computational budget, the researchers ensured a fair comparison. The kernel was recomputed within each fold of the analysis using fold-fitted preprocessing, preventing information from leaking between training and testing sets. Cost was measured in hardware circuit evaluations, a more realistic metric for assessing practical performance. This rigorous approach led to a surprising result: tuned quantum kernels performed statistically indistinguishably from a tuned radial basis function (RBF) kernel on four standard datasets.

This challenges earlier claims of quantum advantage and suggests that previously reported positive results may have been artifacts of inadequate control. The full implementation, experiment configurations, and result tables are publicly available, ensuring reproducibility and allowing other researchers to verify the findings.

The selected bandwidth for the quantum feature map was a parameter tuned during the analysis, not a fixed value. Across tuned folds, the bandwidth exhibited a bimodal distribution, with the majority of folds selecting either the grid floor or a higher value. The team acknowledges that extending this controlled protocol to larger systems, learned feature maps, and real hardware presents further challenges.

However, they argue that these controls are even more critical in those contexts, particularly given the expense of re-tuning on hardware due to limited shot budgets. The research provides an important step towards establishing a reliable framework for evaluating quantum machine learning algorithms and determining their true potential.

Finite Sampling Costs Outweigh Quantum Noise

Finite sampling introduces more substantial errors than gate and readout noise in quantum kernel methods, reducing accuracy by 0.060 on average, while gate and readout noise alone contribute a negligible +0.007 accuracy change. This disparity stems from the nature of the errors themselves; depolarizing noise smoothly contracts the kernel, easily absorbed by classical tuning, whereas finite sampling introduces unstructured errors that push the kernel matrix outside the positive semi-definite cone, requiring significantly more computational repair.

The repair mass, a measure of the effort needed to restore the kernel’s validity, increases by a factor of approximately 5,000 when finite sampling is introduced, highlighting the magnitude of the challenge. “Finite sampling costs an order of magnitude more accuracy than gate and readout noise,” the study reports, emphasizing the practical limitations imposed by the need for extensive sampling to achieve reliable results.

Synthetic Data Highlights Feature Map Advantage

This finding emphasises the critical role of classical optimization in realizing the potential of near-term quantum machine learning. There was a 0.0045 accuracy per qubit, a result that challenges expectations of quantum computers excelling at high-dimensional problems. The researchers argue that the focus should shift from simply demonstrating a quantum effect to understanding the interplay between quantum and classical components within the machine learning pipeline.

The study also highlights the substantial computational cost associated with evaluating quantum kernels, noting that achieving a fidelity kernel requires approximately 5 x $10^{7}$ circuit evaluations with a training size of 10,000. This cost is driven primarily by finite sampling, which the researchers found to introduce errors that outweigh those from gate and readout noise.

👉 More information
🗞 Empirical Evaluation of Quantum Kernel Methods for Classification: A Comparative Study with Classical Kernels
✍️ Ronak Bhalgami (Independent Researcher)
🧠 ArXiv: https://arxiv.org/abs/2610.42029

More like this