Qonscious Framework Enables Agnostic Quantum Backend Evaluation

Researchers have developed a new way to compare the performance of quantum computers from different manufacturers, addressing the challenge of widely varying proprietary metrics. The team proposes a Figure of Merit (FoM) based on Grover’s algorithm, designed as an algorithmic stress test that evaluates quantum backends by combining success probability with penalties for errors, yielding a unified score across diverse hardware. This metric accounts for factors static measurements like coherence times and gate error rates often miss, including dynamic variability across successive executions and the impact of the transpilation process. Implemented on the Qonscious framework, a conditional execution platform that readily provides the polymorphic adapters needed to execute the same metric in a consistent manner, the approach allows for the same performance evaluation to run across both IBM and IonQ systems, as well as simulators, enabling a more holistic assessment of quantum computing capabilities in the noisy intermediate-scale quantum (NISQ) era.

GroverFigureOfMerit: Algorithmic Stress Test for Backend Evaluation

Quantum computing’s rapid development is increasingly hampered by a critical bottleneck: comparing the performance of diverse quantum hardware. While numerous companies now offer access to noisy intermediate-scale quantum (NISQ) processors, evaluating which backend best suits a given algorithm remains a significant challenge, as providers expose capabilities through heterogeneous interfaces with proprietary metrics that vary widely. This disparity has prompted the development of a new, unified evaluation tool, the GroverFigureOfMerit (FoM), designed to act as an algorithmic stress test that holistically evaluates the performance of quantum backends. Researchers are moving beyond reliance on static characterization metrics like T1, T2 coherence times, and gate error rates, recognizing their limitations in capturing real-world performance. The new FoM addresses this by embedding performance evaluation directly within an algorithm, Grover’s search, to assess how well a backend can amplify a target state amidst noise and topological constraints.

Central to this approach is the Qonscious framework, a conditional execution platform that readily provides the polymorphic adapters needed to execute the same metric in a consistent manner on IBM and IonQ quantum backends, and simulators. This framework utilizes these adapters to abstract away the proprietary interfaces of each provider, enabling a truly consistent comparison. The FoM itself combines the probability of successfully finding the target state in Grover’s algorithm with penalties for non-uniform amplification and probability leakage to non-marked states, resulting in a single, comparable score. This score isn’t simply a measure of success probability; it actively penalizes backends that exhibit uneven amplification or allow probability to “leak” into incorrect states, providing a more nuanced evaluation of performance. The team’s systematic analysis of heterogeneity across nine quantum providers revealed significant differences in metric semantics and data accessibility, further motivating the need for consistent metrics.

They found that even within similar physical architectures, variations in implementation create challenges for portable compilation. The researchers emphasize that the goal is not to replace static metrics entirely, but to complement them with dynamic, circuit-based FoMs that capture the holistic performance of a quantum system during actual computation, offering a more reliable guide for developers selecting the optimal backend for their algorithms.

NISQ Era Challenges: Heterogeneity of Quantum Provider Metrics

The current quantum computing landscape is marked by a proliferation of hardware providers, each offering access to increasingly sophisticated, yet fundamentally diverse, systems. This expansion, while promising, introduces a significant challenge: comparing performance across these platforms is far from straightforward. The reliance on traditional, static characterization metrics, like T1 and T2 coherence times, is proving insufficient, as these measurements offer only a partial picture of real-world performance. A core limitation of these static metrics is their inability to capture the dynamic nature of quantum systems. As the research demonstrates, these measurements “fail to capture dynamic variability across successive executions,” meaning a snapshot of a qubit’s properties may not reflect its behavior moments later. This temporal instability is compounded by the crucial, yet often overlooked, impact of the transpilation process.

Converting a high-level quantum algorithm into a series of native gates executable on specific hardware introduces significant circuit depth, potentially negating the benefits of high-fidelity individual gates. For example, superconducting systems from different companies hinder direct comparison. To address this, researchers are moving beyond static characterization and focusing on algorithmic stress tests. The team’s analysis across nine quantum providers motivates the need for consistent metrics, highlighting that the relevant variables for developers are the “effective constraints of the computational graph,” and that standardized methodologies for evaluating performance are essential for building portable compilers and evaluating real hardware performance.

Tiago Restucha and colleagues at Universidad Nacional de La Plata in Argentina are tackling this head-on with the Qonscious framework, a conditional execution platform that readily provides the polymorphic adapters needed to execute the same metric in a consistent manner on IBM and IonQ backends, as well as simulators. The team’s approach moves beyond these by focusing on algorithmic performance, specifically using Grover’s algorithm as an algorithmic stress test. This isn’t simply about achieving a result; the FoM incorporates penalties for imperfections in the process. Qonscious achieves backend independence through effectively translating the unique communication protocols of each quantum processor into a common language. This allows the same performance metric to be run consistently, regardless of the underlying hardware. The researchers conducted a systematic analysis of nine quantum providers, which motivates the need for consistent metrics, revealing significant differences in how calibration data is presented and interpreted.

The escalating competition to build practical quantum computers is increasingly focused on accurately measuring performance, a challenge that extends far beyond simple qubit counts. Researchers are now advocating for a shift towards dynamic, circuit-based Figures of Merit (FoMs) that evaluate performance through actual algorithmic execution. This approach acknowledges that “raw device parameters prove insufficient” to predict how a quantum computer will perform on a specific task. For instance, some providers utilize ECR/SX gates while others rely on CZ/CPHASE, and the way topology, the connectivity between qubits, is exposed differs considerably. This motivates the need for consistent metrics and complicates the development of portable compilers, making direct comparison of static metrics unreliable. Instead, circuit-based FoMs offer a dynamic assessment, capturing performance as it happens and providing a more relevant measure of a quantum computer’s current capabilities.

The pursuit of reliable quantum computation increasingly clashes with a surprising reality: accurately gauging performance isn’t simply about measuring qubit coherence or gate fidelity. While those static characterization metrics remain valuable, a growing body of research demonstrates their inadequacy in predicting how well a quantum computer will actually perform a given task. This consistent approach abstracts away the unique interfaces, allowing for meaningful comparisons. The FoM incorporates the scoring function from recent approaches such as GRADE into Qonscious’s extensible architecture. For example, IBM reports gate errors for each physical link, while other architectures provide global averages, making direct comparison difficult.