Transformer Hits 1.000 Fidelity on 6-Qubit Parameterized Circuits

Mehdi Saeedi, Eddie Richter, and Paul Hartke report achieving median fidelity of 1.000 on 3-6 qubit parameterized circuits using a hybrid approach that combines a 44.8 million-parameter transformer with classical optimization techniques. While this approach attains near-perfect fidelity on parameterized circuits, a limitation emerges when striving for exact equivalence in Clifford+T circuits. Though the model learns valid syntax and accurate T-gate statistics, accuracy degrades as circuit length increases, falling to near zero beyond 26 gates. This failure is termed autoregressive drift, early-token divergence cascading through left-to-right decoding, though inference-time strategies raising exact-match rates from 7% to 22.5% offer a partial solution. The researchers state the contrast is clear: the transformer succeeds when approximate outputs can be refined, but struggles with the need for exact discrete correctness.

Transformer Architecture for Parameterized Circuits

Achieving median fidelity of 1.000 on 3-6 qubit circuits represents a significant leap forward in quantum circuit optimization, as Mehdi Saeedi, Eddie Richter, and Paul Hartke have demonstrated a hybrid approach leveraging a 44.8 million-parameter transformer alongside classical optimization techniques. This success hinges on a carefully designed architecture featuring structured circuit tokenization, where “every token carries circuit-semantic information,” utilizing a vocabulary of 102 tokens. The transformer predicts circuit structure with 100% structural validity across all qubit counts, while a separate regression head outputs angle estimates. This division of labor allows classical algorithms to focus on a reduced problem space. However, the study reveals a critical limitation when applied to Clifford+T circuits, where discrete gates preclude post-processing corrections. While the model maintains 100% structural validity and accurately predicts T-gate counts, exact functional equivalence remains limited. The study traces this failure to autoregressive drift: early-token divergence cascading irrecoverably through left-to-right decoding.

Accuracy degrades as circuit length increases, falling to near zero beyond 26 gates. Interestingly, generating multiple circuit candidates and verifying them, rather than refining the model itself, offers a partial solution, raising exact-match rates. This suggests that the method of utilizing the model is important for achieving accuracy.

Clifford+T Circuit Optimization with Discrete Gates

The pursuit of practical quantum computation increasingly focuses on optimizing circuits for fault tolerance, a challenge where recent advances reveal both promise and fundamental limitations. Researchers are now leveraging transformer-based neural networks to streamline Clifford+T circuits, the standard gate set for error-corrected quantum computers, but achieving precise, reliable results proves surprisingly difficult. Mehdi Saeedi, Eddie Richter, and Paul Hartke have demonstrated a hybrid approach pairing a 44.8 million-parameter transformer with classical optimization, attaining median fidelity of 1.000 on 3-6 qubit parameterized circuits. This success hinges on the ability to refine angle estimates after the transformer establishes the circuit’s basic structure. However, the same model falters when tasked with generating fully discrete Clifford+T circuits, where no post-processing adjustments are possible. The researchers report that “exact equivalence degrades sharply with target length,” observing an accuracy drop from 88% on circuits with nine or fewer gates to near zero beyond 26 gates.

This phenomenon, termed autoregressive drift, stems from early-token divergence cascading irrecoverably through left-to-right decoding. While scaling training data by 2.5 times pushes exact-match rates to 39.5%, the underlying degradation with circuit length persists. This suggests that the method of utilizing the model, rather than its internal design, is a key factor in achieving accuracy. They state that the contrast between settings is their central finding: the transformer excels when approximate outputs can be rescued by post-processing, but struggles with discrete correctness.

A team, led by Mehdi Saeedi and Eddie Richter, has demonstrated a hybrid approach pairing a 44.8 million-parameter transformer with classical optimization, attaining median fidelity of 1.000 on 3-6 qubit parameterized circuits. On Clifford+T circuits, where all gates are discrete and no post-processing is possible, the model learns valid syntax and accurate T-count statistics, yet exact equivalence degrades sharply with target length, from 88% on circuits with ≤9 gates to near zero beyond 26 gates. Researchers pinpoint this failure to autoregressive drift: early-token divergence cascading irrecoverably through left-to-right decoding. Two strategies partially mitigate the drift: inference-time strategies that generate multiple candidates and select via equivalence verification raise exact-match rates from 7% to 22.5%, while scaling training data by 2.5 times pushes them to 39.5%. Yet the degradation with target length persists, even with more data, exact equivalence drops from 94% on short circuits to under 4% beyond 26 gates. The contrast between settings is the central finding: when approximate outputs can be rescued by post-processing, the transformer succeeds; when exact discrete correctness is required, autoregressive drift limits reliability, with both inference-time search and data scaling as effective levers.

The ability to reliably translate quantum algorithms into executable circuits hinges on maintaining precision throughout the optimization process; however, recent research reveals a critical limitation in transformer-based approaches when generating discrete gate sequences. While a 44.8 million-parameter transformer achieves median fidelity of 1.000 on 3-6 qubit circuits, unlike parameterized circuits where post-processing can compensate for structural inaccuracies, discrete Clifford+T circuits offer no such recourse. Exact equivalence degrades sharply with target length, from 88% on circuits with ≤9 gates to near zero beyond 26 gates. Two strategies partially mitigate the drift: inference-time strategies that generate multiple candidates and select via equivalence verification raise exact-match rates from 7% to 22.5%, while scaling the training data by 2.5 times pushes them to 39.5%. Yet the degradation with target length persists, even with more data, exact equivalence drops from 94% on short circuits to under 4% beyond 26 gates.

While achieving near-perfect replication of parameterized circuits with a 44.8 million-parameter transformer represents a significant advance, the model’s performance falters when absolute accuracy is paramount. This isn’t a limitation of the model’s learning capacity, but rather a consequence of where initial errors cascade through the decoding process, rendering longer circuits unreliable. Interestingly, Mehdi Saeedi, Eddie Richter, and Paul Hartke found that improving the model itself offered limited gains. Scaling training data by 2.5 times increased exact-match rates to 39.5%, but the degradation with circuit length persisted. Instead, focusing on how the model is used, inference-time strategies, yielded a surprising boost of 5%. The team tested several approaches, finding that “candidate pooling and model ensembling each yield only marginal gains” beyond the 22.5% achieved with the multi-candidate strategy. This highlights a pragmatic approach to near-term quantum compilation: leveraging the model’s strengths on shorter circuits within existing compilers, rather than striving for perfect, end-to-end synthesis of complex algorithms.

The ability to refine quantum circuits, reducing the number of costly T-gates while maintaining functionality, is proving surprisingly sensitive to the scale of training data, according to recent work by Mehdi Saeedi, Eddie Richter, and Paul Hartke. While a 44.8 million-parameter transformer achieves median fidelity of 1.000 on 3-6 qubit circuits, researchers found that even with increased data, this highlights that the problem isn’t a lack of learned patterns, but an inability to recover from initial errors in discrete token sequences. This suggests that, for now, the most effective path toward reliable quantum circuit optimization lies in mitigating the consequences of errors, rather than eliminating them entirely. The team’s experiments with fine-tuning, candidate pooling, and model ensembling yielded only marginal gains, reinforcing the idea that the core challenge is the propagation of errors in discrete outputs, a limitation that even substantial data scaling cannot fully resolve.

Researchers are increasingly turning to machine learning to tackle the complex task of optimizing quantum circuits, specifically minimizing the count of expensive T-gates crucial for fault-tolerant computation. While established classical techniques, rooted in ZX-calculus, matroid partitioning, and phase polynomial representations, remain effective, they are fundamentally limited by their reliance on manually discovered rules. Mehdi Saeedi, Eddie Richter, and Paul Hartke investigated whether a neural network could learn circuit optimization rules directly from data, a question central to advancing beyond these established methods. Their approach utilizes a 44.8 million-parameter encoder-decoder transformer with a structured tokenization scheme, where “every token carries circuit-semantic information.” This architecture demonstrated considerable success when paired with classical optimization for parameterized circuits, achieving a median fidelity of 1.000 on circuits with 3-6 qubits. The model, despite learning valid syntax and accurate T-count statistics, exhibited a phenomenon termed autoregressive drift, where early-token divergence cascades irrecoverably through left-to-right decoding. Further investigation revealed that scaling training data by 2.5 times pushed exact-match rates to 39.5%, but the underlying degradation with target length persisted, highlighting the fundamental challenge of maintaining discrete accuracy during autoregressive generation.

Researchers are pioneering a method of representing quantum circuits as sequences of discrete tokens, a technique central to their work on automated circuit optimization. Unlike earlier machine learning approaches that treat quantum assembly language as simple text, this system employs where “every token carries circuit-semantic information,” according to the team’s published findings. This means each token directly corresponds to a meaningful element of the circuit, a specific gate type or qubit, rather than merely a syntactic symbol. The architecture utilizes a 44.8 million-parameter encoder-decoder transformer, designed to learn and apply optimization rules directly from data. Inference-time strategies raised exact-match rates from 7% to 22.5%, demonstrating that the way the model is used can significantly impact accuracy. While scaling training data by 2.5 times pushes rates to 39.5%, the underlying degradation with target length persists.