Junjie Luo of RIKEN Center for Quantum Computing and Yasunari Suzuki have developed DEMVal, a testing method that uncovered six defects in detector error models produced by quantum error correction compilers, all of which were confirmed and fixed by the developers at NVIDIA, Riverlane and Microsoft. These defects, present in NVIDIA’s CUDA-Q QEC, Riverlane’s Deltakit and the deq component of Microsoft’s QDK-EC, silently degrade logical error rates because decoders consume the models without verification.
Unlike existing tests relying on output sampling, DEMVal validates models exactly against a reconstructed reference, achieving sensitivity independent of physical error rates. To their knowledge, DEMVal is the first systematic testing method for the error-model semantics of QEC compilers.
DEMVal: Validating Detector Error Models in QEC Compilers
DEMVal, a new testing method, establishes an exact comparison between detector error models (DEMs) generated by quantum error correction (QEC) compilers and independently derived references, bypassing limitations of output-sampling techniques. This precise matching of errors, identified by the measurements they affect, and their associated rates allows for sensitivity independent of physical error rates, revealing defects undetectable through statistical means. The method’s core innovation lies in its ability to validate models exactly, rather than relying on probabilistic assessments of circuit performance.
The approach constructs a reference DEM in one of three ways, each targeting a specific aspect of the QEC compilation pipeline, and then compares the compiler-emitted model against it. This comparison isn’t merely statistical; it demands that rates of matched errors align within a defined numerical tolerance. These were not crashes or output errors, but subtle inaccuracies in how the compilers modeled error probabilities and their impact on quantum systems.
All six defects were subsequently confirmed by the developers and either corrected or slated for integration into future releases. Sampling methods could have only exposed four of these defects, those that drop, merge, or misplace error mechanisms, and even then, would have required millions of samples per experiment to achieve statistical significance. A complete model, as defined by Stim, is a list of lines detailing each error mechanism, its probability and the detectors and logical observables it affects.
DEMVal uses this format to facilitate precise comparison between the compiler’s output and the reconstructed reference. The researchers emphasize that DEMVal validates compilers individually against an independently derived reference, rather than attempting to establish a single, universal validation standard. This approach, they argue, is important for identifying subtle semantic errors that might otherwise go unnoticed.
While analytical moment tests would have missed the two rate-level defects, the exact model comparison employed by DEMVal successfully detected all six with no samples required. This highlights the method’s superior sensitivity and its potential to significantly improve the reliability of QEC compilers, a critical step towards realizing fault-tolerant quantum computing.
Silent Defects Degrade Logical Error Rates in Quantum Compilers
The discovery highlights a critical vulnerability in quantum computing software development: errors in how noise is modeled can subtly undermine the very error correction they are meant to enable. Existing methods for verifying quantum software typically focus on whether a program crashes or produces statistically correct outputs, but these approaches are insufficient for identifying defects that only distort error rates. This allows for detection of even minute discrepancies in the described noise characteristics.
DEMVal’s sensitivity stems from its approach to validation. It does not rely on observing the physical execution of a circuit, but instead compares the description of errors. A compiler’s detector error model lists the probability of each error mechanism and the measurements it affects, and DEMVal verifies this list against a ground truth established through alternative means.
DEMVal’s Three Construction Methods for Comprehensive Testing
DEMVal employs three distinct construction methods to build a reference model against which quantum error correction (QEC) compiler outputs are rigorously tested, each targeting different stages of the compilation pipeline. These constructions, labeled A, B, and C, differ in their trusted inputs, allowing for comprehensive coverage of potential defects from the initial specification to the final emitted model. Construction A relies on the source program itself as a trusted input, while construction B uses the circuit generated in an earlier stage, and construction C depends on an artifact produced by the compiler.
This tiered approach ensures that errors introduced at any point in the process are detectable. The validation harness central to DEMVal operates by comparing the emitted model from the compiler with the independently derived reference model, calculating a maximum divergence value. A compilation is flagged as inconsistent if the divergence exceeds this tolerance, indicating a semantic defect.
The oracle employed possesses three key properties, ensuring a precise and reliable assessment of model correctness. The study focused on three research questions: the effectiveness and precision of DEMVal in detecting real defects, the necessity of an exact comparison method versus statistical sampling, and the differing testing scopes of the three reference constructions. The location of each defect aligned with the predicted testing scope of the construction used to detect it, demonstrating the design’s effectiveness.
The study also revealed that the six defects fell within the testing scope of their respective constructions, and no single construction could identify all of them, reinforcing the value of the multi-faceted approach. The researchers note that the defects themselves are attributable to the tools, even if they stem from an artifact used in the reference model. This is because the exporter produced the artifact that the reference followed.
Limitations of Existing Quantum Software Testing Techniques
Existing quantum software testing techniques struggle to identify defects that subtly distort rates within detector error models (DEMs). This means a flawed DEM can silently degrade the logical error rate without triggering a detectable failure in traditional testing scenarios. Applied to CUDA-Q QEC, Riverlane’s Deltakit and the deq component of QDK-EC, DEMVal found six defects, all confirmed by the developers and fixed.
Six Confirmed Defects Found in QEC Compiler Error Models
Each of these defects was subsequently confirmed by the respective development teams and has since been addressed, demonstrating the method’s practical utility in improving compiler reliability. The discovery highlights vulnerabilities previously undetectable by standard testing techniques, which often rely on statistical analysis of sampled outputs. This approach bypasses the limitations of sampling-based tests, which struggle to resolve subtle rate-level defects at the low error rates where quantum error correction operates effectively.
Construction A rebuilds the model from the original code and noise model, while Construction B cross-checks artifacts generated during a single compilation process. Construction C re-applies a published noise model to the circuit exported by the compiler, providing a further layer of validation. These three construction methods, rather than being merely implementation details, function as a design dimension of the testing process, allowing for targeted identification of specific defect types.
The six defects uncovered by DEMVal fell into complementary testing scopes, demonstrating the value of this multi-faceted approach. Three of the defects resulted in models that were fundamentally incorrect regardless of the noise semantics, with entire classes of error symptoms either missing or incorrectly merged. The researchers distinguish their work from existing techniques focused on reconstructing noise models from experimental data, noting that their approach flows in the opposite direction, from specification to model.
They also highlight that their method complements, rather than replaces, existing decoder and error correction protocol evaluations. The team’s contribution centers on constructing the reference model and reporting divergences, providing an important step toward more robust and dependable quantum computing systems. While substantial work exists evaluating decoders and error correction protocols, DEMVal offers a unique perspective by focusing on the integrity of the detector error model itself.
The study demonstrates that even analytically optimal moment tests, designed to maximize the information gained from limited samples, are unable to detect rate-level defects at realistic error rates. “DEMVal reported divergences in all three subjects,” the researchers state, “and inspecting the inconsistent experiments yielded six distinct defects.” The implications of this work extend beyond the specific compilers tested, suggesting a broader need for more rigorous testing methodologies in the development of quantum software. The researchers’ framework, applied to CUDA-Q QEC, Deltakit and QDK-EC’s deq, provides a pathway toward more trustworthy quantum error correction, ultimately contributing to the advancement of the field.
👉 More information
🗞 A Testing Method for Detector Error Models in Quantum Error Correction Compilers
✍️ Junjie Luo and Yasunari Suzuki (RIKEN Center for Quantum Computing, Wako, Saitama, Japan)
🧠 ArXiv: https://arxiv.org/abs/2611.20501




See today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals.
