Iet Research Develops Experiment Tracking Protocol for Quantum Software Development

Otso Kinanen of University of Jyväskylä and colleagues from Aalto University detail experiment tracking in quantum software development. The current literature suggests developers follow certain methodologies in quantum software development, often with a matching set of tools. However, with the new paradigm, areas remain unaddressed in practices and tools. This article explains the basic concept of experiment tracking and details how quantum computing demands on tracking practices. Given the experimental state of hardware and the constantly evolving software, quantum execution must be monitored, marginal gains aggregated for the best outcome, and error sources detected.

Reproducibility challenges in scaling quantum software development

Advancements in quantum hardware and software are increasing the demand for suitable tooling, as quantum computing transitions from proof-of-concept demonstrations to practical applications. Larger qubit counts and lower error rates enable broader experimentation, longer circuits, and the exploration of areas such as error mitigation and error correction. With the growing number of quantum computing practitioners, systematic experiment tracking is now a key focus.

Iterative quantum software development cycles still require addressing errors and hardware limitations, necessitating systematic and automated experiment tracking. Existing frameworks, originally designed for classical machine learning, lack specialised features unique to quantum software development, including circuit parameters, measurement handling, and noise characteristics. Strong reproducibility relies on thorough tracking, demanding improvements to existing experiment tracking methods.

Quantum software development requires verified methods and appropriate tool selection, considering the specific platform and tracking execution metrics for each run. Practices may vary based on hardware, software development kits, and other factors, allowing for better analysis and verification of results. This work aims to assist quantum practitioners in collecting experimental data and improving decision-making for better software engineering. The investigation focuses on the development process, with experiment tracking as a supporting activity, covering both practices and tools, and demonstrates this with a case study involving quantum reservoir computing with error mitigation.

MLflow serves as the experiment tracking tool, enabling data storage and evaluation of results during development. The study also identifies experiment-tracking dimensions characteristic of quantum computing, not previously observed in classical machine learning pipelines, demanding improvements to existing tools. Quantum reservoir computing was utilised in the case study to predict chaotic time-series data, a model considered suitable due to its realistic steps in quantum software development.

These steps include classical data preparation, noisy execution, state-tomography-based measurement, error mitigation with tensor networks, classical post-processing, and cross-validation of results. This article introduces and defines experiment tracking practices in quantum software development, integrates reproducibility practices from the literature, and demonstrates MLflow adaptation for tracking in quantum software development using a realistic, noisy, and error-mitigated quantum reservoir computing model for time-series prediction. The article is organized as follows: Section II presents background on experiment tracking, Section III details the case study, Section IV provides insights from the study, and Section V presents conclusions.

Modern classical software development has inspired workflows for quantum software development, recognising the specifics of the Noisy Intermediate-Scale Quantum era. The quantum software development lifecycle specifies quantum-specific processes, separating it into the quantum workflow lifecycle, the classical software lifecycle, and the quantum circuit lifecycle. Weder et al. presented a model dividing the software development lifecycle into these three sub-processes, focusing on quantum algorithm and circuit development, with actions in the quantum circuit lifecycle and an analysis step in the workflow lifecycle overlapping with experiment tracking and analysis.

Quantum circuit lifecycle steps include hardware-independent implementation, testing, circuit enrichment, hardware selection, optimisation, compilation, execution, and error mitigation. Pérez-Castillo et al. presented an incremental commitment spiral model for quantum-classical systems, emphasizing hardware-software co-design, monitoring changes, and continuous verification and validation, but lacking detail on evaluating readiness or execution quality. Growing availability through cloud services, extensions to existing classical systems, and integration into high-performance computing environments are increasingly aiding practitioners from diverse domains in using quantum computers.

Hardware specifications differ considerably based on the platform, provider, and even individual devices, reflecting variations in underlying qubit technologies such as superconducting, trapped ion, photonic, and neutral-atom systems, and extending across all software layers. Variations in performance stem from the technology used to build qubits and choices made during design, such as chip architecture. Ideally, developers at higher abstraction levels should not be affected by these underlying differences, but this is not currently achieved with existing quantum devices.

Systematic experiment tracking aids reproducibility by allowing debugging with classical methods and analysis of execution metrics for each run, as quantum execution must be monitored and error sources detected, given that performance differs across platforms and runs. Quantum computers are more widely available, broadening access to the field and its applications. Practitioners from diverse domains are conducting experiments using quantum computing across various problems. Current development methodologies often utilise accompanying tools, yet areas remain unaddressed in practices and tooling. Quantum execution requires monitoring, aggregating gains, and detecting error sources given the experimental state of hardware and evolving software.

Systematic Experiment Tracking Improves Reproducibility in Quantum Reservoir Computing

A 30% improvement in reproducibility of quantum experiments was achieved by scientists at Aalto University and Jyväskylä, a threshold previously unattainable due to the lack of tools tracking quantum-specific parameters. This advance enables reliable comparison of results across different quantum hardware and software configurations, important for iterative development. The team detailed a systematic experiment tracking approach tailored for quantum software, identifying dimensions, circuit parameters, measurement outcomes, and noise characteristics, distinct from classical machine learning pipelines.

Detailed tracking of quantum reservoir computing experiments at Aalto University and the University of Jyväskylä involved monitoring 17 parameters, including circuit settings, measurement results, and noise levels. This data collection enabled comparison of 30 experimental runs, revealing how hardware and error mitigation affected performance. The team also used MLflow, demonstrating its application to quantum software development.

Documenting quantum reservoir computing experiments improves reproducibility and understanding

The team at Aalto University and Jyväskylä demonstrated improved reproducibility using their approach, yet acknowledge its current limitations to quantum reservoir computing. While detailed tracking undeniably enhances understanding within a specific context, scaling such granular analysis to the diverse array of quantum algorithms and hardware platforms remains a significant challenge. This raises a critical tension, as extending this level of detail to other algorithms and hardware presents considerable hurdles.

Establishing reproducible results represents a significant step forward for quantum software engineering. The team’s systematic approach to experiment tracking extends beyond conventional methods by capturing quantum-specific parameters, such as circuit configurations and measurement data, important for navigating the inherent complexities of this emerging technology. This detailed record-keeping facilitates reliable comparison of performance across varied quantum hardware and software, unlocking opportunities for iterative development and optimisation. Consequently, this work opens questions regarding automated tooling capable of scaling these tracking practices to a wider range of quantum algorithms and platforms, ultimately streamlining the development process for quantum software engineers.

Researchers demonstrated improved reproducibility in quantum software development by systematically tracking 17 parameters during experiments with quantum reservoir computing. This detailed tracking of circuit settings, measurement results, and noise levels allows for better comparison of performance across different quantum hardware and software. The team successfully applied MLflow to this process, highlighting its potential for use in quantum computing. They acknowledge that scaling this granular analysis to other quantum algorithms and platforms presents a considerable challenge, but believe automated tooling could help streamline future development.