The rapid expansion of Machine Learning (ML) has raised ethical and privacy concerns. To address these, a technique called Private Aggregation of Teacher Ensembles (PATE) was developed. This study is the first to apply PATE to an ensemble of quantum neural networks (QNN), marking a significant step towards ensuring privacy in quantum machine learning (QML) models. The study also explores the implementation of PATE with variational quantum circuits (VQC), demonstrating over 99% accuracy with significant privacy guarantees. This opens up new possibilities for the application of quantum computing in privacy-preserving machine learning.
Quantum Privacy Aggregation of Teacher Ensembles (QPATE) for Privacy-Preserving Quantum Machine Learning
Machine Learning (ML) has seen a rapid expansion in its utility over the last two decades, but this growth has also presented ethical challenges, particularly in terms of privacy. A technique known as Private Aggregation of Teacher Ensembles (PATE) was developed by Nicholas Papernot et al. to address these concerns. This study is the first to apply PATE to an ensemble of quantum neural networks (QNN), paving a new way for ensuring privacy in quantum machine learning (QML) models.
Privacy Concerns in Machine Learning
ML is being utilized in a wide range of applications and often raises privacy and ethical concerns. For instance, privacy leaking is a major concern as demonstrated in large language models like GPT trained with sensitive texts. Many applications deploy the model directly on the device, which would allow an adversary to use the model parameters directly to release private data. Differential Privacy (DP) seeks to address these privacy concerns through its privacy-loss framework. One of the newer techniques to ensure differential privacy is known as Privacy Aggregation of Teacher Ensembles (PATE), pioneered by Nicholas Papernot et al. in 2017.
PATE and Quantum Machine Learning
PATE provides strong privacy guarantees by training in a two-layer fashion. The training set is divided between N teacher models, who each train independently of each other. Then the teacher ensembles predict the labels of a disjoint training set through a noisy aggregation. The disjoint training set with newly aggregated noisy labels is used to train a student model. The student, never having access to the original training set or any teacher ensembles model parameters, is deployed into the application. With the recent advancements of quantum computing hardware, it is a natural next step to investigate Quantum Machine Learning (QML).
Differential Privacy in Machine Learning
Differential Privacy (DP) has manifested as the standard tool for gauging privacy loss. The notion of a privacy budget determines the amount of information that an adversary can extract. Information can be divided into two groups: general information and private information. DP puts limits on how much private information can be ascertained from querying a database or in the case of machine learning, a classifier.
Implementing PATE with Variational Quantum Circuits
Currently, there have been some investigations at the intersection of quantum computing and privacy-preserving machine learning, yet there is no research implementing PATE with variational quantum circuits (VQC). This study aims to implement an ensemble of hybrid quantum-classical classifiers and train them using privacy aggregation of teacher ensembles (PATE). After training, the student model will satisfy privacy loss limits. Classical classifiers with PATE training will be used as controls in the study. The study demonstrates that the privacy-preserving VQC on MNIST handwritten digits has over 99% accuracy with significant privacy guarantees.
Conclusion
The application of PATE to an ensemble of quantum neural networks (QNN) is a significant step towards ensuring privacy in quantum machine learning (QML) models. This approach provides strong privacy guarantees and opens up new possibilities for the application of quantum computing in privacy-preserving machine learning.
The article titled “Quantum Privacy Aggregation of Teacher Ensembles (QPATE) for Privacy-preserving Quantum Machine Learning” was published on January 14, 2024. The authors of this article are William H. Watkins, H. Wang, Su Gon Bae, H. Eric Tseng, Jiook Cha, Samuel Yen-Chi Chen, and Shinjae Yoo. The article was sourced from arXiv, a repository maintained by Cornell University. The article can be accessed through its DOI reference: https://doi.org/10.48550/arxiv.2401.07464.