AI Health Models Leak Patient Data Despite Privacy Safeguards, Research Reveals

Scientists are increasingly focused on developing clinical prediction models that simultaneously guarantee predictive accuracy, interpretability, and patient privacy. José Ramón Pareja Monturiol, Juliette Sinnott, and Roger G. Melko, alongside colleagues from the Universidad Complutense de Madrid, University of Waterloo, and the Perimeter Institute for Theoretical Physics, demonstrate a significant vulnerability in current approaches such as logistic regression and shallow neural networks to privacy attacks that reveal training data. Their research introduces a novel quantum-inspired defence, utilising tensor trains to obfuscate model parameters without sacrificing predictive performance or interpretability. This tensorization technique not only reduces the risk of data leakage, achieving levels comparable to differential privacy, but also enhances interpretability by enabling efficient computation of key statistical distributions, establishing a practical pathway towards truly private and effective clinical prediction models.

Tensor train decomposition for privacy-preserving clinical prediction

Researchers have developed a new approach to safeguarding sensitive medical data used in machine learning models while simultaneously enhancing interpretability and maintaining predictive accuracy. This work addresses a critical challenge in clinical prediction, where models like logistic regression offer transparency but are vulnerable to privacy breaches, and more complex neural networks, while potentially more accurate, lack inherent interpretability.
The study introduces a quantum-inspired defense mechanism based on tensorizing discretized models into tensor trains, effectively obscuring model parameters without compromising performance. Empirical evaluations demonstrate that this tensorization process significantly reduces the risk of privacy attacks, diminishing white-box attacks to random guessing and achieving black-box protection comparable to differential privacy.

The research begins by highlighting the inherent privacy risks associated with machine learning in clinical settings, where models trained on patient data can inadvertently reveal individual information. Investigations reveal that both logistic regression and shallow neural networks leak significant training-set information, with logistic regression proving particularly susceptible under white-box access conditions.

Furthermore, standard practices such as cross-validation unexpectedly exacerbate these vulnerabilities, enabling accurate identification of training data even through public web interfaces. To counter these risks, the team proposes a novel defense rooted in tensor network models, specifically tensor trains, building upon recent advances in tensorizing pre-trained machine learning models.

This quantum-inspired technique involves transforming clinical models into a tensor train format, which fully obfuscates parameters while preserving accuracy. The researchers applied this method to LORIS, a publicly available logistic regression model for immunotherapy response prediction, and to comparable neural network models trained for the same task.

Results indicate that tensorization effectively degrades attack performance across all access levels, offering a practical foundation for private, interpretable, and effective clinical prediction. Importantly, the tensor train models not only maintain the interpretability of logistic regression but also extend it to neural networks, enabling efficient computation of marginal and conditional distributions for enhanced feature sensitivity analysis.

Membership inference and tensorisation for clinical prediction model privacy

A membership inference attack underpinned the assessment of privacy risks associated with clinical prediction models. Researchers designed this attack under both black-box and white-box access conditions to identify training datasets used for model creation. The methodology involved training multiple shadow models, each with varied hyperparameters and datasets, to establish a baseline for comparison.

An adversarial meta-classifier then predicted which public datasets comprised the original model’s training set, effectively revealing training-set membership. To evaluate the proposed defense, logistic regression (LR) and shallow neural network (NN) models were trained on the same immunotherapy response prediction task as the publicly available LORIS model.

These models were then subjected to tensorization, a quantum-inspired technique employing tensor trains to obfuscate parameters. Discretized output scores were integrated into the tensorization process to further enhance black-box privacy, controlling the granularity of output to modulate privacy protection analogous to noise calibration in Differential Privacy.

Performance was compared against models protected with Differential Privacy, assessing predictive accuracy alongside privacy leakage. The study leveraged LORIS, hosted on a U.S. government website, as a primary test case, enabling attacks via its public web interface. Crucially, the research demonstrated that tensorization reduced white-box attacks to random guessing and provided comparable black-box protection to Differential Privacy, all while maintaining accuracy levels similar to the unprotected models. Furthermore, the work revealed that cross-validation, a common practice in LR models like LORIS, can significantly compromise privacy, allowing accurate training-set identification even with limited access.

Tensor train parameter obfuscation mitigates privacy risks in immunotherapy prediction models

Logistic regression (LR) models proved particularly vulnerable to privacy attacks in white-box scenarios, leaking significant training-set information during empirical assessments. Investigations into both LR and shallow neural network (NN) models, trained for immunotherapy response prediction, revealed that cross-validation practices in LRs exacerbate these privacy risks.

To address these vulnerabilities, a quantum-inspired defense was proposed, utilising tensorizing discretized models into tensor trains (TTs), which fully obfuscates parameters while preserving accuracy. White-box attacks were reduced to random guessing through this tensorization, and black-box attacks experienced comparable degradation to that achieved by Differential Privacy.

Tensor train models retain the interpretability of logistic regression and extend it through efficient computation of marginal and conditional distributions, also enabling this higher level of interpretability for neural networks. Results demonstrate that tensorization is widely applicable and establishes a practical foundation for private, interpretable, and effective clinical prediction.

The study assessed privacy risks using a membership inference attack under both black-box and white-box access, employing a shadow model approach with varied hyperparameters and datasets. Analysis of the publicly available LORIS model, an LR model for immunotherapy response prediction, alongside shallow NNs, showed that tensorizing the models degraded attack performance across all access levels.

Specifically, white-box attacks were reduced to random guessing, while black-box protection matched that of Differential Privacy, all while maintaining predictive accuracy close to the unprotected models. Discretization step sizes for output scores provided control over privacy protection, analogous to tuning Differential Privacy with calibrated noise.

Furthermore, the research demonstrated that cross-validation, when used to deploy averaged models, severely compromises privacy, enabling accurate training-set identification even from public web interface access. TT approximations preserve key properties of LORIS, such as response monotonicity, and enhance interpretability through efficient computation of marginals and conditionals, supporting feature-sensitivity analysis and enabling the construction of cancer-type-specific models without retraining. The tensorization process is general and can be applied post-training as a practical strategy for privacy-preserving, interpretable, and effective models in sensitive clinical domains.

Tensor train decomposition safeguards clinical machine learning model privacy and interpretability

Researchers have developed a novel defence against privacy vulnerabilities in machine learning models used in clinical settings. This approach, based on tensorizing discretized models into tensor trains, effectively obscures model parameters while maintaining predictive accuracy. Investigations revealed that both logistic regression and shallow neural networks leak significant training data information, with logistic regression proving particularly susceptible to attacks when full access to the model is available.

Furthermore, standard practices like cross-validation can inadvertently increase these privacy risks. The proposed tensor train methodology mitigates these vulnerabilities, reducing the effectiveness of both white-box and black-box attacks to levels comparable with differential privacy. Importantly, this technique preserves the interpretability inherent in logistic regression and extends it to neural networks through efficient computation of statistical distributions.

This allows for a greater understanding of model predictions and facilitates the development of private, interpretable, and effective clinical prediction tools. The authors acknowledge that achieving strong privacy guarantees often involves a trade-off with model performance and potential exacerbation of group disparities.

Future research should focus on refining the application of tensorization to more complex models and datasets. Further investigation into the optimal balance between privacy, accuracy, and interpretability is also warranted. The demonstrated widespread applicability of this tensorization technique establishes a practical foundation for routinely incorporating privacy-preserving measures into sensitive domains like clinical prediction.