AI Safety Boosted As New System Spots Unusual Data with 1% Error

Researchers are increasingly focused on out-of-distribution (OOD) detection as a crucial step towards deploying reliable and safe artificial intelligence systems. Amey P. Pasarkar from the Lewis-Sigler Institute For Integrative Genomics, Princeton University, and Adji Bousso Dieng from the Department of Computer Science, Princeton University, alongside et al., present a novel approach that moves beyond traditional methods reliant on model confidence or likelihood estimates. Their work introduces the Vendi Novelty Score (VNS), a detector formulated from a diversity perspective, quantifying novelty by measuring how a test sample alters the diversity of the in-distribution feature set without requiring complex density modelling. This research is significant because VNS not only achieves state-of-the-art performance across image classification tasks but also maintains this level of accuracy using a remarkably small fraction of the training data, offering a practical solution for resource-limited applications.

The research demonstrates state-of-the-art OOD detection performance across multiple image classification benchmarks and network architectures, representing a significant advancement in the field.

Current OOD detection methods often depend on model confidence scores or likelihood estimates, which can be unreliable when distributional assumptions are not met. This study proposes a third paradigm, moving beyond these limitations by focusing on the diversity of in-distribution data. VNS builds upon Vendi Scores, a family of similarity-based diversity metrics, to assess novelty in a non-parametric manner.

By quantifying the increase in diversity caused by a test sample, VNS provides a clear indication of whether the input is genuinely novel or simply a variation of known data. This approach naturally integrates both local and global novelty signals, enhancing its accuracy and adaptability. Remarkably, the VNS maintains its high performance even when utilising only 1% of the training data.

This efficiency is crucial for deployment in resource-constrained environments, such as mobile devices or edge computing systems. The research highlights a reduction in the False Positive Rate to 6.9% when using VNS, a considerable improvement over a previously achieved rate of 22.0% obtained by incorporating prediction probabilities.

This substantial decrease demonstrates the effectiveness of the diversity-based approach in distinguishing between in-distribution and out-of-distribution samples. This work moves beyond reliance on model confidence or likelihood estimates by formulating OOD detection as a diversity problem, avoiding restrictive distributional assumptions.

The core of the method lies in calculating how much a test sample increases the Vendi Score of the existing in-distribution feature set, establishing a principled measure of novelty. Experiments involved multiple image classification benchmarks, including CIFAR-10, CIFAR-100, and ImageNet-1K, alongside various network architectures such as ResNet, ViT, and Swin-T.

A key methodological innovation was the aggregation of class-conditional novelty scores using the model’s prediction probabilities. This refinement significantly improved the separation between in-distribution and out-of-distribution samples, resulting in a reduction in the False Positive Rate to 6.9 percent.

This represents a substantial improvement over a previously achieved rate of 22.0 percent obtained by incorporating prediction probabilities alone. This represents a substantial improvement over a previously attained rate of 22.0% when utilising prediction probabilities. The research introduces a diversity-based paradigm for OOD detection, formulating it around the concept of quantifying how much a test sample alters the diversity of the in-distribution feature set.

VNS operates without requiring density modelling and naturally integrates both class-conditional and dataset-level novelty signals. The study demonstrates that VNS is a linear-time, non-parametric detector, enabling deployment even in environments with limited memory or access. Performance was evaluated across multiple image classification benchmarks and network architectures, consistently yielding state-of-the-art OOD detection results.

Specifically, the work details a method for approximating changes in the largest eigenvalue of a density matrix with an error of O(1/N 2 ), contributing to the computational efficiency of the VNS approach. The core of VNS lies in its ability to compute novelty scores efficiently. Class-conditional novelty contributions are calculated using a rank-1 approximation, reducing computational demands while maintaining accuracy.

A probability-weighted aggregation scheme, incorporating the top K classes with the largest predicted probabilities, further refines the novelty assessment. The global novelty component, modelled through a first-order approximation of the effect of a test sample on the overall dataset diversity, is then integrated with the local novelty signals.

This research highlights the potential for improved OOD detection in practical applications. By achieving a False Positive Rate of 6.9%, VNS offers a significant step towards safer and more reliable system deployment. The authors acknowledge limitations including the use of tunable hyperparameters selected with a validation set of noise images, which may not fully represent real-world deployment scenarios, and the reliance on the cosine kernel for similarity measurement. Future research will focus on adapting the method to alternative kernels and extending its application beyond image classification to other areas of machine learning and the natural sciences.