Researchers at MIT have developed a new technique to assess the reliability of massive deep-learning models called foundation models before they are deployed for specific tasks. These models, which serve as the backbone for powerful artificial intelligence tools like ChatGPT and DALL-E, can offer incorrect or misleading information, leading to serious consequences in safety-critical situations. The researchers’ approach involves comparing several large models and choosing the one that works best for a particular task by assessing the consistency of representations each model learns about the same test data point.
Led by senior author Navid Azizan, the team includes Young-Jin Park, Hao Wang, and Shervin Ardeshir from Netflix. The technique could be especially useful in healthcare settings where datasets may not be accessible due to privacy concerns. The researchers’ method allows users to quantify how reliable a representation model is for any given input data, enabling them to select the best one for their task.
Assessing the Reliability of General-Purpose AI Models
The development of massive deep-learning models, known as foundation models, has revolutionized the field of artificial intelligence. These models are pretrained using general data and can be adapted to specific tasks after training. However, assessing their reliability is a challenging task due to their abstract representations.
Measuring Consensus
Traditional machine-learning models are trained to perform a specific task and make concrete predictions based on input data. In contrast, foundation models generate an abstract representation of the input data point. To assess the reliability of these models, researchers have employed an ensemble approach by training several models that share many properties but are slightly different from one another.
The idea is to measure consensus among the models. If all the foundation models give consistent representations for any data in the dataset, then it can be said that the model is reliable. However, comparing abstract representations is a complex task.
Neighborhood Consistency
To solve this problem, researchers have used an idea called neighborhood consistency. They prepare a set of reliable reference points to test on the ensemble of models. Then, for each model, they investigate the reference points located near that model’s representation of the test point. By looking at the consistency of neighboring points, they can estimate the reliability of the models.
Aligning Representations
Foundation models map data points to a representation space, which can be thought of as a sphere. Each model maps similar data points to the same part of its sphere, so images of cats go in one place and images of dogs go in another. However, each model would map animals differently in its own sphere.
The researchers use the neighboring points like anchors to align those spheres so they can make the representations comparable. If a data point’s neighbors are consistent across multiple representations, then one should be confident about the reliability of the model’s output for that point.
Advantages and Limitations
When tested on a wide range of classification tasks, this approach was found to be much more consistent than baselines. It wasn’t tripped up by challenging test points that caused other methods to fail. Moreover, this approach can be used to assess reliability for any input data, so one could evaluate how well a model works for a particular type of individual.
However, one limitation comes from the fact that they must train an ensemble of foundation models, which is computationally expensive. In the future, researchers plan to find more efficient ways to build multiple models, perhaps by using small perturbations of a single model.
Importance and Future Directions
The topic of quantifying uncertainty at the representation level is increasingly important but challenging. This work is a promising step towards high-quality uncertainty quantifications for embedding models. In the future, researchers aim to extend this approach to enable it to scale to foundation-size models without requiring model-ensembling.
This research has significant implications for the development and deployment of reliable AI systems in various domains, including healthcare, finance, and transportation. As AI models become increasingly ubiquitous, ensuring their reliability is crucial for building trust and avoiding potential risks.
External Link: Click Here For More
