Contextual Runtime Monitors Advance Safe AI-Based Control Ensembles for Autonomy

Researchers are tackling the critical challenge of ensuring safety in AI-based autonomous systems, where machine learning controllers can falter when faced with unfamiliar situations. Alejandro Luque-Cerpa, Mengyuan Wang, and Emil Carlsson, from Chalmers University of Technology, University of Gothenburg, and Sleep Cycle AB respectively, alongside Sanjit A. Seshia, Devdatt Dubhashi, and Hazem Torfah from Chalmers University of Technology and University of Gothenburg, present a new framework for learning context-aware runtime monitors. This work is significant because it moves beyond simply averaging controller outputs, instead focusing on intelligently selecting the best controller for specific operating conditions, offering both theoretical safety guarantees and improved performance in complex scenarios like autonomous driving.

This work is significant because it moves beyond simply averaging controller outputs, instead focusing on intelligently selecting the best controller for specific operating conditions, offering both theoretical safety guarantees and improved performance in complex scenarios like autonomous driving.

Context-aware monitoring for robust AI control is crucial

Machine-learning (ML) controllers are. e., that resorts to too many unnecessary. e., ones that can operate safely across most, if not all, contexts within this OD? As illustrated in Figure 2, they identify several possible scenarios along this. g., the ensemble of CNN-based controllers from their example above), the context corresponds to the environmental settings in which the system is deployed (e. g., weather, time of day, road features, etc. ), sometimes it could also include the system state, and the rewards are determined by the satisfaction of a system-level specification that defines the system’s safety requirements (e. g., avoiding lane. g., MDP in shielding) along with a conservative environment model? In contrast, their monitors do not make such assumptions, are defined solely over observable features, and do not assume direct monitorability of the specification? Similar argument holds for methods based on barrier certificates or safety filters? Lastly, there is a series of works that address the problem of predictive safety monitoring, e. g., in RL settings to predict the impact of actions on safety, and also those in adversarial settings? Such approaches can be adapted to their settings to predict?

Contextual Monitor Learning for Control Ensembles enables robust

Researchers addressed the challenge of declining accuracy in machine-learning controllers when deployed in unfamiliar environments, a critical safety concern in autonomous applications. If no controller meets the safety criteria, the monitor activates a fail-safe mechanism, guaranteeing safety at the expense of optimal performance. In these simulations, an autonomous vehicle was equipped with an ensemble of image-based controllers, each realised by convolutional neural networks trained on different datasets. This method achieves a balance between leveraging the strengths of individual controllers and ensuring safety by switching to a verified, albeit less optimal, control policy when necessary. The study pioneers a formalisation of the problem, defining safety constraints and enabling the learning of contextual monitors with statistical guarantees. The innovative approach allows for the exploitation of inherent biases in individual controllers, offering a significant advancement over traditional ensemble techniques that often reduce variance without fully utilising contextual specialisation.

Contextual monitoring enhances control ensemble performance by adapting

Experiments revealed that traditional ensemble methods, while improving robustness, often dilute the specialized strengths of individual controllers in varying operating contexts. Measurements confirm that this positional setting provides rich theoretical insights and establishes a foundation for more complex, state-based settings, although the extension to state-full contexts is reserved for future work. The work defines a monitoring system (MGS) consisting of n controllers managed by a contextual monitor, with a monitoring policy π mapping a context ξ from a context domain DVcont to a controller c from a controller set C. Tests prove that the framework’s safety is defined by finite trace specifications, ensuring system traces do not deviate from a defined set φ.

An example safety specification in an autonomous car scenario requires no lane invasions and maintaining a safe distance to other objects. The contextual monitor then decides, based on observed context, which controller is safest in terms of satisfying this specification. Scientists computed a policy π that approximates an optimal policy π*, measuring optimality in terms of regret, defined as the maximal difference between the loss suffered by the optimal controller and the loss of the computed policy, maximised over all contexts. The regret is quantified as max ξ∈DVcont LS(π(ξ),φ) −LS(π∗(ξ),φ), where LS represents the probability of satisfying the specification φ. The learning approach restricts monitors to modelling violation probability using logistic regression, assuming a vector θc such that Pr(Y= 1|c, ξ) = σ(θ⊤ cξ), where σ is the logistic function and Y is a Bernoulli variable indicating violation. The research delivers a learning algorithm based on contextual bandits, updating the vector θc with maximum-likelihood estimates after each round t.

Contextual monitoring for robust control ensembles is crucial

Scientists have developed a new framework for creating context-aware runtime monitors used in ensembles of controllers, particularly for cyber-physical systems. Machine-learning controllers are increasingly used in autonomous systems, but their performance can decrease when faced with unfamiliar environments. Traditional ensemble methods often dilute the strengths of individual controllers, whereas this research focuses on identifying and utilising contextual strengths instead. The authors acknowledge that while their monitors performed well in scenarios with contextual bias and when handling out-of-distribution data, performance was comparable to non-contextual ensembles when no bias was present and controllers covered the entire context space. Neural network-based monitors may require extensive simulations to achieve high performance and lack the statistical guarantees offered by their approach. Future work could explore the use of active learning strategies, as demonstrated by their comparison of active and passive learning methods for neural network monitors, potentially enhancing data collection efficiency and monitor performance.