AI Steers Towards Fully Autonomous Driving, Overcoming Complex Real-World Obstacles

Automated driving holds immense potential, yet achieving truly fully autonomous capabilities remains a significant hurdle due to the complexities of real-world environments. Lars Ullrich, Michael Buchholz, and Klaus Dietmayer, from the Friedrich-Alexander-Universität Erlangen-Nürnberg and Ulm University respectively, alongside Knut Graichen et al., present a comprehensive analysis of the current state of artificial intelligence in autonomous driving. Their work demonstrates AI’s capacity to surpass traditional methods in handling complexity and achieving greater autonomy, while simultaneously highlighting critical questions regarding robustness and generalisability. This research is significant because it not only identifies existing limitations but also maps out foreseeable technological advancements and the crucial research needs to propel the field towards genuinely driverless vehicles.

Current limitations and artificial intelligence integration in automated driving systems present unique challenges

Scientists have demonstrated significant advancements in automated driving (AD) through the application of artificial intelligence (AI), surpassing the capabilities of classical approaches in handling complex, real-world scenarios. This research, published in IEEE Access on 29 January 2026, meticulously analyses the current state of AD, identifies existing limitations, and explores foreseeable technological possibilities to pave the way for fully autonomous vehicles.
The team achieved a critical examination of AD systems, focusing on limitations hindering the transition to full autonomy, while simultaneously outlining necessary steps and emerging technologies, particularly AI, to overcome these hurdles. The study reveals that while modular, service-oriented software architectures currently dominate the AD landscape, incorporating AI into individual sub-modules like perception and planning presents challenges regarding safety and explainability.

Researchers conducted a qualitative, knowledge-driven literature review combined with forward-looking perspectives to address prospective opportunities, challenges, and emerging needs in the field. This approach allows for a comprehensive understanding of the complexities involved in achieving fully autonomous driving and highlights the need for adaptable systems capable of operating in dynamic environments.

This work establishes the increasing importance of AI in highly automated vehicles, particularly in managing the complexities of decision-making within unpredictable environments and intricate interactions between technical and non-technical participants. The research further explores promising new learning techniques, such as zero-shot, one-shot, few-shot, and meta-learning, which are crucial for generalization and real-world application of AI in autonomous systems.

Moreover, the potential of foundation models (FMs) in areas like language and vision, and their application to scenario engineering, are investigated, alongside the challenges they present. Experiments show that reconsidering the current state of AD in light of these advancements is essential for achieving fully autonomous driving, especially considering the rapid evolution of technological developments.

The research, accomplished within the “AUTOtech.agil” project (FKZ 01IS22088Y, FKZ 01IS22088W) and financially supported by the Federal Ministry of Education and Research of Germany, contributes by critically examining AD, analysing the steps towards higher autonomy, and extending the analysis of challenges associated with scalable deployment. This detailed analysis opens up new research questions and ultimately aims to outline the current status and limitations of AD systems, emphasising the need for adaptation and scalability.

Identifying architectural characteristics of current automated driving systems is crucial for development

Researchers conducted a comprehensive analysis of automated driving (AD) to identify challenges and opportunities concerning autonomous functionalities. The study employed a qualitative, knowledge-driven literature review combined with forward-looking perspectives to address prospective opportunities, challenges, and emerging needs within the field.

This methodology enabled the team to critically examine the current state of AD, focusing on limitations hindering progress towards fully autonomous driving. The work details how current AD systems typically utilise a modular, service-oriented software architecture, illustrated by a core function chain operating between sensors and actuators.

Individual modules encapsulate functionalities like perception and planning, implemented through numerous sub-modules or services. Perception modules, for example, collect data from lidar, radar, and cameras, alongside pre-processed vehicle state estimation, to generate environment models consisting of free space and occupancy maps.

This architecture manages complexity by logically separating functions, but introduces challenges regarding safety and explainability as AI integration increases. Scientists investigated the state-of-the-art in AI usage within these sub-modules, noting the significant contribution of publicly available datasets and associated challenges in areas such as perception, object detection, object tracking, and planning.

AI methods are increasingly applied to predict the behaviour of other road users, demonstrating superior accuracy compared to traditional techniques. However, the research highlights a critical tension between improved performance and the lack of validation methods, which elevates the risk of catastrophic consequences when human oversight is removed.

The study also explored emerging learning techniques, including zero-shot, one-shot, few-shot, and meta-learning, recognising their relevance for generalization, a crucial factor for real-world AI applications. Furthermore, the potential of foundation models in areas like language and vision, and their application in scenario engineering, were analysed alongside the associated challenges.

Scene awareness techniques across automated driving autonomy levels vary significantly

Scientists are investigating automated driving (AD) and its transition to full autonomy, acknowledging the complexities of real-world environments. Research demonstrates that artificial intelligence can surpass classical approaches in handling these complexities and achieving higher levels of autonomy.

The team analysed the current state of AD, outlining limitations and identifying potential technological advancements to reconsider fully autonomous driving. Experiments revealed diverse approaches to scene awareness (SA) across various AD stack levels. Vanilla E2E autonomy methods utilise E2E-CNN’16 with CNNs at Level 1 SA, while DRL-Framework’17 employs latent input vectors and spatial features.

CIL’18 and CILRS’19 leverage joint input representations and latent perception states, respectively, both at Level 1 and 2 SA. UD-CIL’20 further refines this by utilising intermediate learned perception features, also at Levels 1 and 2. Measurements confirm that methods like MaRLn’20 and DeepImitative’20 explore reinforcement learning and latent representations of map features, past RNNs, and future MLPs to predict affordances and states.

LSD’20 and LBC’20 incorporate context-embeddings and teacher-guided policy learning, while WOR’21 focuses on learned policies. Roach’21 achieves BEV representation and measurement vectors with IL-agent and RL-coach integration. Tests prove that TCP’22 uses image and measurement features for trajectory-guided control prediction.

Data shows that UrbanDriver’22 employs vectorized representations and a differentiable traffic simulator, while TRAVL’22 utilises BEV tensors and learned policies. ThinkTwice’23 leverages BEV features and compact environment/mission vectors for prediction. Hydra-MDP++’24 integrates image and lidar tokens with a teacher-student model.

Results demonstrate that S2P autonomy, exemplified by ALVINN’88, relies on neural networks at all SA levels. TransFuser’22 incorporates auxiliary maps, depth, and bounding boxes with semantic segmentation features. The study recorded that PlanT’22 uses input and embedding tokens, while DriveAdapter’23 employs BEV features and a teacher-student model.

DriveMLM’23 integrates image and cloud token embeddings with LLM-based reasoning. Drive-GPT4’24 and DriveVLM’24 utilise question-answering and chain-of-thought descriptions for reasoning and prediction. V2X-VLM’24 focuses on scene description prompting and VLM-based interpretation. Alphamayo-R1’25, a VLA model, integrates Chain of Causation reasoning with the Cosmos-Reason backbone, representing a step towards improved reasonability.

Validating artificial intelligence for safe and reliable autonomous systems is crucial for public trust

Scientists are increasingly focused on automated driving and its potential to revolutionise transportation systems. This research analyses the current state of automated driving, identifying limitations and exploring technological possibilities to achieve fully autonomous functionality. The study highlights a predominant shift towards modular, service-oriented software architectures incorporating artificial intelligence subsystems for tasks like perception, planning, and object tracking.

Current automated driving systems largely rely on AI to improve accuracy in areas such as object detection and prediction of other road users, surpassing the performance of traditional methods. However, the authors emphasise a critical hurdle: validating the reliability of these AI components to guarantee safety when human oversight is removed.

Ensuring error detection and prevention is paramount, as the lack of robust validation poses a risk of catastrophic consequences. Acknowledging the challenges of complete error elimination, the authors suggest a need for comprehensive monitoring systems capable of intervening when AI components malfunction.

Future research should prioritise developing and validating these safety mechanisms alongside continued advancements in AI performance. This work contributes to a more nuanced understanding of the path towards fully autonomous driving, emphasising that technological progress must be coupled with rigorous safety assurance to realise its full potential.