Researchers from Skoltech and Sberbank’s Center for Practical Artificial Intelligence have developed TOHA, a cost-efficient method for pinpointing unsupported responses in large language models. The TOpology-based HAllucination detector analyzes a model’s attention maps as graphs, identifying inconsistencies between source context and generated text without requiring extensive training data. Published in the proceedings of the A*-ranked Association for Computational Linguistics conference, ACL 2026, the approach offers a new way to assess AI reliability, as Aleksey Zaytsev of Skoltech explains.
TOHA Method Detects Hallucinations via Attention Map Topology
The TOHA method achieves detection accuracy comparable to SelfCheckGPT, yet requires significantly less computational power, a key advantage for real-world applications. This topology-based approach, short for TOpology-based HAllucination detector, moves beyond methods demanding extensive annotated data or repeated response generation by focusing on the internal workings of large language models. Researchers represent a model’s attention matrices as graphs, then calculate a characteristic called MTop-Div to quantify differences between the context and generated response structures; higher divergence values correlate with unsupported answers.
The technique’s efficiency stems from its reliance on internal signals rather than external training, requiring only a small set of examples for initial configuration before evaluating new responses. This targeted calibration of attention allows the model to adhere more closely to the provided context and independently correct inaccuracies, differing from methods that rely on brute-force computational approaches.
Sergey Ryabov, Senior Managing Director and Director of AI Transformation at Sberbank, explained, “The fight against AI hallucinations is entering a new stage,” highlighting the shift towards more resource-conscious detection strategies. The practical implications of this are considerable, particularly for industries where AI-driven errors carry significant risk. Aleksey Zaytsev, Associate Professor at Skoltech and Head of the joint Skoltech-Sberbank Laboratory, emphasized the importance of verifying AI responses in applied systems.
The team’s work demonstrates that extracting a reliability signal is possible through careful analysis of the model’s internal representation of text, the company says. The activation of specific topological features signals potential hallucinations without requiring the training of an additional detector model or repeated response generation. Implementation of the TOHA method is already underway within SIRIN, an open-source library developed by Sberbank’s Center for Practical Artificial Intelligence. This integration facilitates the development of corporate assistants and other solutions that use knowledge bases, offering a readily available tool for enhancing AI trustworthiness.
The library’s open-source nature allows for wider adoption and further refinement of the technique by the research community and industry professionals, potentially impacting sectors like banking, insurance, and legal services where accurate information is paramount. Researchers also suggest that studying the topological properties of attention graphs offers a new avenue for understanding the internal mechanisms of language models themselves, potentially leading to further improvements in AI reliability and performance.
When language models are used in applied systems, it is important not only to obtain a convincing-sounding answer but also to understand whether it is actually based on the information that was provided to the model.
Aleksey Zaytsev, Associate Professor at Skoltech and Head of the joint Skoltech-Sberbank Laboratory (LARSS)
See today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals.
