Personalised AI: Knowledge Graphs Enhance Long-Term Conversational Memory.

Researchers demonstrate improved personalised responses from large language models by integrating external memory in the form of knowledge graphs. These graphs, constructed and updated by the model itself, utilise novel hyperedge connections and maintain robustness even with temporally complex and contradictory dialogue data, as evidenced by performance on established benchmarks.

The pursuit of genuinely personalised artificial intelligence necessitates systems capable of retaining and effectively utilising an individual’s unique history, a challenge that extends beyond simply augmenting large language models (LLMs) with broader factual knowledge. Researchers are now exploring methods of external memory, specifically knowledge graphs, to provide LLMs with a persistent, evolving record of user interactions. A team comprising Mikhail Menschikov, Dmitry Evseev, Evgeny Burnaev and Nikita Semenov from Skoltech, alongside Ruslan Kostoev, Ilya Perepechkin, Ilnaz Salimov and Victoria Dochkina from Sberbank of Russia, and Petr Anokhin from AIRI, detail their work in ‘PersonalAI: Towards digital twins in the graph form’. They present an advanced knowledge graph architecture, incorporating novel hyperedge structures, designed to unify graph construction and knowledge extraction, and demonstrate its robustness through experiments on established question-answering benchmarks, including a modified DiaASQ dataset incorporating temporal and contradictory information.

Arize AI’s Memorize represents an advancement in question answering over lengthy documents, integrating external memory with large language models (LLMs). Current LLMs, despite improvements in retrieval augmented generation, exhibit limitations in retaining and utilising extensive personal or contextual information due to fixed context windows, the maximum amount of text an LLM can process at once. Memorize addresses this by constructing and maintaining a knowledge graph, a network of interconnected information, dynamically updated by the LLM itself, to serve as this external memory.

The system operates by dividing long documents into smaller segments, termed ‘chunks’, and converting these into vector embeddings. Vector embeddings are numerical representations capturing the semantic meaning of each chunk, allowing for efficient similarity comparisons. These embeddings are stored in a vector database, facilitating the retrieval of the most relevant information when a question is posed. The retrieved chunks are then assembled into a context, which the LLM utilises to formulate an answer. A key innovation lies in the graph’s structure, incorporating standard edges – representing direct relationships between concepts – alongside two types of hyperedges. Hyperedges allow a single edge to connect more than two nodes, enhancing the graph’s capacity to represent complex relationships and dependencies within the data.

Experiments conducted on established benchmarks – TriviaQA, HotpotQA, and DiaASQ – demonstrate the system’s effectiveness. The authors meticulously controlled context lengths during evaluation, ensuring a robust assessment of performance independent of varying input sizes. To further test the system’s capabilities, the DiaASQ benchmark was augmented with temporal parameters and contradictory statements, simulating more complex dialogue scenarios where information evolves or conflicts. Despite these modifications, the question-answering system maintained a consistent level of performance, demonstrating its ability to handle temporal dependencies and inconsistencies, which are crucial for applications requiring dynamic knowledge representation.

The research emphasises the importance of careful data preprocessing and the use of deterministic generation. Deterministic generation involves configuring the LLM with specific parameters, such as temperature and top-p sampling, to ensure reproducibility of results, mitigating the inherent randomness of LLM outputs. This ensures that consistent inputs yield consistent outputs, facilitating reliable evaluation and comparison of different system configurations.

Future work should investigate scaling the knowledge graph to accommodate even larger volumes of personal data and more complex relationships. Exploring alternative hyperedge structures and graph traversal algorithms could further optimise knowledge retrieval efficiency. Additionally, research into methods for automatically validating and correcting inconsistencies within the knowledge graph would enhance the system’s reliability and trustworthiness. Finally, evaluating the system’s performance in real-world scenarios, such as customer service or personalised education, would provide valuable insights into its practical applicability and potential impact.