Personaagent with GraphRAG Achieves 11.1% Personalization Gains Via Community-aware Knowledge Graphs for Large Language Models

The pursuit of truly personalized artificial intelligence takes a significant step forward with a new framework for building AI agents that adapt to individual user preferences, developed by Siqi Liang from Purdue University, Yudi Zhang from Iowa State University, and Yue Guo from Columbia University, et al. This research introduces PersonaAgent, a system that embodies a user’s unique persona and leverages a novel Knowledge-Graph-enhanced Retrieval-Augmented Generation (GraphRAG) mechanism to access and utilise relevant information. By constructing a knowledge graph from user data and identifying communities of related information, the team enables the agent to generate dynamic prompts that combine personal history with broader interaction patterns, resulting in consistently persona-aligned behaviour. The results demonstrate substantial improvements across several benchmarks, including an 11. 1% increase in news categorization accuracy, a 56. 1% improvement in movie tagging, and a 10. 4% reduction in product rating errors, marking a considerable advance in the development of intelligent, personalised AI systems.

This system generates personalized prompts by combining summaries of a user’s historical behaviors and preferences, extracted from the knowledge graph, with relevant global interaction patterns identified through graph-based community detection. Experiments using the LaMP benchmark demonstrate significant improvements across three decision-making tasks: news categorization, movie tagging, and product rating.

On the LaMP-2N news categorization task, the framework achieved 0. 804 accuracy and 0. 591 F1, representing a 1. 0% and 11. 1% improvement over prior methods.

Even more substantial gains were observed in movie tagging, with accuracy increasing from 0. 513 to 0. 653 (+27. 3%) and F1 from 0. 424 to 0.

662 (+56. 1%). For product rating, the system reduced Mean Absolute Error (MAE) from 0. 241 to 0. 216 (-10.

4%) and Root Mean Squared Error (RMSE) from 0. 509 to 0. 484 (-4. 9%). Further analysis revealed that even smaller language models, such as LLaMA3, can outperform competing methods using this framework; for example, accuracy on movie data improved by 13.

6%. A case study demonstrated that incorporating globally similar interactions from other users into the personalization prompt substantially improves classification accuracy. In one instance, the system corrected a misclassification of an article about a Parkland shooting survivor by recognizing connections to youth activism and gun law reform, demonstrating the value of community context in balancing personalization with generalizability. These results highlight the benefits of integrating structured user memory with graph-based retrieval for accurate and consistent personalization.

Community Context Improves Personalized AI Responses

The researchers introduced PersonaAgent with GraphRAG, a framework designed to deliver more accurate, explainable, and consistent personalized experiences from artificial intelligence systems. This system combines user-specific information with broader patterns of community interaction, allowing the agent to balance individual preferences with collective knowledge. The method utilizes a knowledge graph to retrieve and summarise relevant information, dynamically tailoring prompts to reflect both a user’s history and wider trends. Evaluations on a standard benchmark demonstrated significant improvements in several tasks, including news categorization, movie tagging, and product rating, compared to existing approaches.

The system effectively leverages community context to refine its understanding, particularly when user preferences are limited or skewed. Looking ahead, the team suggests exploring multi-agent collaboration, where multiple persona agents interact and share knowledge, to further enhance robustness and collective intelligence. They also propose incorporating inverse reinforcement learning to enable agents to better infer user preferences from behaviour, adapting to evolving goals while remaining consistent with established values.