Rebuttalagent Achieves 18.3% Persuasion Boost Via Theory of Mind Modelling

Researchers are tackling the surprisingly difficult problem of crafting effective rebuttals to academic peer review, a crucial yet often overlooked aspect of the research process. Zhitao He, alongside Zongwei Lyu from Hong Kong University of Science and Technology and Yi R Fung, present a novel framework , RebuttalAgent , which uniquely grounds rebuttal strategy in Theory of Mind, allowing it to model the reviewer’s perspective and formulate persuasive responses. This work is significant because it moves beyond simple linguistic imitation, addressing the core challenge of strategic communication under uncertainty, and introduces both a large-scale dataset, RebuttalBench, and a robust evaluation metric, Rebuttal-RM, demonstrably exceeding the performance of leading AI models and even GPT-4.1.

Researchers are tackling the surprisingly difficult problem of crafting effective rebuttals to academic peer review, a crucial yet often overlooked aspect of the research process.

Theory of Mind for Academic Rebuttal requires anticipating

Scientists have unveiled RebuttalAgent, a groundbreaking framework that integrates Theory of Mind (ToM) into the challenging process of academic rebuttal. This innovative approach addresses a significant gap in artificial intelligence research, where automated rebuttal systems have historically struggled due to their reliance on superficial linguistic imitation rather than strategic, perspective-taking reasoning. This dataset was synthesised using a novel critique-and-refine approach, ensuring a robust and diverse training ground for the model.

This allows the agent to continuously refine its rebuttal strategies without relying on external reward models, enhancing its adaptability and effectiveness. Remarkably, Rebuttal-RM achieves scoring consistency with human preferences, surpassing even the capabilities of the powerful GPT-4.1 judge. This breakthrough establishes a new paradigm for AI-assisted academic rebuttal, moving beyond simple imitation to embrace strategic reasoning and perspective-taking. The work opens exciting possibilities for assisting authors in crafting more persuasive and effective responses to reviewer comments, ultimately streamlining the peer review process and fostering more constructive scientific discourse. The generated rebuttal content, while intended as a reference for authors, is not a replacement for critical analysis, but rather a tool to inspire and assist in drafting compelling responses.

ToM-TSR Pipeline and RebuttalBench Dataset Construction represent significant

This pipeline models reviewer mental state, formulates a persuasion strategy, and generates a strategy-grounded response, addressing a critical gap in automated academic writing assistance. This dataset enables the development of robust rebuttal systems capable of handling complex academic discourse. Initially, the team employed an LLM-as-Extractor, guided by a specifically designed prompt, to segment raw reviews into discrete, actionable comments. This process involved identifying and separating each distinct point of criticism, achieving 98% accuracy on a manually verified sample of 100 reviews.

Following comment extraction, a three-stage context retrieval pipeline isolated the most relevant manuscript content for each comment. The manuscript was first segmented into paragraphs, then encoded alongside the target comment using a pre-trained embedding model, allowing for the quantification of relevance based on high-dimensional vector representations. This was followed by a reinforcement learning phase, leveraging a self-reward mechanism to enable scalable self-improvement and optimise performance. Experiments employed the GRPO algorithm to optimise the agent’s responses, generating three candidate responses per comment and selecting the best based on the self-reward signal. Rebuttal-RM demonstrated scoring consistency with human preferences, surpassing the performance of powerful judge models like GPT-4.1. This work establishes a new paradigm for automated academic rebuttal, moving beyond surface-level linguistics to address the core challenge of persuasive communication under information asymmetry.

RebuttalAgent outperforms baselines using a TSR pipeline

Experiments demonstrate that Rebuttal-RM, a fine-tuned Qwen3-8B model used for evaluating rebuttal quality, outperforms all baselines in aligning with human judgements, achieving an average score of 0.812. Notably, Rebuttal-RM surpasses GPT-4.1 and DeepSeek-r1 by 9.0% and 15.2% respectively, as measured by a suite of six statistical metrics including Pearson correlation (r), Spearman’s rank correlation (β), and Kendall’s tau (τ). These metrics assess both overall correlation and classification accuracy through coarse-grained and fine-grained accuracy assessments. Benchmarking RebuttalAgent against foundation models like GPT-4.1, DeepSeek-R1, and Gemini-2.5, as well as agent-based methods, reveals substantial improvements.

RebuttalAgent achieves an average metric increase of 18.3% across various evaluation criteria, including Rigor, Soundness, Significance, and Presentation. Specifically, the model attained scores of 9.23 for Rigor, 8.91 for Soundness, and 9.59 for Significance, demonstrating a marked advancement in the quality of generated rebuttals. Ablation studies confirm the importance of the ToM component, with its removal leading to a noticeable performance decrease. The team reports a 21.6% improvement in Soundness and a 22.1% improvement in Significance when utilizing the full TSR pipeline.

ToM agent learns persuasive rebuttal strategies through interactive

Acknowledging limitations, the authors emphasise that the generated rebuttals are intended as reference material to inspire authors, not to replace critical analysis. They also note the potential for the system to inadvertently learn and reinforce biases present in the training data, and have taken steps to mitigate this by excluding comments related to experimental results. Future research directions include expanding the framework to assist researchers across various disciplines and fostering more effective human-AI collaboration in scientific discourse. This work represents a significant step towards enhancing the clarity and constructive nature of academic dialogue, offering a valuable resource for scholars navigating the rebuttal process.