Llms Enable Strategic Computation Allocation with ROI-Reasoning for Tasks under Strict Global Constraints

The challenge of equipping large language models with the ability to manage their own computational resources is now being addressed by researchers at Renmin University of China. Muyang Zhao, Qi Qi, and Hao Sun propose a new framework, ROI-Reasoning, which allows models to strategically allocate computation based on anticipated task difficulty. Their work formalises the problem as an Ordered Stochastic Multiple-Choice Knapsack Problem, highlighting the need for models to predict return on investment before committing to a solution. This research is significant because it moves beyond simply achieving strong reasoning performance and towards building genuinely budget-aware and rational artificial intelligence systems, demonstrably improving scores and reducing wasted computation in mathematical reasoning tasks.

Researchers developed ROI-Reasoning, a two-stage framework designed to imbue LLMs with intrinsic rationality when faced with computational constraints, moving beyond simply achieving strong reasoning performance to actively managing the resources required for each task. The initial stage, Meta-Cognitive Fine-Tuning (MFT), teaches the model to predict reasoning cost and expected utility before generating a solution. MFT employs a structured difficulty tag, Level-k, to represent anticipated token consumption, categorising effort into levels corresponding to predefined token ranges.

This process began with fine-tuning on single problems to establish mappings between difficulty and cost levels, then expanded to multi-problem sequences to encourage cost prediction within a shared budget. Following MFT, Rationality-Aware Reinforcement Learning (RARL) places the model in simulated multi-problem exams with a strict global token budget, B. Each exam presents N problems in a fixed order, demanding sequential processing and adherence to the budget. The team engineered a training regime where the model learns to optimise both problem-solving and budget allocation through trial and error, maximising overall return on investment, and allowing a \box{ NA } token to signal strategic abstention from low-ROI problems.

The complete MFT procedure alternates between single and multi-problem instances, bridging difficulty assessment and contextual decision-making. This methodology enables the model to become a rational agent capable of strategic resource allocation and prioritisation, ultimately improving performance under tight computational budgets. The researchers detail dataset construction and rejection sampling rules in an appendix, ensuring reproducibility and transparency.

ROI-Reasoning Boosts Mathematical Problem Solving Performance

Scientists achieved substantial improvements in multi-problem mathematical reasoning through ROI-Reasoning. Experiments revealed that the approach consistently enhanced overall scores while significantly reducing regret under strict computational limits, delivering a system capable of making informed decisions about when to solve problems and when to abstain. The core of ROI-Reasoning is a two-stage process beginning with Cognitive Fine-Tuning, which trains models to predict both reasoning cost and expected utility prior to generating a solution, allowing the model to make explicit solve-or-skip decisions.

Following this, Rationality-Aware Reinforcement Learning optimises sequential decision-making, enabling the model to develop long-horizon allocation strategies. Tests prove that this combination allows the model to jointly improve problem-solving competence and planning ability, learning to invest computation in high-return problems and conserve budget. Detailed performance comparisons were conducted across various difficulty levels and token budgets, utilising benchmarks of mathematical reasoning. Results demonstrate that, under a 1024-token constraint on Medium difficulty papers, the method achieved a score of 1.13, alongside a scoreeasy of 1.15 and a regret of 0.02.

Under a 512-token budget, the model maintained a score of 0.97, a scoreeasy of 1.08, and a regret of 0.11, consistently outperforming baseline models like DeepSeek-V3.2 (685B parameters). Further analysis of token length distributions revealed that the model dynamically adjusts its reasoning length based on problem difficulty and budget constraints. Histograms comparing Qwen2.5-1.5B-Instruct, Qwen2.5-1.5B-Instruct + MFT, and Qwen2.5-1.5B-Instruct + MFT + RARL show a clear shift towards more efficient token usage with the full ROI-Reasoning framework. Specifically, the team measured advantage from group-level rewards, computing the importance sampling ratio at each token position to refine the reinforcement learning process, opening possibilities for deploying complex reasoning tasks on resource-constrained devices.

ROI-Reasoning Improves Budgeted Language Model Performance

This research introduces ROI-Reasoning, a framework designed to equip large language models with budget-aware rationality during multi-task reasoning. Through a two-stage process of Meta-Cognitive Fine-Tuning and Rationality-Aware Reinforcement Learning, the framework enables models to make informed decisions about whether to attempt a problem given limited computational budgets. Experiments on mathematical reasoning benchmarks demonstrate that ROI-Reasoning consistently improves overall performance while minimising regret when operating under strict token constraints. Analysis of model behaviour reveals that the framework encourages budget-aware behaviour, with models shortening reasoning or abstaining from harder problems more effectively than baseline models. The authors acknowledge limitations including the use of token count as a proxy for computational cost and reliance on coarse difficulty supervision, suggesting future research extend the framework to more complex agentic settings involving heterogeneous subtasks, non-stationary rewards, and richer cost models.