Vtc-R1 Achieves 3.4x Reasoning Speed-Up with Vision-Text Compression

Scientists are tackling the efficiency challenges of long-context reasoning in large language models, a key limitation hindering progress on complex tasks. Yibo Wang, Yongcheng Jing, and Shunyu Liu from institution 1, alongside Rong-cheng Tu and Chengyu Wang, present VTC-R1, a novel paradigm integrating vision-text compression directly into the reasoning process. Unlike existing methods requiring complex training or external models, VTC-R1 compresses intermediate reasoning steps into images, effectively creating a visual “memory” for vision-language models. This approach achieves 3.4x token compression and demonstrably outperforms standard reasoning techniques on benchmarks including MATH500 and GPQA-D, while also delivering a 2.7x speedup in inference latency, offering a potentially scalable solution for demanding applications.

Unlike existing methods requiring complex training or external models, VTC-R1 compresses intermediate reasoning steps into images, effectively creating a visual “memory” for vision-language models.

Vision-text compression for efficient long-context reasoning enables scalable

Scientists have developed a novel approach to enhance long-context reasoning in large language models, addressing the efficiency bottlenecks that arise with complex tasks. This innovative method tackles the computational challenges associated with the quadratic growth of transformer architectures, which significantly impacts both computation and memory costs as context expands. Researchers segmented long reasoning traces into shorter segments, rendering the preceding segments as images to form paired image-text data. The work establishes that transforming textual reasoning into visual representations preserves critical fine-grained information, unlike existing methods that often discard it.

By leveraging the symbolic structure of mathematical reasoning, the team has created a principled testbed for studying reasoning-oriented vision-text compression. The study unveils a method where preceding reasoning steps are rendered into images and used as optical memory, enabling VLMs to process information more efficiently. The team’s code is publicly available, facilitating further research and development in this promising area of artificial intelligence and offering a pathway towards more efficient and scalable reasoning systems.

Visual Reasoning via Compressed Image Memory enables efficient

Researchers constructed a dedicated training dataset, OpenR1-Math-220K, to facilitate this process and then fine-tuned representative VLMs, specifically Glyph and Qwen3-VL, using this dataset. This approach achieved a 3.4x token compression rate, significantly reducing the computational load without discarding crucial fine-grained information. The team engineered a system where the reasoning process is segmented into discrete steps, with each completed step immediately rendered into an image. This image, representing the accumulated reasoning history, is then concatenated with the current question and input into the VLM for the subsequent reasoning step.

Experiments employed a rendering process to transform textual reasoning traces into visual representations, enabling the VLMs to encode information using fewer vision tokens. To rigorously evaluate VTC-R1, researchers conducted extensive experiments on established benchmarks including MATH500, AIME25, AMC23 and GPQA-D. Performance comparisons consistently demonstrated that VTC-R1 outperformed standard long-context reasoning techniques across all datasets. This substantial improvement highlights the potential of VTC-R1 as a scalable solution for reasoning-intensive applications, addressing the quadratic complexity inherent in transformer architectures.

The approach enables the preservation of fine-grained information by leveraging visual encoding, unlike methods relying on summarization or pruning. Scientists harnessed the symbolic structure of mathematical reasoning as a principled testbed, demonstrating the effectiveness of reasoning-oriented vision-text compression. The technique reveals that high-density visual representations can effectively support multi-step reasoning processes, paving the way for more efficient and scalable LLMs. The code implementing VTC-R1 is publicly available, facilitating further research and development in this area.

VTC-R1 boosts reasoning via visual token compression

Experiments were conducted using a training dataset constructed from OpenR1-Math-220K, and representative VLMs, Glyph and Qwen3-VL, were fine-tuned. Extensive testing on MATH500, AIME25, AMC23, and GPQA-D revealed substantial performance gains with VTC-R1. Notably, on the challenging MATH500 benchmark, the team recorded a 5.6% accuracy improvement, while AMC23 saw a 3.4% increase when compared to the baseline. On the Qwen3-VL architecture, VTC-R1 consistently achieved competitive or improved accuracy, further validating its effectiveness. Data shows that on the out-of-distribution GPQA-Diamond benchmark, VTC-R1 yielded accuracy improvements of 7.6% using Glyph and 11.1% using Qwen3-VL, indicating strong generalisation beyond in-distribution mathematical problems.

On the Glyph architecture, a speedup of at least 1.4x was observed across all benchmarks, with gains reaching 1.7x and 1.6x on the most difficult tests. The Qwen3-VL architecture experienced even greater improvements, with inference speedup reaching up to 6.6x. The latency is computed as LAT = t2 −t1m × n, where t1 and t2 represent the start and end times of the entire inference process, respectively, and n is a scaling factor. Researchers discovered that the token count reduction and latency improvement do not scale linearly. For example, on Glyph for AMC23, the token count was reduced by approximately 1.3x, while the latency improved by 1.6x, suggesting that vision-text compression delivers additional efficiency gains beyond simple token reduction. Analysis of iteration epochs revealed that accuracy consistently improved with increased reasoning iterations, converging from approximately the fifth epoch onward. The team demonstrated that VTC-R1 can overcome training context limitations, surpassing the baseline accuracy when the maximum number of newly generated tokens is set to 8,192.

Visual compression boosts large language model reasoning capabilities

This approach was tested using a training dataset constructed from OpenR1-Math-220K, and applied to models including Glyph and Qwen3-VL. Qualitative analysis reveals the method’s ability to perform solution verification, reasoning summarisation, error correction, and continuation of preceding reasoning steps. The authors acknowledge that this work focuses on improving efficiency in long-context reasoning, and further research could explore the application of vision-text compression to other areas of artificial intelligence. They suggest that their work may inspire exploration of efficient reasoning beyond purely text-based paradigms. While the results demonstrate a significant improvement in both accuracy and speed, the authors do not speculate on broader societal impacts beyond advancing the field of large language model reasoning.