A single abdominal CT study, often comprising 300-600 axial slices, presents a significant challenge for artificial intelligence; standard 2D image analysis models struggle to reconstruct the necessary 3D anatomical context. NVIDIA is addressing this gap with NV-Reason-CT, a vision language model purpose-built for 3D CT analysis that extends chain-of-thought reasoning to full volumetric scans.
Building on the methodology of NV-Reason-CXR, validated by a multi-reader clinical study accepted at RSNA 2026, NV-Reason-CT supports multistep follow-up conversation across chest and abdomen, generating structured diagnostic reports and emulating radiologist internal thinking. It is an AI foundation model designed for researchers and developers building specialized CT analysis applications to post-train for their use case.
NV-Reason-CT Extends Chain-of-Thought to 3D CT Volumes
NV-Reason-CT processes full computed tomography volumes as true 3D inputs, a departure from many existing systems that analyze individual slices in isolation and risk losing crucial anatomical context. This approach preserves through-plane anatomical continuity, allowing the vision language model to reason about structures as a radiologist would while scrolling through a study.
The system combines a dedicated full 3D vision transformer with a language model specifically trained to generate chain-of-thought reasoning, mirroring the systematic analytical process employed by clinicians, NVIDIA says. The model’s ability to articulate its reasoning is a key differentiator, according to NVIDIA; “Being able to review the model’s thought process—not just its conclusions—is what makes it possible to trust and act on its findings.” This transparency extends beyond benchmark scores, with radiologists consistently highlighting the time savings achieved in structured reporting.
Generating detailed reports covering 30 chest and 29 abdominal abnormalities is typically a time-consuming task, even for experienced practitioners, but NV-Reason-CT produces this output in seconds, reducing cognitive load while maintaining radiologist oversight. That prior work confirmed radiologist time savings without compromising diagnostic accuracy, and the current iteration aims to replicate that success in the more complex domain of 3D CT analysis. The model’s outputs are auditable in a way that traditional “black-box” systems are not, allowing clinicians to examine the chain-of-thought, confirm alignment with their own reasoning, and identify any discrepancies.
This level of transparency is important for clinical adoption, and NVIDIA intends NV-Reason-CT to be a foundation for the broader medical AI community. Researchers can utilize the model checkpoints and post-training recipes to investigate chain-of-thought reasoning in medical imaging, fine-tune the model with institution-specific CT datasets, or integrate it into multimodal research pipelines.
Complementary models, such as NV-Generate-CTMR and NV-Segment-CTMR, further expand the possibilities for end-to-end radiology AI pipelines, covering synthetic data generation, segmentation, and transparent clinical reasoning. “NV-Reason-CT provides the kind of systematic, step-by-step reasoning that reflects how we actually think through a CT study,” the company states, emphasizing the model’s alignment with established clinical workflows. By processing the entire volume at once, NV-Reason-CT aims to provide a more comprehensive and nuanced understanding of the anatomy and potential abnormalities within.
NV-Reason-CT provides the kind of systematic, step-by-step reasoning that reflects how we actually think through a CT study.
Baris Turkbey, M.D., F.S.A.R., Senior Clinician, National Institutes of Health
Full 3D ViT Encoder Captures Volumetric Anatomical Continuity
A dedicated, full three-dimensional vision transformer (ViT) encoder now processes computed tomography volumes natively, extracting features that conventional two-dimensional approaches cannot replicate; this advancement forms the core of NVIDIA’s NV-Reason-CT system. Unlike methods adapting 2D encoders by treating slices as independent entities, NV-Reason-CT analyzes the entire CT volume as a unified three-dimensional input, preserving anatomical continuity between planes. The system’s architecture incorporates 3D vision token grid coordinates passed to a large language model (LLM) to account for spatial relationships between tokens throughout the LLM layers via 3D MRoPE.
This nuanced understanding of spatial context is critical for accurate diagnosis, particularly in complex anatomical regions where subtle variations can indicate pathology. A reward function, anatomy-aware and focused on the accuracy of identified abnormalities and diagnoses, guides the model to generate reasoning that is both structurally sound and clinically correct.
Structured Reports & Reasoning Mirror Radiologist Workflows
NV-Reason-CT generates structured diagnostic reports using a curated CT ontology encompassing 30 chest and 29 abdominal abnormalities, including lung nodules, pneumothorax, hepatic lesions, and renal cysts, designed to align with existing clinical documentation workflows. This detailed structuring moves beyond simple image labeling, a critical distinction given that radiologists assess patients through systematic anatomical reviews and differential diagnoses. The system’s ability to articulate reasoning, examining anatomical regions, surfacing findings, and considering diagnostic possibilities, directly emulates the cognitive process of an experienced radiologist interpreting a scan.
Unlike many existing medical AI models, NV-Reason-CT supports iterative clinical reasoning through multi-turn dialogue, allowing for follow-up questions and reconsideration of findings across different anatomical regions. This conversational depth is important for building trust, as radiologists do not finalize diagnoses on initial observation but instead refine their understanding through ongoing analysis.
The model’s training incorporated refusal examples for invalid prompts and mismatched image-text pairs, enhancing its robustness and preventing spurious outputs. Data used to train the model included CT-RATE, NIH CT datasets, and CancerVerse, supplemented with synthetic reasoning data grounded in expert radiologist annotations. The development team employed reinforcement learning (RL) in a second training stage to further refine the quality of reasoning produced by the model.
Beyond benchmark performance metrics, NV-Reason-CT received positive clinical validation from radiologists at the National Institutes of Health, who confirmed both the quality of the generated structured reports and the clinical plausibility of the reasoning traces. This validation is particularly significant given the challenges of assessing AI reasoning in complex 3D medical imaging. The architecture of NV-Reason-CT is intended to be adaptable for integration into existing radiology workflows.
Two-Stage Training Pipeline Leverages Radiologist Annotations
This detailed processing distinguishes it from simpler 2D analysis methods and allows for a more comprehensive understanding of the scanned anatomy. The system’s training methodology mirrors that of NV-Reason-CXR, beginning with a two-stage pipeline of supervised fine-tuning followed by reinforcement learning (RL). The initial supervised fine-tuning stage uses a mixture of data including structured reports, expert radiologist reasoning annotations, and general visual question answering datasets, creating a curriculum spanning approximately 550,000 structured QA examples.
Radiologists contributed detailed dictations capturing their internal review process, specifically noting examinations of anatomical regions, significant findings, considered differentials, and final assessments; this data covers section-level anatomy questioning, laterality-specific localized findings, severity-level questioning, and binary abnormality identification. “The model is designed to respond not as a classifier, but as a teacher: explaining the problem, walking through the evidence, and arriving at a diagnosis through visible logical steps,” explains the development team.
The model’s reasoning process is demonstrated through an example case where it identifies a pattern concerning for thoracic malignancy with nodal involvement, alongside scattered bilateral ground-glass opacities and indeterminate right rib sclerotic lesions. The system’s output, as illustrated in the example, mirrors a radiologist’s structured report, noting both normal and abnormal findings with calibrated uncertainty before arriving at a conclusion.
NV-Reason-CT is positioned as part of the broader NVIDIA Medical AI ecosystem, alongside tools like NV-Generate-CTMR, which synthesizes realistic 3D CT and MRI volumes for data augmentation and research purposes, creating a pipeline for comprehensive medical imaging analysis.
3D MRoPE Accounts for Spatial Relationships in LLM Layers
Utilizing 3D Multi-headed Relative Positional Encoding (MRoPE) within its language model layers to maintain spatial awareness, the architecture of NV-Reason-CT directly addresses the limitations of applying two-dimensional image analysis techniques to the complexities of computed tomography. Unlike standard vision language models that process CT volumes as independent slices, NVIDIA’s system processes the entire volume as a three-dimensional input, preserving critical anatomical relationships that define pathologies.
This approach allows the model to reason about the shape, extent, and density of structures in a clinically meaningful way, mirroring how radiologists interpret scans. The core of this spatial understanding lies in the integration of 3D vision tokens with the language model, passing all vision tokens, derived from CT volumes resampled to 192³ voxels at 2 mm isotropic resolution, along with their 3D grid coordinates.
This ensures the language model accounts for the spatial relationships between these tokens throughout its layers, a feature enabled by the 3D MRoPE implementation, according to NVIDIA. The encoder itself is built upon the Primus 3D ViT architecture, initialized with Colipri weights, and generates a context of 24x24x24 = 13,824 vision token context from each scan. This contrasts sharply with approaches that flatten volumetric data into 2D sequences, inevitably losing crucial spatial information.
The language model component, Qwen3.5-4B LLM, is specifically trained to emulate a radiologist’s reasoning process, systematically reviewing anatomy, noting both normal and abnormal findings, and articulating uncertainty. The implementation of 3D MRoPE is a foundational element enabling the model’s ability to reason about spatial extent, cross-sectional morphology, and inter-slice relationships.
These perceptual foundations are essential for accurate CT interpretation, as they allow the model to understand how structures appear and interact across the entire volume. This is particularly important for identifying subtle abnormalities that might be missed when analyzing individual slices in isolation. The team reports that the model architecture combines the Qwen3.5-4B LLM with the 3D ViT, with all weights retrained end-to-end on a large cohort of CT data.
The training process itself reinforces this spatial reasoning capability, utilizing a two-stage pipeline of supervised fine-tuning and reinforcement learning. Radiologists contributed detailed chain-of-thought dictations, capturing their internal review processes and providing a rich dataset for training the model to emulate their reasoning, the company says. The resulting system doesn’t simply identify abnormalities; it articulates why it believes they are present, offering a level of transparency and interpretability previously unavailable in automated CT analysis.
See today’s quantum computing news on Quantum Zeitgeist for the latest breakthroughs in qubits, hardware, algorithms, and industry deals.
