Liver Tumor Segmentation Achieves 78.3% Accuracy Using Unpaired CT and MRI Data

Accurate identification of liver tumours relies on combining information from different medical imaging techniques, yet creating effective diagnostic tools is hampered by the limited availability of paired and aligned image datasets. Arunkumar V, Firos V M, Senthilkumar S, and Gangadharan G R present a new approach, A-QCF-Net, which overcomes this challenge by constructing a single segmentation model from separate, unpaired CT and MRI scans. The team leverages the efficiency of quaternion neural networks and introduces an innovative data-driven attention module that facilitates knowledge transfer between the imaging streams, allowing them to share crucial information such as anatomical boundaries and soft tissue contrast. Validating their framework on standard datasets, the researchers demonstrate a significant improvement in tumour segmentation accuracy, exceeding existing methods by substantial margins and confirming the model’s ability to focus on clinically relevant features, ultimately offering a pathway to utilise the wealth of unpaired medical images currently available in healthcare settings.

Accurate delineation of pathology remains a challenge, but the development of deep learning models is limited by the scarcity of large, paired, and spatially aligned datasets. This paper addresses this fundamental limitation by proposing an Adaptive Quaternion Cross-Fusion Network (A-QCF-Net) that learns a single unified segmentation model from completely separate and unpaired CT and MRI cohorts. The architecture exploits the parameter efficiency and expressive power of Quaternion Neural Networks to construct a shared feature space, enabling effective learning from limited data. At its core is the Adaptive Quaternion Cross-Fusion (A-QCF) block, a data driven attention module that enables bidirectional knowledge transfer between the imaging modalities.

Medical Image Segmentation, Techniques and Challenges

Recent research details a wide range of techniques and research directions in medical image segmentation, with a strong emphasis on addressing challenges like limited annotated data, cross-modality image analysis, and improving robustness. Scientists are employing unpaired image-to-image translation, such as CycleGAN, to augment data and enable segmentation in modalities with fewer labels, alongside self-supervised learning methods that learn from unlabeled data. Domain adaptation techniques, including adversarial learning, transfer knowledge from labeled to unlabeled datasets, reducing the need for extensive manual annotation. Multi-modal image fusion is also a key area of investigation, with networks designed to leverage complementary information from different imaging modalities like CT and MRI for improved segmentation.

Attention mechanisms, such as those used in Attention U-Nets, help networks focus on relevant features across modalities, while various fusion strategies explore optimal ways to combine information. Quaternion Neural Networks are being investigated as a way to effectively represent and process multi-dimensional medical image data, potentially improving feature extraction and fusion. Advanced network architectures, including U-Net and its variants, remain foundational, with researchers incorporating transformers to capture long-range dependencies and improve contextual understanding. Hybrid architectures combining CNNs and transformers are also proving promising, as are diffusion models for medical image segmentation and translation.

Improving robustness and generalization is achieved through domain generalization techniques, robust segmentation tools like TotalSegmentator, and noise adaptation methods using GANs. Deep learning, particularly convolutional neural networks and increasingly transformers, dominates the field, with data augmentation and transfer learning playing crucial roles in addressing data scarcity. Researchers are actively exploring new network architectures and fusion strategies to improve performance, and quaternion neural networks offer a potential solution for better representing and processing multi-dimensional medical image data.

Adaptive Fusion of Unpaired CT and MRI Data

Scientists have achieved a significant breakthrough in medical image segmentation by developing an Adaptive Quaternion Cross-Fusion (A-QCF-Net) capable of learning from completely separate and unpaired CT and MRI datasets. This innovative work overcomes a critical limitation in medical AI, namely the scarcity of large, paired, and spatially aligned multimodal imaging datasets, and unlocks the potential of commonly available, independent imaging archives. The architecture leverages the efficiency and expressive power of Quaternion Neural networks to construct a shared feature representation, enabling a single model to process both modalities effectively. At the heart of the A-QCF-Net is the Adaptive Quaternion Cross-Fusion (A-QCF) block, a data-driven attention module that facilitates bidirectional knowledge transfer between the CT and MRI streams.

This block dynamically modulates information flow, allowing the streams to exchange modality-specific expertise, such as sharp anatomical boundaries from CT and subtle soft tissue contrast from MRI, thereby regularizing and enriching feature representations. The team validated the framework by jointly training a single model on the unpaired LiTS (CT) and ATLAS (MRI) datasets, demonstrating its ability to generalize across modalities without requiring paired data. The jointly trained model achieves Tumor Dice scores of 76.7% on CT images and 78.3% on MRI images, significantly exceeding the performance of a strong unimodal nnU-Net baseline by margins of 5.

4% and 4.7%, respectively. Comprehensive explainability analysis, utilizing Grad-CAM and Grad-CAM++, confirms that the model correctly focuses on relevant pathological structures, ensuring the learned representations are clinically meaningful and trustworthy. This breakthrough delivers a robust and clinically viable paradigm for utilizing large, unpaired imaging archives, paving the way for more powerful and practical medical AI applications. The resulting model exhibits a generalized understanding of hepatic anatomy, informed by both CT and MRI, and performs well on either modality alone during inference.

Adaptive Cross-Fusion Improves Liver Tumor Segmentation

The research team presents A-QCF-Net, a new framework that addresses a significant challenge in medical imaging: the lack of paired datasets for training effective multimodal models. This innovative approach successfully learns a unified representation for image segmentation by combining data from separate CT and MRI scans, even when those scans are not directly linked. At the heart of the system is an Adaptive Quaternion Cross-Fusion block, which facilitates the exchange of knowledge between the two imaging types, allowing the model to leverage the strengths of each modality, sharp anatomical details from CT and subtle soft tissue contrast from MRI. The results demonstrate substantial improvements in liver tumor segmentation, exceeding the performance of existing methods by considerable margins on both CT and MRI images.

Importantly, explainability analysis confirms the model focuses on clinically relevant areas, indicating the learned representations are meaningful and trustworthy. This achievement establishes a new standard for segmenting images when paired data is unavailable, offering a practical solution for utilizing the wealth of unpaired imaging data commonly found in healthcare settings. Future research will extend the framework to incorporate data from multiple imaging types, such as PET scans, and explore hybrid training strategies combining unpaired and paired data. A key next step involves large-scale validation across multiple clinical sites and scanners to assess the model’s robustness and generalizability, ultimately paving the way for prospective clinical trials and integration into routine radiological workflows.