SegReg: Segmenting OARs by Registering MR Images and CT Annotations


ISBI 2024



Zeyu Zhang1,2,*
Xuyin Qi3
Bowen Zhang1
Biao Wu1
Hien Le4
Bora Jeong5,6

Zhibin Liao1
Yunxiang Liu1
Johan Verjans1
Minh-Son To1,7
Richard Hartley2,✉

1Australian Institute for Machine Learning
2The Australian National University
3School of Computer and Mathematical Sciences, The University of Adelaide
4Australian Bragg Centre for Proton Therapy and Research, South Australian Health and Medical Research Institute
5JBI, The University of Adelaide
6Department of Otolaryngology, Modbury Hospital
7Flinders Health and Medical Research Institute, Flinders University
*Work done while being a visiting student researcher at Australian Institute for Machine Learning.

[Paper]

[arXiv]

[GitHub]

[BibTeX]


News:



alt text

SegReg is a simple yet effective approach which harnesses co-registered MRI in conjunction with planning CT to perform multimodal OAR segmentation. The pipeline involves an Elastic Symmetric Normalization (ElasticSyN) transformation for registering MRI to CT and an nnUNet model for OAR segmentation, which effectively combines the geometric accuracy of CT with the superior soft-tissue contrast of MRI.


Abstract

Organ at risk (OAR) segmentation is a critical process in radiotherapy treatment planning such as head and neck tumors. Nevertheless, in clinical practice, radiation oncologists predominantly perform OAR segmentations manually on CT scans. This manual process is highly time-consuming and expensive, limiting the number of patients who can receive timely radiotherapy. Additionally, CT scans offer lower soft-tissue contrast compared to MRI. Despite MRI providing superior soft-tissue visualization, its time-consuming nature makes it infeasible for real-time treatment planning. To address these challenges, we propose a method called SegReg, which utilizes Elastic Symmetric Normalization for registering MRI to perform OAR segmentation. SegReg outperforms the CT-only baseline by 16.78% in mDSC and 18.77% in mIoU, showing that it effectively combines the geometric accuracy of CT with the superior soft-tissue contrast of MRI, making accurate automated OAR segmentation for clinical practice become possible.


Visualization



alt text











Results

Comparative Results


alt text

The comparative results presented in Table I highlight a notable improvement in our model's performance when compared to the CT-only baseline and other established OAR segmentation models.



alt text

The results compared with nnUNet baseline, including a detailed breakdown for each semantic, are presented in Table II. Our model demonstrates a notable performance improvement, especially in small and tiny organs, including the cochlea, anterior/posterior eyeball, lacrimal gland, optic nerves, and parotid gland. Specifically, there are improvements of 31.56%/26.26%, 32.23%/44.37%, 48.98%/58.08%, 34.84%/46.34%, 38.94%/28.36%, and 40.61%/25.46% in DSC for left and right organs respectively.



alt text

In addition, when considering models that also incorporate MRI data, it's worth noting that both MAML and MFM employ a four-fold cross-validation approach on the entire dataset without a separate hold-out test set, we have also conducted experiments with our SegReg model under the same setting. The results presented in Table III demonstrate that our SegReg model continues to outperform two-stream networks, irrespective of the modality fusion architectures used in multi-modal OAR segmentation.



Ablation Studies


alt text

To assess the extent of improvement achieved by the proposed registered MRI in OAR segmentation, we carried out an experiment focusing solely on the registered MRI data. The results in Table IV indicate that even when only using registered MRI, we achieve better performance in semantic classification, with an improvement of 3.55% in mDCS. Despite the geometric fidelity of registered MRI not matching that of the originally annotated CT scans, which has shown in agnostic metrics, the improvement in semantics still demonstrates MRI offers superior localization and contrast in soft tissue for segmentation models compared with CT scans, making it more precisely to distinguish and delineate different OARs. Furthermore, combining the original CT scans with registered MRI leverages the superior soft-tissue contrast of MRI to enhance semantic knowledge, while benefiting from the high geometrical accuracy of CT to improve mask shapes for OAR segmentation. This results in an improvement of 16.78% in mDSC and 18.77% in mIoU.



alt text

We also explore various transformation components of the MRI registration in SegReg, including Translation, Rigid transformation (translation and rotation), Affine transformation (translation, rotation, and scaling), and Elastic transformation (affine and deformable transformation), in comparison to the Elastic Symmetric Normalization. The results in Table V indicate that Elastic Symmetric Normalization, as employed in SegReg, outperforms any other registration method in OAR segmentation.



alt text

Furthermore, we investigated the impact of a two-stream backbone on multi-modal OAR segmentation, comparing it to the vanilla single-stream network. We replaced the nnUNet backbone with the MAML backbone in SegReg, and the performance is presented in VI. The results indicate that the two-stream architecture has minimal impact compared to the significant contribution of registration transformation to overall OAR segmentation performance. Using a single-stream backbone remains a simple yet effective approach for registration segmentation.