Capsule-Expert Routing UNet: A Hybrid 2.5D Convolution-Attention Architecture with Mixture-of-Experts for 3D Medical Segmentation

Nand Kumar Yadav, Rodrigue Rizk, William CW Chen, KC Santosh

Keywords: 3D Image Segmentation, Inception-style 2.5D convolutions, Capsule Routing, Mixture- of-experts

Abstract:

Recent advances in 3D medical image segmentation have been driven by hybrid CNN-Transformer architectures that capture long-range dependencies at the cost of heavy parameters. This paper introduces Capsule-Expert Routing UNet (CER-UNet), a novel encoder–decoder model that achieves strong global context modeling with substantially lower computational parameters. CER-UNet integrates two complementary contributions: (1) a statistical attention module that performs computationally efficient long-range interaction via low-rank covariance pooling and channel-wise statistics, coupled with a 2.5D hybrid convolutional design featuring Inception-style multi-scale
depthwise-separable kernels. (2) a Capsule-Expert Mixture-of-Experts (CapMoE) routing mechanism that introduces dynamic feature routing across hierarchical scales, enabling lightweight multi-scale fusion and expert specialization while avoiding the instability of full attention-based routing mechanism. CER-UNet preserves the strong context modeling of recent UNet-like CNN-Transformer hybrids but surpasses them in accuracy-efficiency trade-off.CER-UNet achieves an average Dice of 92.52% on ACDC, 84.94% on BTCV, and 86.64% on Synapse, while using nearly only 32M parameters. Across all three benchmarks, it consistently outperforms competitive Transformer-based methods and conventional 2D/2.5D and 3D segmentation networks, highlighting a strong accuracy-efficiency trade-off. Extensive experiments across multiple 3D medical segmentation benchmarks demonstrate that CER-UNet delivers robust state-of-the-art per-
formance with significantly lower computational overhead.

Primary Subject Area: Segmentation

Secondary Subject Area: Detection and Diagnosis

Registration Requirement: Yes

Reproducibility: https://anonymous.4open.science/r/CER-Unet-BC53

Visa & Travel: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 364