Federated In-Context Prompt Selection for Multi-Modal 3D Dental Imaging: A Theoretical Framework with Privacy-Preserving Guarantees

Ushashi Bhattacharjee, Tirtho Roy

Keywords: Federated learning · Vision-language models · Medical imaging · Privacy preservation · Multi- modal learning · Prompt engineering · Differential privacy · Byzantine resilience

Abstract:

Vision-language models show remarkable capabilities in medical imaging analysis, yet their deploy-
ment in federated healthcare environments faces key challenges in privacy preservation, data heterogeneity, and
adversarial robustness. We present FedDental3D-ICL, a novel theoretical framework for federated in-context
prompt learning that enables privacy-preserving collaboration across healthcare institutions without sharing
sensitive patient data or model parameters. Our framework introduces four core algorithmic contributions:
Multi-Modal Prompt Space (MMPS) abstraction unifying visual and textual prompt representations across 2D
and 3D medical imaging modalities; Cross-Modal Prompt Alignment (CMPA) ensuring semantic consistency
through information-theoretic contrastive objectives; Hierarchical Multi-Modal Optimization (HMMO) pro-
viding rigorous convergence guarantees for non-convex federated objectives; and Byzantine-Resilient Cross-
Modal Aggregation (BRCMA) with differential privacy bounds. Our theoretical analysis establishes conver-
gence rates of O(1/√T ), communication complexity reductions from O(K · d) to O(K log |P |), and (ε, δ)-
differential privacy guarantees with optimal composition bounds.

Changes Summary:

Enhanced Mathematical Framework: We addressed the most critical reviewer feedback by providing comprehensive definitions for all previously undefined mathematical constants. I explicitly defined the convergence analysis constants C₁, C₂, C₃, and C₄ in equations (21)-(24), where C₁ = L√2(L(θ₀) - L*) represents initial suboptimality dependency, C₂ = 2η²L² captures learning rate and smoothness interactions, C₃ = 4ηL quantifies heterogeneity impact, and C₄ = 4ηL∆² measures Byzantine attack magnitude. Additionally, I defined all problem parameters including σ² (bounded gradient variance), ζ² (data heterogeneity), ∆² (Byzantine attack bounds), and f (Byzantine client count) in equations (25)-(28). This directly resolved Reviewer 2wbg's concern about "important constants being introduced without explicit definitions."

Improved Technical Specifications: I provided concrete implementation details for the fusion function through Definition 8, specifying F(z_v, z_t, z_3D) = σ(W₁[z_v; z_t; z_3D] + b₁) with explicit parameter dimensions and activation functions. I also added Remark 1 detailing the computational complexity O(d²) and memory requirements, addressing concerns about underspecified components. The Lipschitz continuity assumption was made explicit in equation (29), strengthening our theoretical foundations.

Documentation and Clarity Improvements: I added a comprehensive caption for Figure 1 explaining the system architecture and multi-modal data flow across federated dental institutions. The mathematical justifications throughout Section 4 were expanded to provide clearer theoretical grounding for our approach.

Acknowledged Limitations: While I strengthened the mathematical rigor, I recognize that fundamental conceptual issues raised by reviewers remain unaddressed, including the core algorithmic framework and practical implementation details that would require substantial additional work.

Main Tex File: main.tex

Confirm Latex Only: true

Authors Changed: true

Submission Number: 10