Information-Geometric Perspectives on Merging Variational Foundation Models

Nour Jamoussi, Giuseppe Serra, Photios A. Stavrou, Marios Kountouris

Keywords: Variational Foundation Models, Merge, Bayesian Learning, Information Geometry

Abstract: We propose an information-geometric framework for merging variational foundation models that preserves global robustness while integrating domain-specific knowledge in a principled manner. Assuming that the foundation models have been pretrained or fine-tuned using the Improved Variational Online Newton (IVON) optimizer, matching Adam’s computational cost while providing Bayesian advantages, we formulate the merging problem between the pretrained and fine-tuned models as an information-geometric projection. Under mild assumptions, this reduces to computing a barycenter in the variational parameter space, yielding a computationally efficient and theoretically grounded merging rule. The framework naturally extends to multi-model barycentric merging, minimizing the average discrepancy among fine-tuned models.

Serve As Reviewer: ~Nour_Jamoussi1

Submission Number: 16