Demystifying Network Foundation Models

Roman Beltiukov, Satyandra Guthula, Wenbo Guo, Walter Willinger, Arpit Gupta

Keywords: network foundation model, foundation model, computer networks

TL;DR: We present a latent knowledge exploration of network foundation models, their representation space utilization, alignment with domain-expert features, and causal sensitivity to perturbations.

Abstract: This work presents a systematic investigation into the latent knowledge encoded within Network Foundation Models (NFMs). Different from existing efforts, we focus on hidden representations analysis rather than pure downstream task performance and analyze NFMs through a three-part evaluation: Embedding Geometry Analysis to assess representation space utilization, Metric Alignment Assessment to measure correspondence with domain-expert features, and Causal Sensitivity Testing to evaluate robustness to protocol perturbations. Using five diverse network datasets spanning controlled and real-world environments, we evaluate four state-of-the-art NFMs, revealing that they all exhibit significant anisotropy, inconsistent feature sensitivity patterns, an inability to separate the high-level context, payload dependency, and other properties. Our work identifies numerous limitations across all models and demonstrates that addressing them can significantly improve model performance (up to 0.35 increase in $F_1$ scores without architectural changes).

Code URL: https://github.com/maybe-hello-world/demystifying-networks

Primary Area: Datasets & Benchmarks illustrating Different Deep learning Scenarios (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Submission Number: 2650