Advancing Agentic AI: Decentralized and Verifiable Collaboration for Next-Generation Foundation Model Development

Arpita Sarker, Arpita Sarker, Alexander Jesser

Keywords: Agentic AI, Foundation Models, Decentralized Learning, Federated Learning, Knowledge Distillation, Multi-Agent Systems, Verifiable Machine Learning, IOTA Tangle, IPFS Storage, Model Provenance, Distributed Consensus, Privacy-Preserving AI, Collaborative AI Workflows, Trustworthy AI, Peer-to-Peer Model Training

TL;DR: We propose a decentralized, verifiable framework for training foundation models using federated learning, knowledge distillation, and distributed consensus—achieving high accuracy without compromising privacy or trust.

Abstract:

Foundation models such as large language models have achieved remarkable performance by leveraging massive
centralized datasets and compute. However, concerns around data privacy, governance, and trust motivate new
agentic workflows where multiple parties (agents) collaboratively develop models without central custodians. We
propose a decentralized framework for verifiable multi-agent model training that integrates federated learning,
distributed ledger technologies, and knowledge distillation. In our approach, each participant maintains local data
and models, contributing updates that are logged on a tamper-proof DAG ledger for transparency and account-
ability. A voting-based consensus mechanism enables multi-agent governance, ensuring only high-quality model
updates are merged. To aggregate knowledge from diverse sources, we employ cross-silo knowledge distilla-
tion, including distilling large teacher models (e.g. LLaMA, BioGPT) into smaller models in a federated setting.
Empirical evaluations on collaborative learning scenarios – including named entity recognition (F1=96.23%),
medical code classification (F1=79.11%), and question-answering tasks – demonstrate that our decentralized
training achieves performance comparable to centralized methods while preserving privacy and trust. This work
advances agentic AI by enabling next-generation foundation model development through privacy-preserving,
trustable collaboration.

Submission Number: 37