Position: AI Scaling: From Up to Down and Out

Yunke Wang, Yanxi Li, Chang Xu

TL;DR: AI Scaling redefines AI's future by integrating Scaling Up, Scaling Down, and Scaling Out to achieve efficiency, equity, and transformative progress toward AGI.

Abstract: AI Scaling has traditionally been synonymous with Scaling Up, which builds larger and more powerful models. However, the growing demand for efficiency, adaptability, and collaboration across diverse applications necessitates a broader perspective. This position paper presents a holistic framework for AI scaling, encompassing Scaling Up, Scaling Down, and Scaling Out. It argues that while Scaling Up of models faces inherent bottlenecks, the future trajectory of AI scaling lies in Scaling Down and Scaling Out. These paradigms address critical technical and societal challenges, such as reducing carbon footprint, ensuring equitable access, and enhancing cross-domain collaboration. We explore transformative applications in healthcare, smart manufacturing, and content creation, demonstrating how AI Scaling can enable breakthroughs in efficiency, personalization, and global connectivity. Additionally, we highlight key challenges, including balancing model complexity with interpretability, managing resource constraints, and fostering ethical development. By synthesizing these approaches, we propose a unified roadmap that redefines the future of AI research and application, paving the way for advancements toward Artificial General Intelligence (AGI).

Lay Summary:

Most advances in AI today come from building ever-larger models, which is a strategy known as "Scaling Up." But this approach faces growing challenges: it’s costly, energy-intensive, and often inaccessible to many communities and applications. Our research asks: how can we rethink AI scaling to make it more efficient, adaptable, and widely usable?

In this position paper, we propose a new framework that combines three directions: Scaling Up, Scaling Down, and Scaling Out. Scaling Down makes models smaller and faster, enabling them to run on everyday devices. Scaling Out connects many specialized models or AI agents together, allowing them to collaborate like teams. This shift offers a more flexible and sustainable path forward for AI.

We show how this broader approach to scaling can unlock breakthroughs in healthcare, smart factories, and online platforms like TikTok and YouTube, where AI needs to be responsive, personal, and fast. Our framework also addresses key concerns like fairness, energy use, and ethical deployment.

Ultimately, we argue that the future of AI will depend not just on building bigger models, but on building better systems—ones that are distributed, efficient, and ready to work in the real world.

Primary Area: Research Priorities, Methodology, and Evaluation

Keywords: AI Scaling, Scaling Up, Scaling Down, Scaling Out, Artificial General Intelligence

Submission Number: 353