LLM-Assisted Fast and Customized Model Generation: A Preliminary Exploration

ICLR 2025 Conference Submission301 Authors

Keywords: Customized Model Generation, Hypernetworks, Large Language Models

Abstract: The rapid advancement of AI models has significantly impacted daily life, with Large Language Models (LLMs) playing a pivotal role in automating tasks and providing all-in-one solutions via API services. Meanwhile, there is a growing demand for private, resource-constrained, customizable, and high-performance models tailored to specific user needs. However, many users struggle to deploy these models due to limited resources or technical expertise. In this work, we try to address these challenges by focusing on two primary objectives: (1) to meet the specific needs of a broad range of users, and (2) to lower the barriers to AI model usage (\textit{e.g.}, resource constraints, technical expertise) for most users. In our preliminary exploration, we introduce FLAME, a framework that determines and generates AI models based on data or task descriptions provided by users. While existing solutions rely on pre-built models or extensive finetuning, FLAME leverages LLMs (\textit{e.g.}, GPT4-turbo) to capture data patterns and task features from user input, converting them into user requirements and structured metadata (\textit{e.g.}, task type, model architecture, and classifier dimension). Then, FLAME uses them as guidance to generate customized models by hypernetworks. This approach significantly improves efficiency, achieving up to 270x faster model production compared to finetuning-based paradigms (e.g., all-parameter and LoRA fine-tuning) while maintaining comparable performance across various tasks. We validate the effectiveness of FLAME through comprehensive experiments on Natural Language Processing (NLP), Computer Vision (CV), and tabular datasets, demonstrating its ability to quickly deliver high-quality, customized models.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 301