TeNet: Text-to-Network for Compact Policy Synthesis

Ariyan Bighashdel, Kevin Sebastian Luck

Keywords: Language grounding, Hypernetworks, Policy generation, Meta-learning, Robotics

Abstract: Large vision-language-action (VLA) models such as PaLM-E, SayCan, and RT-2 enable robots to follow natural language instructions, but their billions of parameters make them impractical for high-frequency real-time control. At the other extreme, compact sequence models such as Decision Transformers are efficient but not language-enabled, relying on trajectory prompts and failing to generalize across diverse tasks. We propose TeNet (Text-to-Network), a framework that bridges this gap by instantiating lightweight, task-specific policies directly from natural language descriptions. TeNet conditions a hypernetwork on LLM-derived text embeddings to generate executable policies that run on resource-constrained robots. To enhance generalization, we introduce grounding strategies that align language with behavior, ensuring that instructions capture both linguistic content and action semantics. Experiments on state-based Mujoco and Meta-World benchmarks show that TeNet achieves robust performance in multi-task and meta-learning settings while producing policies that are orders of magnitude smaller. These results position language-enabled hypernetworks as a promising paradigm for compact, language-conditioned control in state-based simulation, complementary to large-scale VLAs that tackle vision-based robotics at massive scale.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 8704