Language as Kernels
Primary Area: transfer learning, meta learning, and lifelong learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Zero-shot learning, LLMs, Kernel Machines
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: In the realm of natural language understanding, the synergy between large language models (LLMs) and prompt engineering has unfurled an impressive tapestry of performance. Nonetheless, this prowess has often been overshadowed by the formidable computational resource requirements, rendering LLMs inaccessible in resource-constrained milieus. In this study, we embark on a journey to reconcile this paradox by introducing a nimble and elegant solution --- the kernel machine paradigm. Within these hallowed pages, we present a compelling proof, demonstrating the mathematical equivalence of zero-shot learning and kernel machines. This novel approach, marked by its computational thriftiness, bestows upon us the ability to harness the latent potential of LLMs, even when confined to the humble CPUs. The marriage of this approach with neural nets, renowned for their boundless abstraction capabilities, culminates in remarkable accomplishments with in the realm of language understanding. Our paramount contribution lies in unveiling a path less traveled, where the integration of kernel machines and LLMs unveils a promising vista, enabling the realization of sophisticated language processing tasks in resource-constrained environments.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 655
Loading