Towards Sampling Data Structures for Tensor Products
Keywords: sampling, data structures, tensor products
Abstract: This paper studies the computational challenges of attention-based models in artificial intelligence by introducing innovative sampling methods to accelerate attention computation in large language models (LLM). Inspired by the recent progress of LLM in real-life applications, we introduces a streaming sampler question for attention setting. Our approach significantly reduces the computational burden of traditional attention mechanisms while maintaining or enhancing model performance. We demonstrate these methods' effectiveness from theoretical perspective, including space, update time. Additionally, our framework exhibits scalability and broad applicability across various model architectures and domains.
Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 12185
Loading