NeurIPS 2023 Workshop WANT Submissions

Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators
Yaniv Blumenfeld, Itay Hubara, Daniel Soudry
- Published: 28 Oct 2023, Last Modified: 30 Nov 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
Training Bayesian Neural Networks with Sparse Subspace Variational Inference
Junbo Li, Zichen Miao, Qiang Qiu, Ruqi Zhang
- Published: 28 Oct 2023, Last Modified: 30 Nov 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
LightSeq: : Sequence Level Parallelism for Distributed Training of Long Context Transformers
Dacheng Li, Rulin Shao, Anze Xie, Eric P. Xing, Joseph E. Gonzalez, Ion Stoica, Xuezhe Ma, Hao Zhang
- Published: 28 Oct 2023, Last Modified: 01 Dec 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs
Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, Jianfeng Gao
- Published: 28 Oct 2023, Last Modified: 30 Nov 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
Mengzhou Xia, Tianyu Gao, Zhiyuan Zeng, Danqi Chen
- Published: 28 Oct 2023, Last Modified: 01 Dec 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
ReLoRA: High-Rank Training Through Low-Rank Updates
Vladislav Lialin, Sherin Muckatira, Namrata Shivagunde, Anna Rumshisky
- Published: 28 Oct 2023, Last Modified: 01 Dec 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
Generalisable Agents for Neural Network Optimisation
Kale-ab Tessera, Callum Rhys Tilbury, Sasha Abramowitz, Ruan John de Kock, Omayma Mahjoub, Benjamin Rosman, Sara Hooker, Arnu Pretorius
- Published: 28 Oct 2023, Last Modified: 23 Nov 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
ReffAKD: Resource-efficient Autoencoder-based Knowledge Distillation
Divyang Doshi, Jung-Eun Kim
- Published: 28 Oct 2023, Last Modified: 28 Nov 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
Sparse Backpropagation for MoE Training
Liyuan Liu, Jianfeng Gao, Weizhu Chen
- Published: 28 Oct 2023, Last Modified: 30 Nov 2023
- WANT@NeurIPS 2023 Oral
- Readers: Everyone
MatFormer: Nested Transformer for Elastic Inference
Fnu Devvrit, Sneha Kudugunta, Aditya Kusupati, Tim Dettmers, Kaifeng Chen, Inderjit S Dhillon, Yulia Tsvetkov, Hannaneh Hajishirzi, Sham M. Kakade, Ali Farhadi, Prateek Jain
- Published: 28 Oct 2023, Last Modified: 29 Nov 2023
- WANT@NeurIPS 2023 Oral
- Readers: Everyone
FlexTrain: A Dynamic Training Framework for Heterogeneous Devices Environments
Mert Unsal, Ali Maatouk, Antonio De Domenico, Nicola Piovesan, Fadhel Ayed
- Published: 28 Oct 2023, Last Modified: 23 Nov 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
AI4HPC: Library to Train AI Models on HPC Systems using CFD Datasets
Eray Inanc, Rakesh Sarma, Marcel Aach, Rocco Sedona, Andreas Lintermann
- Published: 28 Oct 2023, Last Modified: 14 Nov 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
ConcatPlexer : Additional Dim1 Batching for Faster ViTs
Donghoon Han, Seunghyeon Seo, Donghyeon Jeon, Jiho Jang, Chaerin Kong, Nojun Kwak
- Published: 28 Oct 2023, Last Modified: 28 Oct 2023
- WANT@NeurIPS 2023 Oral
- Readers: Everyone
Training and inference of large language models using 8-bit floating point
Sergio P. Perez, Yan Zhang, James Briggs, Charlie Blake, Josh Levy-Kramer, Paul Balanca, Carlo Luschi, Stephen Barlow, Andrew W Fitzgibbon
- Published: 28 Oct 2023, Last Modified: 27 Jun 2024
- WANT@NeurIPS 2023 Oral
- Readers: Everyone
Maestro: Uncovering Low-Rank Structures via Trainable Decomposition
Samuel Horváth, Stefanos Laskaridis, Shashank Rajput, Hongyi Wang
- Published: 28 Oct 2023, Last Modified: 28 Oct 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone
Scene-adaptive Knowledge Distillation for Sequential Recommendation via Differentiable Architecture Search
Lei Chen, Fajie Yuan, Jiaxi Yang, Chengming Li, Min Yang
- Published: 28 Oct 2023, Last Modified: 23 Nov 2023
- WANT@NeurIPS 2023 Poster
- Readers: Everyone