ICML 2023 Workshop ES-FoMO Submissions

SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores
Zhiyu Mei, Wei Fu, Guangju Wang, Huanchen Zhang, Yi Wu
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Oral
- Readers: Everyone
Fine-Tuning Language Models with Just Forward Passes
Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D. Lee, Danqi Chen, Sanjeev Arora
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Oral
- Readers: Everyone
Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning
Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers, William Yang Wang
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Constant Memory Attention Block
Leo Feng, Frederick Tung, Hossein Hajimirsadeghi, Yoshua Bengio, Mohamed Osama Ahmed
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Sequence Parallelism: Long Sequence Training from System Perspective
Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, Yang You
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
On IO-Efficient Attention Mechanisms: Context-Aware Bifurcated Attention and the Generalized Multi-Group Attention
Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Haifeng Qian, Hantian Ding, Qing Sun, Jun Wang, Liangfu Chen, Jiacheng Guo, Parminder Bhatia, Ramesh Nallapati, Sudipta Sengupta, Bing Xiang
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Oral
- Readers: Everyone
Scaling In-Context Demonstrations with Structured Attention
Tianle Cai, Kaixuan Huang, Jason D. Lee, Mengdi Wang
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone