ICLR 2024 Workshop ME-FoMo Submissions

Orchid: Flexible and Data-Adaptive Convolution for Sequence Modeling
Mahdi Karami, Ali Ghodsi
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
On the Representation Gap Between Modern RNNs and Transformers: The Curse of Memory Efficiency and the Fix of In-Context Retrieval
Kaiyue Wen, Xingyu Dang, Kaifeng Lyu
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
On Fairness Implications and Evaluations of Low-Rank Adaptation of Large Models
Ken Liu, Zhoujie Ding, Berivan Isik, Sanmi Koyejo
- Published: 04 Mar 2024, Last Modified: 10 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Can Mamba Learn How To Learn? A Comparative Study on In-Context Learning Tasks
Jongho Park, Jaeseung Park, Zheyang Xiong, Nayoung Lee, Jaewoong Cho, Samet Oymak, Kangwook Lee, Dimitris Papailiopoulos
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Preserving Principal Subspaces to Reduce Catastrophic Forgetting in Fine-tuning
Jörg K.H. Franke, Michael Hefenbrock, Frank Hutter
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Provably Robust DPO: Aligning Language Models with Noisy Feedback
Sayak Ray Chowdhury, Anush Kini, Nagarajan Natarajan
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Editing Large Language Models: Problems, Methods, and Opportunities
ICLR 2024 Workshop ME-FoMo Submission92 Authors
- Published: 04 Mar 2024, Last Modified: 14 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Is Mamba Capable of In-Context Learning?
ICLR 2024 Workshop ME-FoMo Submission90 Authors
- Published: 04 Mar 2024, Last Modified: 24 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
MathSensei: Mathematical Reasoning with a Tool-Augmented Large Language Model
DEBRUP DAS, Debopriyo Banerjee, Somak Aditya, Ashish Kulkarni
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Selecting Large Language Model to Fine-tune via Rectified Scaling Law
Haowei Lin, Baizhou Huang, Haotian Ye, Qinyu Chen, Zihao Wang, Sujian Li, Jianzhu Ma, Xiaojun Wan, James Zou, Yitao Liang
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Oral
- Readers: Everyone
Pre-training and In-context Learning IS Bayesian Inference a la De Finetti
Naimeng Ye, Hanming Yang, Andrew Siah, Hongseok Namkoong
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
QuRating: Selecting High-Quality Data for Training Language Models
Alexander Wettig, Aatmik Gupta, Saumya Malik, Danqi Chen
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Transformers Learn Nonlinear Features In Context
Juno Kim, Taiji Suzuki
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
GistScore: Learning Better Representations for In-Context Example Selection with Gist Bottlenecks
Shivanshu Gupta, Clemens Rosenbaum, Ethan R. Elenberg
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Scaling Laws for Downstream Task Performance of Large Language Models
Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, Sanmi Koyejo
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Lessons from Identifiability for Understanding Large Language Models
Patrik Reizinger, Szilvia Ujváry, Anna Mészáros, Anna Kerekes, Wieland Brendel, Ferenc Huszár
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
The Effect of Model Capacity on the Emergence of In-Context Learning
Berkan Ottlik, Narutatsu Ri, Daniel Hsu, Clayton Sanford
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
What makes vision transformers robust towards bit-flip attack?
Xuan Zhou, Souvik Kundu, Peter Anthony Beerel
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Asymmetry in Low-Rank Adapters of Foundation Models
Jiacheng Zhu, Kristjan Greenewald, Kimia Nadjahi, Haitz Sáez de Ocáriz Borde, Rickard Brüel Gabrielsson, Leshem Choshen, Marzyeh Ghassemi, Mikhail Yurochkin, Justin Solomon
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
"I'm not Racist but…": Discovering Bias in the Internal Knowledge of Large Language Models
Abel Salinas, Louis Penafiel, Robert McCormack, Fred Morstatter
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Best Arm Identification for Prompt Learning under a Limited Budget
Chengshuai Shi, Kun Yang, Jing Yang, Cong Shen
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Few-Shot Adaptation of Vision-Language Foundation Models via Dual-Path Inference
Ce Zhang, Simon Stepputtis, Katia P. Sycara, Yaqi Xie
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Fourier Circuits in Neural Networks: Unlocking the Potential of Large Language Models in Math Reasoning and Modular Arithmetic
Jiuxiang Gu, Chenyang Li, Yingyu Liang, Zhenmei Shi, Zhao Song, Tianyi Zhou
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability
Zhuoyan Xu, Zhenmei Shi, Yingyu Liang
- Published: 04 Mar 2024, Last Modified: 10 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone
Simple linear attention language models balance the recall-throughput tradeoff
Simran Arora, Sabri Eyuboglu, Michael Zhang, Aman Timalsina, Silas Alberti, James Zou, Atri Rudra, Christopher Re
- Published: 04 Mar 2024, Last Modified: 02 Apr 2024
- ME-FoMo 2024 Poster
- Readers: Everyone