ICML 2024 Workshop TF2M Submissions

Zero-Shot Generalization of GNNs over Distinct Attribute Domains
Yangyi Shen, Beatrice Bevilacqua, Joshua Robinson, Charilaos Kanatsoulis, Jure Leskovec, Bruno Ribeiro
- Published: 18 Jun 2024, Last Modified: 17 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Understanding and Mitigating Tokenization Bias in Language Models
Buu Phan, Marton Havasi, Matthew J. Muckley, Karen Ullrich
- Published: 18 Jun 2024, Last Modified: 07 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Transformers need glasses! Information over-squashing in language tasks
Federico Barbero, Andrea Banino, Steven Kapturowski, Dharshan Kumaran, João Guilherme Madeira Araújo, Alex Vitvitskyi, Razvan Pascanu, Petar Veličković
- Published: 18 Jun 2024, Last Modified: 19 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation
Chanwoo Park, Mingyang Liu, Dingwen Kong, Kaiqing Zhang, Asuman E. Ozdaglar
- Published: 18 Jun 2024, Last Modified: 15 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Implementability of Information Elicitation Mechanisms with Pre-Trained Language Models
Zachary Robertson, Hannah Cha, Andrew Sheha, Sanmi Koyejo
- Published: 18 Jun 2024, Last Modified: 03 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Setting the Record Straight on Transformer Oversmoothing
Gbetondji Jean-Sebastien Dovonon, Michael M. Bronstein, Matt Kusner
- Published: 18 Jun 2024, Last Modified: 20 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Models
Sanae Lotfi, Yilun Kuang, Marc Anton Finzi, Brandon Amos, Micah Goldblum, Andrew Gordon Wilson
- Published: 18 Jun 2024, Last Modified: 24 Jul 2024
- TF2M 2024 Oral
- Readers: Everyone
Beyond Model Collapse: Scaling Up with Synthesized Data Requires Reinforcement
Yunzhen Feng, Elvis Dohmatob, Pu Yang, Francois Charton, Julia Kempe
- Published: 18 Jun 2024, Last Modified: 19 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Multilingual Compression Parity: How Efficiently Large Language Models Represent Information Across Languages?
Alexander Tsvetkov, Alon Kipnis
- Published: 18 Jun 2024, Last Modified: 19 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
One-Shot Safety Alignment for Large Language Models via Optimal Dualization
Xinmeng Huang, Shuo Li, Edgar Dobriban, Osbert Bastani, Hamed Hassani, Dongsheng Ding
- Published: 18 Jun 2024, Last Modified: 13 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Models That Prove Their Own Correctness
Noga Amit, Shafi Goldwasser, Orr Paradise, Guy N. Rothblum
- Published: 18 Jun 2024, Last Modified: 19 Jul 2024
- TF2M 2024 Oral
- Readers: Everyone
The Geometry of Categorical and Hierarchical Concepts in Large Language Models
Kiho Park, Yo Joong Choe, Yibo Jiang, Victor Veitch
- Published: 18 Jun 2024, Last Modified: 08 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Getting More Juice Out of the SFT Data: Reward Learning from Human Demonstration Improves SFT for LLM Alignment
Jiaxiang Li, Siliang Zeng, Hoi To Wai, Chenliang Li, Alfredo Garcia, Mingyi Hong
- Published: 18 Jun 2024, Last Modified: 20 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Unified Taxonomy in AI Safety: Watermarks, Adversarial Defenses, and Transferable Attacks
Grzegorz Gluch, Sai Ganesh Nagarajan, Berkant Turan
- Published: 18 Jun 2024, Last Modified: 31 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
How Transformers Learn Diverse Attention Correlations in Masked Vision Pretraining
Yu Huang, Zixin Wen, Yuejie Chi, Yingbin Liang
- Published: 18 Jun 2024, Last Modified: 16 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
In-Context Learning with Representations: Contextual Generalization of Trained Transformers
Tong Yang, Yu Huang, Yingbin Liang, Yuejie Chi
- Published: 18 Jun 2024, Last Modified: 20 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Modeling the Plurality of Human Preferences via Ideal Points
Daiwei Chen, Yi Chen, Aniket Rege, Ramya Korlakai Vinayak
- Published: 18 Jun 2024, Last Modified: 07 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Efficient Document Ranking with Learnable Late Interactions
Himanshu Jain, Ziwei Ji, Sashank J. Reddi, Ankit Singh Rawat, Felix Yu, Aditya Krishna Menon, Sadeep Jayasumana
- Published: 18 Jun 2024, Last Modified: 03 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems
Bingcong Li, Liang Zhang, Niao He
- Published: 18 Jun 2024, Last Modified: 18 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Preference Learning Algorithms Do Not Learn Preference Rankings
Angelica Chen, Sadhika Malladi, Lily H Zhang, Xinyi Chen, Qiuyi Zhang, Rajesh Ranganath, Kyunghyun Cho
- Published: 18 Jun 2024, Last Modified: 21 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Fine-Tuning Large Language Models with User-Level Differential Privacy
Zachary Charles, Arun Ganesh, Ryan McKenna, Hugh Brendan McMahan, Nicole Elyse Mitchell, Krishna Pillutla, J Keith Rush
- Published: 18 Jun 2024, Last Modified: 20 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Mission Impossible: A Statistical Perspective on Jailbreaking LLMs
Jingtong Su, Julia Kempe, Karen Ullrich
- Published: 18 Jun 2024, Last Modified: 19 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression
Xingwu Chen, Lei Zhao, Difan Zou
- Published: 18 Jun 2024, Last Modified: 03 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
Self-Play Preference Optimization for Language Model Alignment
Yue Wu, Zhiqing Sun, Huizhuo Yuan, Kaixuan Ji, Yiming Yang, Quanquan Gu
- Published: 18 Jun 2024, Last Modified: 03 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone
In-Context Learning from Training on Unstructured Data: The Role of Co-Occurrence, Positional Information, and Training Data Structure
Kevin Christian Wibisono, Yixin Wang
- Published: 18 Jun 2024, Last Modified: 22 Jul 2024
- TF2M 2024 Poster
- Readers: Everyone