NeurIPS 2024 Workshop Red Teaming GenAI Submissions

Decompose, Recompose, and Conquer: Multi-modal LLMs are Vulnerable to Compositional Adversarial Attacks in Multi-Image Queries
Julius Broomfield, George Ingebretsen, Reihaneh Iranmanesh, Sara Pieri, Ethan Kosak-Hine, Tom Gibbs, Reihaneh Rabbany, Kellin Pelrine
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
MedAIScout: Automated Retrieval of Known Machine Learning Vulnerabilities in Medical Applications
Athish Pranav Dharmalingam, Gargi Mitra
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Between the Bars: Gradient-based Jailbreaks are Bugs that induce Features
Kaivalya Hariharan, Uzay Girit
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Adversarial Negotiation Dynamics in Generative Language Models
Arinbjörn Kolbeinsson, Benedikt Kolbeinsson
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attacks
Nathalie Maria Kirch, Severin Field, Stephen Casper
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Does Refusal Training in LLMs Generalize to the Past Tense?
Maksym Andriushchenko, Nicolas Flammarion
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
A Formal Framework for Assessing and Mitigating Emergent Security Risks in Generative AI Models: Bridging Theory and Dynamic Risk Mitigation
Aviral Srivastava, Sourav Panda
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Decoding Biases: An Analysis of Automated Methods and Metrics for Gender Bias Detection in Language Models
Shachi H. Kumar, Saurav Sahay, Sahisnu Mazumder, Eda Okur, Ramesh Manuvinakurike, Nicole Marie Beckage, Hsuan Su, Hung-yi Lee, Lama Nachman
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
SkewAct: Red Teaming Large Language Models via Activation-Skewed Adversarial Prompt Optimization
Hanxi Guo, Siyuan Cheng, Guanhong Tao, Guangyu Shen, ZHUO ZHANG, Shengwei An, Kaiyuan Zhang, Xiangyu Zhang
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Keep on Swimming: Real Attackers Only Need Partial Knowledge of a Multi-Model System
Julian Collado, Kevin Stangl
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Plentiful Jailbreaks with String Compositions
Brian R.Y. Huang
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Curiosity-driven Red teaming for Large Language Models
Zhang-Wei Hong, Idan Shenfeld, Tsun-Hsuan Wang, Yung-Sung Chuang, Aldo Pareja, James R. Glass, Akash Srivastava, Pulkit Agrawal
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
TOFU: A Task of Fictitious Unlearning for LLMs
Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Chase Lipton, J Zico Kolter
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Rethinking LLM Memorization through the Lens of Adversarial Compression
Avi Schwarzschild, Zhili Feng, Pratyush Maini, Zachary Chase Lipton, J Zico Kolter
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Oral
- Readers: Everyone
Infecting LLM Agents via Generalizable Adversarial Attack
Weichen Yu, Kai Hu, Tianyu Pang, Chao Du, Min Lin, Matt Fredrikson
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Oral
- Readers: Everyone
A Realistic Threat Model for Large Language Model Jailbreaks
Valentyn Boreiko, Alexander Panfilov, Vaclav Voracek, Matthias Hein, Jonas Geiping
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Oral
- Readers: Everyone
Stability Evaluation of Large Language Models via Distributional Perturbation Analysis
Jiashuo Liu, Jiajin Li, Peng Cui, Jose Blanchet
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
LLM-Assisted Red Teaming of Diffusion Models through "Failures Are Fated, But Can Be Faded"
Som Sagar, Aditya Taparia, Ransalu Senanayake
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Semantic Membership Inference Attack against Large Language Models
Hamid Mozaffari, Virendra Marathe
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Lessons From Red Teaming 100 Generative AI Products
Blake Bullwinkel, Amanda J. Minnich, Shiven Chawla, Gary David Lopez Munoz, Martin Pouliot, Whitney Maxwell, Joris de Gruyter, Katherine Pratt, Saphir Qi, Nina Chikanov, Roman Lutz, Raja Sekhar Rao Dheekonda, Bolor-Erdene Jagdagdorj, Rich Lundeen, Sam Vaughan, Victoria Westerhoff, Pete Bryan, Ram Shankar Siva Kumar, Yonatan Zunger, Mark Russinovich
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs
Aly M. Kassem, Omar Mahmoud, Niloofar Mireshghallah, Hyunwoo Kim, Yulia Tsvetkov, Yejin Choi, Sherif Saad, Santu Rana
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
An Adversarial Perspective on Machine Unlearning for AI Safety
Jakub Łucki, Boyi Wei, Yangsibo Huang, Peter Henderson, Florian Tramèr, Javier Rando
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAI
Ambrish Rawat, Stefan Schoepf, Giulio Zizzo, Giandomenico Cornacchia, Muhammad Zaid Hameed, Kieran Fraser, Erik Miehling, Beat Buesser, Elizabeth M. Daly, Mark Purcell, Prasanna Sattigeri, Pin-Yu Chen, Kush R. Varshney
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraints
Jonathan Nöther, Adish Singla, Goran Radanovic
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Poster
- Readers: Everyone
Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
Rylan Schaeffer, Dan Valentine, Luke Bailey, James Chua, Zane Durante, Cristobal Eyzaguirre, Joe Benton, Brando Miranda, Henry Sleight, Tony Tong Wang, John Hughes, Rajashree Agrawal, Mrinank Sharma, Scott Emmons, Sanmi Koyejo, Ethan Perez
- Published: 09 Oct 2024, Last Modified: 03 Jan 2025
- Red Teaming GenAI Workshop @ NeurIPS'24 Oral
- Readers: Everyone