Patch Explorer: Interpreting Diffusion Models through Interaction

Imke Grabe, Jaden Fiotto Kaufman, Rohit Gandikota, David Bau

Keywords: diffusion models, interface, interpretability

TL;DR: Patch Explorer is an interface to visualize and manipulate the cross-attention heads in a diffusion model.

Abstract: We introduce Patch Explorer, an interactive interface for visualizing and manipulating the patches as they are processed by cross-attention heads. Built on interventions via NNsight, our interface lets users inspect and manipulate individual attention heads over layers and timesteps. Interaction via the interface reveals that attention heads independently capture semantics, like a unicorn's horn, in diffusion models. Next to offering a way to analyze its behavior, users can also intervene with Patch Explorer to edit semantic associations within diffusion models, like adding a unicorn horn to a horse. Our interface also helps understand the role of a diffusion timestep through precise interventions. By providing a visualization tool with interactivity based on attention heads, we aim to shed light on their role in generative processes.

Public: Yes

Submission Number: 7