BRAT: Bonus oRthogonAl Token for Architecture Agnostic Textual Inversion

TMLR Paper3104 Authors

Abstract: Textual Inversion remains a popular method for personalizing diffusion models, in order to teach models new subjects and styles. We note that textual inversion has been under-explored using alternatives to the UNet, and experiment with textual inversion with a vision transformer. We also seek to optimize textual inversion using a strategy that does not require explicit use of the UNet and its idiosyncratic layers, so we add bonus tokens and enforce orthogonality. We find the use of the bonus token improves adherence to the source images and the use of the vision transformer improves adherence to the prompt.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: I accidentally used my GitHub link in the abstract, containing my first and last name in the last revision, so I omitted that.

Assigned Action Editor: ~Russell_Tsuchida1

Submission Number: 3104