Follow the Flow: On Information Flow Across Textual Tokens in Text-to-Image Models

ACL ARR 2025 July Submission613 Authors

Abstract: Text-to-image (T2I) models generate images by encoding text prompts into token representations, which then guide the diffusion process. While prior work has largely focused on improving alignment by refining the diffusion process, we focus on the textual encoding stage. Specifically, we investigate how semantic information is distributed across token representations within and between lexical items (i.e., words or expressions conveying a single concept) in the prompt. We analyze information flow at two levels: (1) In-item representation---whether individual tokens represent their lexical item, and (2) cross-item interaction---whether information flows across the tokens of different lexical items. We use patching techniques to uncover surprising encoding patterns. We find information is usually concentrated in only one or two of the item's tokens---For example, in the item "San Francisco's Golden Gate Bridge", the token "Gate" sufficiently captures the entire expression while the other tokens could effectively be discarded. Lexical items also tend to remain isolated; for instance, the token "dog" encodes no visual information about "green" in the prompt "a green dog". However, in some cases, items do influence each other's representation, often leading to misinterpretations---e.g., in the prompt "a pool by a table", the token pool represents a pool table after contextualization. Our findings highlight the critical role of token-level encoding in image generation, suggesting that misalignment issues may originate already during the textual encoding.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: Multimodality, Interpretability and Analysis of Models for NLP

Contribution Types: Model analysis & interpretability

Languages Studied: English

Reassignment Request Area Chair: This is not a resubmission

Reassignment Request Reviewers: This is not a resubmission

A1 Limitations Section: This paper has a limitations section.

A2 Potential Risks: No

A2 Elaboration: No risks in the interpretability work we did.

B Use Or Create Scientific Artifacts: Yes

B1 Cite Creators Of Artifacts: Yes

B1 Elaboration: 2.2

B2 Discuss The License For Artifacts: Yes

B2 Elaboration: Appendix. A.2

B3 Artifact Use Consistent With Intended Use: Yes

B3 Elaboration: Appendix. A.2

B4 Data Contains Personally Identifying Info Or Offensive Content: N/A

B5 Documentation Of Artifacts: Yes

B5 Elaboration: Appendix. A.2

B6 Statistics For Data: Yes

B6 Elaboration: Appendix. A.2

C Computational Experiments: Yes

C1 Model Size And Budget: Yes

C1 Elaboration: Appendix A.7

C2 Experimental Setup And Hyperparameters: Yes

C2 Elaboration: 2.2

C3 Descriptive Statistics: Yes

C3 Elaboration: 2.2

C4 Parameters For Packages: N/A

D Human Subjects Including Annotators: No

D1 Instructions Given To Participants: N/A

D2 Recruitment And Payment: N/A

D3 Data Consent: N/A

D4 Ethics Review Board Approval: N/A

D5 Characteristics Of Annotators: N/A

E Ai Assistants In Research Or Writing: Yes

E1 Information About Use Of Ai Assistants: Yes

E1 Elaboration: Appendix.1.8

Author Submission Checklist: yes

Submission Number: 613