OATH-Frames: Characterizing Online Attitudes towards Homelessness via LLM Assistants

Anonymous

Abstract: Homelessness in the U.S. is widespread, eliciting complex attitudes among individuals (e.g critical as well as sympathetic), often expressed on social media. These attitudes are challenging to summarize at scale, further obfuscating the broader public opinion which is imperative for guiding public policy and reform efforts. Our work proposes an approach to enable a large-scale study on homelessness via two major contributions. First, with the help of domain experts in social work, we characterize Online Attitudes towards Homelessness in nine hierarchical frames (OATH-Frames) on a collection of 4K social media posts. Further, in an effort to ease the annotation, we employ GPT-4 as an LLM assistant to the experts; GPT-4 + Expert annotation presents an attractive trade off owing to a 6.5× speedup in annotation time despite only incurring a 2 point F1 difference in annotation performance. Our effort results in a collection of 8K social media posts labeled by domain experts (with and without GPT-4 assistance). Second, using predicted OATH-Frames on a Flan-T5-Large model trained on our data, we perform a large-scale analysis on 3.1M posts on homelessness. We find marked differences in perceptions towards homelessness between the east and west coast of the U.S. We also find that posts often pit people experiencing homelessness, specifically veterans, against immigrants and asylum seekers as being either more- or less-deserving of resources and aid.

Paper Type: long

Research Area: Computational Social Science and Cultural Analytics

Contribution Types: Publicly available software and/or pre-trained models, Data resources, Data analysis

Languages Studied: English

Preprint Status: We are considering releasing a non-anonymous preprint in the next two months (i.e., during the reviewing process).

A1: yes

A1 Elaboration For Yes Or No: Page 9

A2: yes

A2 Elaboration For Yes Or No: Page 9

A3: yes

A3 Elaboration For Yes Or No: Abstract, Section 1

B: yes

B1: yes

B1 Elaboration For Yes Or No: Section 2, 3

B2: n/a

B3: yes

B3 Elaboration For Yes Or No: Section 3

B4: yes

B4 Elaboration For Yes Or No: Section 2

B5: yes

B5 Elaboration For Yes Or No: Section 2

B6: yes

B6 Elaboration For Yes Or No: Section 2, Section 3, Section 4, Appendix A

C: yes

C1: yes

C1 Elaboration For Yes Or No: Section 3, Appendix A

C2: yes

C2 Elaboration For Yes Or No: Appendix A

C3: yes

C3 Elaboration For Yes Or No: Section 2, 3, 4

C4: yes

C4 Elaboration For Yes Or No: Appendix A

D: yes

D1: yes

D1 Elaboration For Yes Or No: Section 2, Page 12

D2: yes

D2 Elaboration For Yes Or No: Section 2, Page 12

D3: n/a

D4: yes

D4 Elaboration For Yes Or No: Page 9

D5: yes

D5 Elaboration For Yes Or No: Section 2

E: no

E1: n/a