CryChime: When Large Language Models Learn to Listen to Distant Cries - A Counterfactual PEFT Framework for Urgent Need Detection in Disaster Social Media

Junhong Cai, Geng Zhao, Jiaxin Li

Abstract: In recent years, detecting instantaneously expressed urgent needs or requests in disaster-related posts from disaster-affected social media users has become crucial for disaster response and recovery. To address the gap that the performance of current urgent need detectors based on large language models (LLMs) is below requirements on this task from the domain of disaster response, we propose a novel insight: decomposing and inducting post content expressing disaster-induced urgent needs, into disaster event statements and disaster-induced appeals. The former, widely present and highly coarse-grained homogeneous across disaster-related posts, tends to introduce event-induced model bias leading to false recalls; while the latter, characterized by highly personalized, fine-grained and subjective phrasing, often challenge LLMs to allocate appropriate attentions to the corresponding tokens. In light of this, we propose CryChime, a novel model-agnostic parameter-efficient fine-tuning (PEFT) framework. CryChime represents disaster event statements in a bootstrapping style, and then removes the event-induced bias by orthogonal LoRA-based counterfactual learning. As fine-tuning steps increase, CryChime gradually disentangles the domain knowledge for understanding disaster event statements and disaster-induced appeals in candidate posts, then collaboratively leverage them in performing better urgent need detection. Experimental results on two benchmark datasets show that, compared to the strong baselines, CryChime can more effectively listen to the distant cries from the disaster-affected users. Our instruction-tuning data examples will be released in the further preprint version.

Submission Number: 291