Problem-Oriented Segmentation and Retrieval: Case Study on Tutoring Conversations
Abstract: Many open-ended conversations (e.g., tutoring lessons or business meetings) revolve around pre-defined reference materials, like worksheets or meeting bullets. To provide a framework for studying such conversation structure, we introduce **Problem-Oriented Segmentation & Retrieval** (POSR), the task of _jointly_ breaking down conversations into segments and linking each segment to the relevant reference item. As a case study, we apply POSR to education where effectively structuring lessons around problems is critical yet difficult. We present **LessonLink**, the first dataset of real-world tutoring lessons, featuring 3,500 segments, spanning 24,300 minutes of instruction and linked to 116 SAT Math problems. We define and evaluate several joint and independent approaches for POSR, including segmentation (e.g., TextTiling), retrieval (e.g., ColBERT), and large language models (LLMs) methods. Our results highlight that modeling POSR as one joint task is essential: POSR methods outperform independent segmentation and retrieval pipelines by up to +$76$% on joint metrics and surpass traditional segmentation methods by up to +$78$% on segmentation metrics. We demonstrate POSR's practical impact on downstream education applications, deriving new insights on the language and time use in real-world lesson structures.**
*Pronounced as ``poser'' (\textipa{/\textprimstress poz\textschwa r/}), a perplexing problem.
**You can find our code and LessonLink dataset as a zip file in our submission. If our work is accepted, the public-facing manuscript will include a GitHub link.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: segmentation, discourse, math worksheets
Contribution Types: Data resources, Data analysis
Languages Studied: English
Submission Number: 1005
Loading