Do you need help? Identifying and responding to pilots’ troubleshooting through eye-tracking and Large Language Model

Mengtao Lyu, Fan Li

Abstract: Highlights•Develops a novel VAM method to tokenize eye-tracking data for LLM input.•Proposes the use of empirical VAM to mitigate dependency on LLM training.•Proposes a framework of LLM prompt to process aviation-specific tasks.•Case study to find the optimal input for identifying pilots troubleshooting behavior.•Provides insights into more comprehensive understanding of pilots needs.

External IDs:doi:10.1016/j.ijhcs.2025.103617