Connecting humans and machines: Deep integration of advanced HCI in intelligent engineering

Ching-Hung Lee, Fan Li, Ming-Chuan Chiu, Amy J. C. Trappey, Edward Huang, Pisut Koomsap

Abstract: Insufficient vigilance towards surrounding construction hazards gives rise to numerous construction accidents resulting in injuries and even fatalities, such as severe collisions with construction machines and robots. Striving for proactive safety management, vigilance recognition for construction workers is crucial. Recent endeavors incorporating EEG and deep learning represent remarkable advancements in recognition efficiency and accuracy, compared to conventional self-report questionnaires. However, the scarcity of accurately labeled EEG data, stemming from practical constraints such as experimental costs, limits model performance and generalizability. While transfer learning demonstrates potential in addressing this scarcity, the absence of exploration into EEG-based recognition for construction workers represents a critical research gap. This paper proposes an inductive transfer learning-based framework, encompassing: 1) a pre-trained emotional recognition model leveraging DEAP dataset to extract feature representations, 2) an EEG dataset for vigilance recognition in construction task accurately labeled by HRV-based approach, and 3) domain knowledge transfer achieved by transfer layers and target model development through fine-tuning. Rigorous evaluation primarily reveals a substantial boost in test accuracy for vigilance recognition, rising from 64.66% to 92.29% by leveraging transfer learning. Conclusively, this paper primarily presents a viable approach to enhance EEG-based safety management in construction, addressing limited availability of high-quality data for specific tasks through transfer learning. The developed recognition method excels at accurately detecting workers’ vigilance during continuous tasks, enabling construction managers to implement targeted interventions to minimize safety risks.