vEpiNet: A multimodal interictal epileptiform discharge detection method based on video and electroencephalogram data

Nan Lin, Weifang Gao, Lian Li, Junhui Chen, Zi Liang, Gonglin Yuan, Heyang Sun, Qing Li, Jianhua Chen, Liri Jin, Yan Huang, Xiangqin Zhou, Shaobo Zhang, Peng Hu, Chaoyue Dai, Haibo He, Yisu Dong, Liying Cui, Qiang Lu

Abstract: Highlights•Propose a multimodal deep learning-based diagnosis model to extract features from both EEG and video signals.•Introduce a novel patient detection algorithm that is able to accurately identify patient targets in videos/images.•Design a method to integrate the features extracted from the electrodes and the video.•Invent a visual IED detection software that utilizes the proposed multimodal model to present the detection results in prospective testing visually.