BioTemporal-HAR: Optimizing Behavioral Biometrics through Signal Detection Theory and Multi-Sensor Fusion

Louis Rollet, Vincent Montero Fontaine

Keywords: Human Activity Recognition, Time Series Classification, Behavioral Biometrics, Wearable Sensing, Multi-Sensor Fusion, Signal Detection Theory, CNN-LSTM, Identity Verification, Feature Selection

TL;DR: BioTemporal-HAR studies whether compact phone-watch inertial models can recognize activities and verify identity while explicitly calibrating false-alarm risk with Signal Detection Theory.

Abstract:

Human Activity Recognition (HAR) has increasingly moved beyond coarse activity labeling toward behavioral biometrics, where the temporal structure of motion can support both activity recognition and passive identity modeling. This paper proposes BioTemporal-HAR, a lightweight multi-sensor framework for recognizing activities and evaluating biometric identity from smartphone and smartwatch inertial signals. The framework combines windowed signal processing, feature-selection experiments, compact CNN-LSTM sequence modeling, multi-sensor fusion, and Signal Detection Theory (SDT) calibration.

We target the WISDM Smartphone and Smartwatch Activity and Biometrics dataset, which contains accelerometer and gyroscope streams from 51 subjects performing 18 activities. The proposed pipeline evaluates three related tasks: activity classification, closed-set subject identification, and identity verification through genuine and impostor claim trials. To reduce overconfident conclusions, the evaluation emphasizes leakage-aware train/validation/test splits, phone-only versus watch-only versus fused sensor ablations, and comparisons between full feature sets and selected feature subsets.

A central goal of the work is to separate model discriminability from decision policy. For identity verification, SDT metrics such as sensitivity $d'$ and criterion $c$ are used to analyze the tradeoff between hit rate, miss rate, and false alarm rate. The expected contribution is a reproducible HAR and behavioral biometrics pipeline that tests whether compact multi-sensor models can preserve activity and identity information while making false-alarm behavior explicit under wearable-device constraints.

Submission Number: 1