Joint training does not transfer information between EEG and image classifiers

Hari Bharadwaj, Ronnie Wilbur, Jeffrey Mark Siskind

Keywords: EEG

TL;DR: We demonstrate that a widely published procedure for training EEG classifiers is ineffective.

Abstract:

Caution is necessary with machine-learning methods, and especially
computer-vision methods, to support brain processing claims from
neuroimaging data. Recent papers propose (i) a joint-training process
that does not use class information and (ii) a bidirectional transfer
of (a) image information to an EEG classifier and (b) brain-activity
information to an image classifier, such that the joint embedding
includes the shared image and brain-activity information. These
claims cannot be maintained: the training process is initialized with
class information, and joint training with EEG degrades rather than
improves the performance of the image encoder. Moreover, theoretical
solutions exist that entail no transfer beyond class information in
the joint embedding space.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 4790