Keypoint Annotation for Electrocommunication Source Separation with PIKAChU and RAIChU

Kaden Zheng, Sonja Johnson-Yu, Satpreet Harcharan Singh, Denis Turcu, Federico Pedraja, Pratyusha Sharma, Naomi Saphra, Nathaniel Sawtell, Kanaka Rajan

Keywords: electric fish, source separation, pose estimation, electrocommunication, AI for Science, inverse problems, biology, ethology, elephantfish

TL;DR: We introduce PIKAChU and RAIChU, physics-informed optimizers that auto-label weakly electric fish keypoints from electrode recordings, enabling scalable pose estimation and source separation without human annotation.

Abstract:

Source separation is a key step in understanding animal communication, particularly in acoustic and electric modalities.
Often, positional information is used to determine which animal emitted a given signal.
Automatic keypoint tracking algorithms can provide this information, but they sometimes require a prohibitively large number of hand-labeled examples.
We introduce a methodology for automatically labeling keypoints in weakly electric fish, which sense and communicate by emitting electric pulses.
Specifically, we invert a physics-based electrogeneration model to reconstruct the position of the fish which emitted a recorded pulse.
This approach allows us to transform inexpensive electric-signal recordings into pose estimates.
Our algorithm makes feasible the study of social interactions and communication in schools of weakly electric fish by increasing the efficacy of source separation in electrocommunication settings.
More broadly, we provide a general framework for augmenting pose estimation training data to improve source separation, with potential applications in acoustic modalities and beyond.

Submission Number: 32