Unsupervised Learning of Neurosymbolic Encoders

Eric Zhan, Jennifer J. Sun, Ann Kennedy, Yisong Yue, Swarat Chaudhuri

Abstract: We present a framework for the unsupervised learning of neurosymbolic encoders, which are encoders obtained by composing neural networks with symbolic programs from a domain-specific language. Our framework naturally incorporates symbolic expert knowledge into the learning process, which leads to more interpretable and factorized latent representations compared to fully neural encoders. We integrate modern program synthesis techniques with the variational autoencoding (VAE) framework, in order to learn a neurosymbolic encoder in conjunction with a standard decoder. The programmatic descriptions from our encoders can benefit many analysis workflows, such as in behavior modeling where interpreting agent actions and movements is important. We evaluate our method on learning latent representations for real-world trajectory data from animal biology and sports analytics. We show that our approach offers significantly better separation of meaningful categories than standard VAEs and leads to practical gains on downstream analysis tasks, such as for behavior classification.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

This is the camera ready version of our paper. Major changes include:
- Updated Section 3 on Methods, notably Section 3.2 describing NEAR (previous Appendix Section D), Section 3.4 describing index and posterior collapse with Eq 7 added, as well as some clarifications to text and notation.
- Updated Section 4 on Experiments, notably Section 4.2 to describe role of neural encoder in synthetic experiments and Qualitative Interpretation paragraph, as well as added more details to the description of CalMS21 in Section 4.1.1.
- Updated conclusion (renamed to discussion) with comments on scalability, applicability, problem scope, and limitations.
- Added a link to the code in the abstract.

Code: https://github.com/ezhan94/neurosymbolic-encoders

Assigned Action Editor: ~Yujia_Li1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 306