Boolformer: Symbolic Regression of Logic Functions with Transformers

Stéphane d'Ascoli, Samy Bengio, Joshua M. Susskind, Emmanuel Abbe

Primary Area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: symbolic regression, boolean functions, transformers

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We introduce Boolformer, a Transformer model capable of inferring Boolean functions in symbolic form with state-of-the-art performance. We present applications to real-world binary classification tasks and inference of gene regulatory networks.

Abstract: In this work, we introduce the Boolformer, the first Transformer architecture trained to perform end-to-end symbolic regression of Boolean functions. First, we show that it can predict compact formulas for complex functions which were not seen during training, when provided a clean truth table. Then, we demonstrate its ability to find approximate expressions when provided incomplete and noisy observations. We compare it with classic machine learning approaches on a broad set of real-world binary classification datasets, demonstrating its potential as an interpretable alternative. Finally, we apply it to the widespread task of modelling the dynamics of gene regulatory networks. Using a recent benchmark, we show that Boolformer is competitive with state-of-the art genetic algorithms with a speedup of several orders of magnitude.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2514