FACT: Learning Governing Abstractions Behind Integer Sequences

Peter Belcak, Ard Kastrati, Flavio Schenker, Roger Wattenhofer

Keywords: dataset, benchmark, integer, sequences, abstraction, learning, evaluation

TL;DR: A toolkit with a large dataset of integer sequences comprising both organic and synthetic entries, a library for data pre-processing and generation, a set of model performance evaluation tools, and a collection of baseline model implementations

Abstract: Integer sequences are of central importance to the modeling of concepts admitting complete finitary descriptions. We introduce a novel view on the learning of such concepts and lay down a set of benchmarking tasks aimed at conceptual understanding by machine learning models. These tasks indirectly assess model ability to abstract, and challenge them to reason both interpolatively and extrapolatively from the knowledge gained by observing representative examples. To further aid research in knowledge representation and reasoning, we present FACT, the Finitary Abstraction Comprehension Toolkit. The toolkit surrounds a large dataset of integer sequences comprising both organic and synthetic entries, a library for data pre-processing and generation, a set of model performance evaluation tools, and a collection of baseline model implementations, enabling the making of the future advancements with ease.

Author Statement: Yes

URL: https://github.com/FACT-Development-Team/FACT

Dataset Url:

https://doi.org/10.3929/ethz-b-000562705

Current versions of the FACTLIB and FACT Benchmarking Baseline Models can be found under DOIs 10.3929/ethz-b-000565638 and 10.3929/ethz-b-000565644, respectively. Alternatively, one can also visit the FACT GitHub repository at https://github.com/FACT-Development-Team/FACT and use it to interact with the code and the FACT Development Team.

License:

FACT Dataset:
This work is licensed under the Creative Commons Attribution-NonCommercial 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc/3.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

FACT Benchmarking and FACTLIB:
This work is licensed under the Creative Commons Attribution 4.0 International License.
To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

Contribution Process Agreement: Yes

In Person Attendance: Yes