# LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and the Importance of Object-based Representations

## File descriptions:

The `output-logs` directory includes two subdirectories, `direct-grid` and `object-based`, each representing different approaches in our experiment. Both contain:

- `ARC-subset`: Results for various prompting methods on the 50 ARC tasks subset, with filenames representing the prompting method as outlined in Table 1 and 3 of our paper.
- `1D-ARC` and `Vertical-Horizontal`: Results for generated tasks using the best performing prompting methods, with filenames based on the generated task types as seen in Table 2 and 4 of our paper. The exact mapping of the file names can be found in `dataset/generated/README.md`.
- `350_(direct_grid/object_based).csv`: Results for the remaining 350 tasks from the training set, using the best performing direct-grid/object-based prompts.
- `MiniARC_(direct_grid/object_based).csv`: Results for the MiniARC dataset, using the best performing direct-grid/object-based prompts.

`dataset/generated` contains the generated tasks for the 1D-ARC dataset and the vertical vs horizontal experiments.

`generator` contains the script for generating the 1D-ARC dataset

`main.py` contains the script to run various prompts on the ARC subset, as well as the object-based approaches for the generated tasks.
(the scripts for direct-grid approach for the 1D-ARC dataset can be found under `1d_scripts`)



