# README

This readme contains short descriptions of each file in the repository.

## Code files
**Horn.py**

This file implements the algorithm for learning the Horn Envelope first introduced [in this paper](https://www.sciencedirect.com/science/article/pii/S0888613X23001573). This code abstracts away the implementation of the membership and equivalence queries such that you can re-use this code for models that require different ways of implementing the membership and equivalence queries.

**modern_model.py**

Implements the direct EQ approach for generative langauge models where Equivalence queries are implemented directly. Has a parameter for running the *sampling method* of simulating the equivalence queries with generative models. The *sampling method* was only used with the Mistral7B model as larger models did not complete within the 24hour time limit.

**masked_model.py**

Implements the *sampling method* from the paper by Blum et.al (2024) which simulates the equivalence queries with membership queries on randomly generated examples. This is used with masked models such as RoBERTa-base and RoBERTa-large.

**Parsers.py**

Helper for translating between natural language and logical expressions.

**config.py**

Container for all gender-occupation specific prompts. This was done in an effort to make the code re-usable with different scenarios in the future.

## data folder
Within the data folder is all files connnected to Wikidata such as scripts for extracting information from Wikidata, and the datafiles extracted. There is also a jupyter notebook that was used to create one of the figures in the appendix.

## results folder
Within the results folder are all files connected to the rules extracted from the language models stored as json files. Additionally there is a jupyter notebook which was used to analyse the results and make the plots found in the paper.

## Run the code
The experiments were run on a HPC using slurm files to queue jobs. The RoBERTa-base and RoBERTa-large models were loaded using huggingfaces pipline. The Mistral(7B), Mistral(24B), and DeepSeek-R1(8B)-Distill_Llama models were loaded locally on the server from huggingface.

Links to [Mistral(7B)](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2), [Mistral(24B)](https://huggingface.co/mistralai/Mistral-Small-24B-Instruct-2501), and [DeepSeek(8B)](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B)

The jobs were then queued using the slurm scripts which have been removed because of identifiable information. An anonynous example slurm script, *horn_rules.slurm*, has been left in the repository for illustration purposes.
