This directory contains python scripts and data that can be used to
reproduce the results and figures reported in the paper entitled
"Whitened Self-Attention" and submitted for review to AAAI 2026.


DATA 
====
Subdirectory containing a single file with the complete works of
Charles Dickens (13.8m characters).

README.txt
==========
This file.

standard_self_attention.py
==========================
Code to compute the standard self-attention results for experiments
one and three in the paper. Configured to runs the third experiment in
the paper for equivalent capacity standard self-attention (ECSA).  To
run the first experiment (SA), change the value for n_embd on line 22
of the file from 276 to 256.

whitened_self_attention.py
==========================
Code to compute the whitened self-attention results for experiments
two and four in the paper. Configured to run the fourth experiment in
the paper for whitened self-attention half-batch (WSAHB).  To run the
second experiment (WSA), change the value for batch_size on line 35 of
the file from 128 to 256.

Figure3
=======
Subdirectory with code and data to regenerate Figure 3 in the paper.

Figure5
=======
Subdirectory with code and data to regenerate Figure 5 in the paper.
