# Optimal Lobotomizing: Exploring the effects of model compression on memorization in language models
Author: @spacemanidol

Language models have proven to be incredibly effective methods for language understanding and generation. As they are trained on massive textual datasets they memorize 

### Method

### Prep and Data Gen
1. Find Datasets that focuses on memorization for decoder and encoder models(GPT-NEO)
### Experiments
1. Train models
2. Prune attention heads
3. Prune layers
4. Unstructured pruning



