# Towards Understanding Masked Distillation

## Visual Tokenizer (VQ-KD) Trained on ImageNet-1k


See [TOKENIZER.md](beit2/TOKENIZER.md) for more details.

## Masked Distillation Pre-training on ImageNet-1k

See [PRETRAINING.md](beit2/PRETRAINING.md) for detailed instructions.

## Fine-tuning models on ImageNet-1k

See finetune/train_dino.sh for detailed instructions.

## Primary visualization experiments

See visualization/ for detailed instructions.

## Hessian experiments

See PyHessian/ for detailed instructions.

