# LoRA Meets Second-Order Optimization: Towards Optimal Low-Rank Updates

This repo contains the source code for experiments for our paper.

## Repository Overview

In this project, we experiments with GPT-2 fine-tuning, and Mix-of-Show fine-tuning.

1. GPT-2 Fine-Tuning (see [GPT-2/](GPT2) for experiment code.)
2. Mix-of-Show Fine-Tuning (see [Mix-of-Show/](Mix-of-Show) for experiment code.)
