# Spectral Principal Path (SPP) Toolkit

This repository contains code to support the empirical study of the paper:

**Title:** Convergence of Spectral Principal Paths: How Deep Networks Distill Linear Representations from Noisy Inputs  
**Conference:** NeurIPS 2025 (Under Review)

---

## 🔍 Overview

This project provides tools for understanding how neural networks form robust, linear concept representations by propagating signals through *spectral principal paths*—dominant directions in the singular spectrum of each layer.

Inspired by the **Linear Representation Hypothesis (LRH)** and introducing the **Input-Space Linearity Hypothesis (ISLH)**, the code aims to support representation-level analysis for interpretability and transparency.

---

## 📁 File Structure

```
.
├── 1_train_get_captions.py       # Preprocess and extract caption data from the COCO dataset
├── 2_process_initial_dataset.py  # Prepare dataset and representation structures
├── 3_representation_0316_2.ipynb # Analyze and visualize representation flow and alignment
├── repe/                         # (Placeholder) core implementation for RepE / SPP methods
```

---

## 🚀 Getting Started

### 1. Dependencies

Install Python packages with:

```bash
pip install torch torchvision pandas numpy matplotlib scikit-learn
```

Also ensure access to:
- COCO dataset for caption alignment
- Pretrained vision-language models (e.g., Idefics2, BLIP, etc.)

### 2. Running Scripts

**Step 1:** Train or extract captions  
```bash
python 1_train_get_captions.py
```

**Step 2:** Process dataset for representation analysis  
```bash
python 2_process_initial_dataset.py
```

**Step 3:** Use the notebook for visualization and spectral diagnostics  
Open `3_representation_0316_2.ipynb` in Jupyter and execute cell by cell.

---

## 📊 Representation Engineering

This code supports:
- Measuring **spectral similarity** between layers
- Tracking **singular value concentration** across depth
- Visualizing **LAT (Linear Attribution Tomography)** heatmaps
- Interpreting **concept direction stability** across layers
