# Unified Quantization & Spectral Analysis Framework for Vision-Language Models

This code contains the official implementation for the paper.

The framework provides a modular environment to evaluate, quantize, and analyze Vision-Language Models (VLMs) such as CLIP, SigLIP, ALIGN, and CoCa. It focuses on the interplay between low-bit quantization, feature redundancy, and spectral energy distribution.

---

## 📂 File Registry & Descriptions

### 🚀 Core Execution
*   **`main.py`** (formerly `run_experiments.py`): The primary entry point. Orchestrates the full experimental pipeline, including model loading, quantization, and cross-dataset evaluation. Supports dual-GPU offloading for teacher models.

### 🛠️ Quantization Core (`quantization/`)
*   **`apply.py`**: The high-level API "dispatcher." Contains the `apply_method` functions that handle the transformation of FP32 models into quantized versions (PTQ/QAT).
*   **`modules.py`**: Defines the quantization primitives. Includes `LSQ` autograd functions, `EnhancedFakeQuantizer`, and specialized layers like `QuantizedLinearLayer`, `IRM` (Information Rectification), and `OutlierAware` quantizers.
*   **`training.py`**: Contains the training loops for Quantization-Aware Training (QAT). Manages contrastive losses, feature distillation, and IRM optimization.
*   **`calibration.py`**: Handles the data-driven calibration phase. Computes MinMax scales, percentiles, and Smoothing factors (SmoothQuant).
*   **`utils.py`**: Structural utilities for graph surgery, such as recursively replacing standard `nn.Linear` layers with quantized counterparts while respecting exclusion lists.

### 📊 Benchmark & Analysis Scripts
*   **`benchmark_rank_sweep.py`**: Performs spectral saturation analysis. It measures how many singular values are required to maintain zero-shot accuracy across different quantization methods.
*   **`run_ood_benchmark.py`**: Evaluates Out-of-Distribution (OOD) detection performance using scores like MSP, Energy, MCM, and CLIPN.
*   **`run_robustness_benchmark.py`**: Tests model resilience against synthetic corruptions (ImageNet-C) and natural shifts (ImageNet-A/R).
*   **`run_spurious_benchmark.py`**: Probes spurious correlations using the CounterAnimal dataset to see if quantization exacerbates reliance on background cues.
*   **`run_adaptation_benchmark.py`**: Measures "Catastrophic Forgetting" and adaptation efficiency when fine-tuning quantized models on new domains.
*   **`frequency.py`**: Computes 2D-FFT on model activations to analyze how quantization affects high vs. low-frequency feature preservation.
*   **`redundance.py`**: Uses targeted dropout to probe the intrinsic redundancy of VLM architectures.

### ⚙️ Support Scripts (`scripts/`)
*   **`config.py`**: The central "Control Center." Contains all hyperparameters, model registries, method toggles, and dataset paths.
*   **`data_setup.py`**: Manages data loading for WebDatasets and standard ImageFolders. Includes the **Mock Data** logic for reviewers.
*   **`evaluation.py`**: Standardized zero-shot evaluation logic, including logit-scale tuning and ECE (Expected Calibration Error) calculation.
*   **`datasets_classes.py`**: Stores class names, prompt templates, and metadata for all supported datasets.

### 📊 Calibration And Quantization Data: 

We provide reviewers with sampels for reproducibility, that can be downloaded respectign double-blind procedure:

Please put the following shards in quantization/data_quantization

- https://drive.google.com/file/d/1QTvI1y_S1ro3MZvarIwXkGZCT50PD3W9/view?usp=sharing
- https://drive.google.com/file/d/1eyw3JaBcHhWvMim9G2olCSWzaz9EgauD/view?usp=sharing
- https://drive.google.com/file/d/1fACvVkRfH4TWLRCdU-uY53hIsJnzzyAA/view?usp=sharing




### 📊 Eveluation Data: 
- Please put the downstream datasets that you wish to evaluate on in data/
CIFARs should work out-of-the-box