# README

This project implements **HiQ-Lip**, a hybrid quantum-classical hierarchical approach for estimating the global Lipschitz constant of neural networks. Estimating the Lipschitz constant is crucial for understanding and improving the robustness and generalization capabilities of neural networks. However, exact calculation is NP-hard, and current semidefinite programming (SDP) methods face challenges like high memory usage and slow processing speed.

**HiQ-Lip** addresses these challenges by transforming the estimation into a Quadratic Unconstrained Binary Optimization (QUBO) problem and utilizes a Coherent Ising Machine (CIM) to solve it. The approach incorporates multilevel graph coarsening and refinement strategies to fit the constraints of contemporary quantum hardware.

Experimental evaluations on fully connected neural networks demonstrate that **HiQ-Lip** not only provides estimates comparable to state-of-the-art methods but also significantly accelerates the computation process. In a specific test involving a two-layer neural network with 256 hidden neurons, **HiQ-Lip** doubled the solving speed and provided a more accurate upper bound than the existing best method LiPopt. These findings highlight the promising use of small-scale quantum devices in advancing neural network estimation.

The implementation can be validated by running `mnist_eval.py`. The main components of the project are located in the `HiQLip` folder, which primarily contains two Python files.

---

## Project Structure

### Files

- **Solver and Refinement Classes** (`solver.py`):
  Implements the multilevel graph partitioning algorithm using coarsening and refinement techniques.
  
- **Embedding Coarsening Class** (`embedding_coarsening.py`):
  Performs graph coarsening using node embeddings to reduce the graph size while preserving structural properties.
  
- **Validation Script** (`mnist_eval.py`):
  A script to validate the implementation using the MNIST dataset.

---

## Components

### 1. Refinement Class (`Refinement`)

**Purpose**: Refines the current solution of the graph partitioning problem to iteratively improve the objective function.

**Key Methods and Attributes**:

- `__init__(self, G, spsize, solver, solution)`: Initializes the `Refinement` instance with the graph `G`, subproblem size `spsize`, solver name `solver`, and initial solution `solution`.
- `buildGain(self)`: Constructs the gain map, tracking the potential improvement (gain) of moving a node to the other partition.
- `calc_obj(self, G, solution)`: Calculates the objective value (e.g., cut size) for the current solution.
- `refine_coarse(self)`: Performs coarse refinement on the coarsest level of the graph using the specified solver.
- `refine(self)`: Iteratively refines the solution by generating and solving subproblems to improve the overall partition.
- `refineLevel(self)`: Performs a single level of refinement.
- `updateGain(self, S, changed)`: Updates the gain map based on nodes that have changed partitions.
- `randGainSubProb(self)`: Generates a subproblem focusing on nodes with the highest potential gain to optimize the refinement process.

**Workflow**:

- **Gain Map Construction**: Begins by building the gain map.
- **Subproblem Selection**: Iteratively selects subproblems consisting of nodes with the highest gains.
- **Subproblem Solving**: Solves each subproblem to potentially improve the overall solution.
- **Gain Map Update**: Updates the gain map based on the changes.
- **Iteration**: Repeats the process until convergence or reaching the maximum number of iterations.

---

### 2. Solver Class (`Solver`)

**Purpose**: Manages the overall multilevel graph partitioning process, including coarsening the graph, initial partitioning, and uncoarsening with refinement at each level.

**Key Methods and Attributes**:

- `__init__(self, adj, sp, solver, ratio)`: Initializes the `Solver` instance with the adjacency matrix `adj`, subproblem size `sp`, solver name `solver`, and coarsening ratio `ratio`.
- `solve(self)`: Executes the multilevel partitioning algorithm, including coarsening, initial partitioning on the coarsest graph, and uncoarsening with refinement at each level.

**Workflow**:

1. **Coarsening Phase**:
   - Repeatedly coarsens the graph using the `EmbeddingCoarsening` class until the graph size is below the specified threshold.
   - Stores each coarsened graph and the mapping between nodes in a hierarchy for use during uncoarsening.

2. **Initial Partitioning**:
   - Performs initial partitioning on the coarsest graph using the specified solver (e.g., `CIMSolver`).

3. **Uncoarsening and Refinement Phase**:
   - Projects the solution from the coarsest graph back to the original graph through the hierarchy.
   - Refines the partition at each level using the `Refinement` class to improve the objective function.

---

### 3. Embedding Coarsening Class (`EmbeddingCoarsening`)

**Purpose**: Reduces the graph size by coarsening while preserving essential structural properties using node embeddings.

**Key Methods and Attributes**:

- `__init__(self, G, d, shape, ratio)`: Initializes the `EmbeddingCoarsening` instance with the graph `G`, embedding dimension `d`, shape parameter `shape` (unused), and sparsification ratio `ratio`.
- `embed(self, nodes)`: Optimizes node positions in the embedding space to maximize distances between connected nodes.
- `sparsify(self)`: Reduces the number of edges by removing a fraction based on their lengths in the embedding space, while preserving total edge weight.
- `match(self)`: Matches nodes based on proximity in the embedding space to form clusters for coarsening.
- `coarsen(self)`: Executes the coarsening process by embedding optimization, sparsification, node matching, and building the coarse graph.

**Workflow**:

1. **Embedding Optimization**:
   - Positions nodes in a high-dimensional space reflecting their connectivity.
   - Iteratively adjusts positions to maximize distances between connected nodes.

2. **Graph Sparsification**:
   - Removes a fraction of edges based on their lengths in the embedding space.
   - Redistributes edge weights to preserve total weight.

3. **Node Matching**:
   - Pairs nodes close in the embedding space to form supernodes.
   - Handles unmatched nodes by pairing them with the nearest available nodes.

4. **Coarse Graph Construction**:
   - Builds a new coarse graph where each node represents a cluster (supernode) from the original graph.
   - Updates mappings between fine and coarse nodes for use during uncoarsening.

---

### 4. HiQLip Solver Function (`HiQLipsolver`)

**Purpose**: Serves as the main function to solve the graph partitioning problem by integrating all components.

**Parameters**:

- `w`: Adjacency matrix of the graph as a NumPy array.
- `userid`: User ID (default `'0'`).
- `sdkcode`: SDK code (default `'0'`).

**Returns**:

- `obj`: Objective value after partitioning.
- `solution`: Solution vector indicating the partition assignment for each node.

**Workflow**:

- Initializes the `Solver` class with the given adjacency matrix and parameters.
- Calls the `solve` method to perform multilevel partitioning.
- Adjusts the solution to be in the form of `-1` or `1` for partition assignments.
- Returns the objective value and the final solution vector.

---

## Usage Example

```python
import numpy as np

# Example adjacency matrix
w = np.array([
    [0, 1, 0, 0],
    [1, 0, 1, 1],
    [0, 1, 0, 0],
    [0, 1, 0, 0]
])

# Solve the partitioning problem
obj, solution = HiQLipsolver(w)

print("Objective Value:", obj)
print("Partition Solution:", solution)
```

---

## Validation

To validate the implementation, run the `mnist_eval.py` script, which applies the HiQ-Lip method to a neural network trained on the MNIST dataset.

```bash
python mnist_eval.py
```

---

## Dependencies

- **Python 3.x**
- **NumPy**: For numerical computations and array manipulations.
- **Networkit**: For efficient graph processing and algorithms.
- **Scikit-learn**: Specifically `KDTree` for efficient neighbor searches in the embedding space.
- **Random**: For reproducibility in random processes.

---

## Notes

- **Reproducibility**: The algorithm uses random processes (e.g., random initial embeddings and node shuffling). Setting random seeds ensures reproducibility.
- **Coarsening Importance**: The coarsening process reduces computation on large graphs by working on smaller, representative graphs.
- **Refinement Strategy**: Refinement iteratively improves the solution by focusing on nodes that can potentially provide the most significant gain in the objective function.
- **Future Enhancements**: The `EmbeddingCoarsening` class currently does not utilize the `shape` parameter; it is included for potential future enhancements.

---
