# Lightweight CNNs Under A Unifying Tensor View
This repository serves as the official implementation for paper "Lightweight CNNs Under A Unifying Tensor View". 


## Getting Started

This repository has been tested with Ubuntu 20.04.1 LTS, Python 3.8, Pytorch 1.10.1 and CUDA 11.3.


```
conda create -n utv python=3.8
conda activate utv
pip install -r requirements.txt
conda install pytorch torchvision cudatoolkit=11.3 -c pytorch # Download Pytorch if you haven't
```

## PDP-VGG-16 & PDP-ResNet-34 on CIFAR-10

### How to Use

```
cd tensor_view
./scripts/train.sh
```

### Possible configurations:
```
--model (str): Specify which model to use. Options: ['vgg16', 'resnet34']
--init (str): Whether or not to use CPD initialization. Options; ['random', 'cpd']
--rank (int): Number of CPD ranks
--order (str): Specify the type of decomposition for CONV layers. Options: ['DW_PW', 'PW_DW', 'CPD_svd', 'CPD_tpm']
```

### Experimental Results

### VGG-16 
![](./tensor_view/figures/vgg16_ds_cifar10.png)
![](./tensor_view/figures/vgg16_cpd_cifar10.png)


### ResNet-34
![](./tensor_view/figures/resnet32_cpd_cifar10.png)


## Shift Layer Pruning

### How to Use

```
cd shift_pruning
./scripts/train.sh
```

### Possible configurations:
```
--expansion (int): Expansion ratio for the intermediate bottleneck
--prune (str): Specify the type of pruning used. Options; ['even', 'uneven']
--fold_bn (bool): Whether or not to apply BN folding before shift layer pruning
```

### Experimental Results

![](shift_pruning/figures/shift_prune.png)




## Acknowledgement
Our implementation adapts codes from [pytorch-cifar](https://github.com/kuangliu/pytorch-cifar), the official codes from [Tensorly](http://tensorly.org/stable/index.html) and the codebase released by authors [shiftresnet-cifar](https://github.com/alvinwan/shiftresnet-cifar). 
