# Code for LORE: Jointly Learning the Intrinsic Rank and Ordinal Embedding for Perceptual Data

## Environment Setup
We use [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#activating-an-environment) to install and manage our environment.

1. Install the ```lore_env``` environment with the command ```conda env create -f env/environment.yml```.

2. Activate the environment with the command ```conda activate lore_env```

3. Install the modified cblearn package from source (included in the repo) with the command ```pip install -e env/cblearn```

***Note***. We highly recommend that you run our code with access to a GPU. We have lightly tested our code with CPU backends without it breaking but cannot guarantee that it will work in all cases.

## How to use LORE
The ```lore/``` folder contains the helper functions necessary to train using LORE. Specifically, it contains three important components.

- ```lore.f``` - which contains ```LogisticTripletLoss``` which defines the our modified triplet loss.
- ```lore.g``` - which contains the definitions for the various possible rank regularizations ```Linear``` and ```SchattenPUnnormed``` which are the nuclear norm and the Schatten-$p$-norm respectively.
- ```lore.triplet_algorithm``` which contains the training algorithm for LORE given an f and g. Our python implementation is heavily inspired by [this](https://github.com/Optimizater/Low-rank-optimization-with-active-manifold-identification) matlab implementation.

A full working demo of LORE compared to other OE baselines can be seen in ```demo.ipynb```.

## How to reproduce our results

### Figure 1
If you seek to re-run our experiments for figure 1 please do the following. (This script requires a GPU to run tractably)

1. Run ```python src/run_figure1.py```, this should create a csv file in ```results/figure1/```.
2. Step through the ```src/plot_figure1.ipynb``` notebook to obtain the summary statistics that we used to plot figure 1.

### Artificial Perceptual Experiment
If you seek to re-run our experiments for the artificial perceptual experiment please do the following. (This script requires a GPU to run tractably)

1. Run ```python src/run_artificial_perceptual_experiment.py```, this should create a pkl files in ```results/artificial_perceptual_experiment/```.
2. Step through the ```src/plot_artificial_perceptual_experiment.ipynb``` notebook to obtain the plots in our paper..

### Crowdsourced Datasets
If you seek to re-run our experiments for the Crowdsourced Datasets please do the following. (This script requires a GPU to run tractably). The three datasets are ```cars```, ```food100``` and ```musicians```.

1. Run ```python src/run_{dataset}.py```, this should create a csv file in ```results/dataset/```.
2. Step through the ```src/plot_{dataset}.ipynb``` notebook to obtain the summary statistics used in the table. 

For the interpretability plots for ```food100```, step through ```plot_food100_interpretability.ipynb``` using the embedding learned in the crowdsourced experiment.

### Other figures
We do not include code for the other figures in our paper as they are involve complex grid searches that need to be parallelized across multiple GPUs. However, as we have detailed in our appendix, all one needs to do is run over all parameter combinations for *intrinsic rank*, *number of percepts*, *noise*, *fraction of queries used*, *lambda* for LORE and *embedding dimension* for all other OEs. The procedure to run each individual parameter combination is identical to the one included in ```demo.ipynb```. For further operational details, we direct the reader to our paper.