# Correlative Information Maximization Based Biologically Plausible Neural Networks for Correlated Source Separation

This repository includes the implementation of the Correlative Information Maximization Based Biologically Plausible Neural Networks for Correlated Source Separation, which is submitted to ICLR 2023. All the codes are written in Python 3. This paper proposes biologically plausible neural networks for blind separation of correlated sources exploiting prior domain assumptions via an information maximization criterion.

## CorInfoMax Neural Networks

CorInfoMax NN for Canonical Polytope Representation  |  CorInfoMax NN for "Feature-Based" Polytope Representation
:-------------------------:|:-------------------------:
![Sample Network Figures](./Figures/CorInfoMaxHRep3.png)   |  ![Sample Network Figures](./Figures/CorInfoMaxGen.png)

## Simulations In the Paper

All the simulation codes for the paper manuscript are included inside the folder "Simulations". The subfolders are named accordingly, e.g., "SparseNoisy" folder contains the experiments for the sparse source separation simulations in the paper. The jupyter notebooks inside the folder "Simulations/AnalyzeSimulationResultsFinal" illustrates the plots and tables presented in the paper. For example, the notebook "PlotSimulationResults_GeneralPoly.ipynb" includes the plots for the Appendix D.2.4. To replicate the figures in this specific notebook, you need to follow the below steps,

 * Run both python scripts in the folder "Simulations/GeneralPolytope" with the following commands:

    ``` python CorInfoMax_GeneralPolytope5dimV2.py```

    ``` python CorInfoMax_GeneralPolytope5dimV3.py```

 * When you run these python simulations, the following two pickle files will be created which contains the SINR results of each algorithm. 
    * "Simulations/Results/simulation_results_general_polytope_5dimV2.pkl"
    * "Simulations/Results/simulation_results_general_polytope_5dimV3.pkl"

 * The jupyter notebook "Simulations/AnalyzeSimulationResultsFinal/PlotSimulationResults_GeneralPoly.ipynb" reads the above pickle files and visualize the results. Moreover, the performances of the baseline algorithms are also reported.

To replicate each simulation in the paper, you can adapt the above procedure for the other sections (for antisparse, sparse, and simplex blind source separation (BSS) simulations). The experiment procedure for video separation is included in "Simulations/VideoSeparation" as a separate readme file. The sparse dictionary learning experiment is located in "Simulations/SparseDictionaryLearning", and the notebook inside this folder produces the sparse receptive fields from prewhitened Olshaussen's image pathces (you need "imagepatcheselfwhitened.mat" file whereas it is not included here due to its file size).

# Python Version and Dependencies

* Python Version : Python 3.8.12

* pip version : 21.2.4

* Required Python Packages : Specified in requirements.txt file.

* Platform Info : OS: Linux (x86_64-pc-linux-gnu) CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz

# Folder Contents

## src
This file is the source code for each BSS algorithm we used in the paper. The following is the full list.

Python Script         |  Explanation
:--------------------:|:-------------------------:
BSSbase.py            | Base class for blind source separation algorithms
CorInfoMaxBSS.py      | Our proposed correlative information maximization (framework) neural networks 
BCA.py                | Biologically plausible online bounded component analysis (BCA) implementation
WSMBSS.py             | Implementation of Weighted similarity mathcing-based (WSM) determinant maximization neural networks for blind separation of correlated sources
BSMBSS.py             | Implementation of Bounded similarity matching (BSM) for uncorrelated antisparse sources
NSMBSS.py             | Implementation of Nonnegative similarity matching (NSM) for uncorrelated nonnegative sources
LDMIBSS.py            | Implementation of Log-det (LD-) Mutual Information maximization (LD-InfoMax) framework for blind separation of correlated sources (This is the batch version of our proposed online biologically plausible framework)
PMF.py                | Implementation of Polytopic Matrix Factorization
ICA.py                | Implementation of several independent component analysis frameworks 
bss_utils.py          | Utility functions for blind source separation experiments
dsp_utils.py          | Utility functions for digital signal processing
polytope_utils.py     | Utility functions for polytope operations
visualization_utils.py| Utility functions for visualizations
numba_utils.py        | Utility functions using numba library of Python
general_utils.py      | Other utility functions

## Notebook_Examples
This file includes the jupyter notebook experiments of the algorithms CorInfoMax, LD-InfoMax, BCA, PMF, BSM, NSM, and ICA. The subfolder names are given accordingly, and notebooks are presented to experiment and visualize BSS settings with different algorithms. These notebooks can also be used for debugging and tutorials.

## Simulations
Each simulation presented in the paper can be reproduced using the codes in this folder.

Simulation Folder                     |  Explanation
:------------------------------------:|:-------------------------:
AntisparseCorrelated                  | (Signed) antisparse source separation simulations
NonnegativeAntisparseCorrelated       | Nonnegative antisparse source separation simulations
SparseNoisy                           | Sparse source separation simulations
NNSparseNoisy                         | Nonnegative sparse source separation simulations
SimplexNoisy                          | Simplex source separation simulations
SparseDictionaryLearning              | Sparse dictionary learning experiment
General_Polytope                      | A BSS simulation on a 5-dimensional identifiable polytope presented in the paper
VideoSeparation                       | Video separation demo code
ImageSeparation                       | Image separation demo (not presented in the paper)
4PAM_DigitalComm                      | A BSS simulation setting with 4PAM digital communication signals
AblationStudies                       | Ablation studies on hyperparameter selections
AnalyzeSimulationResultsFinal         | Producing the plots and table for simulation results


