# NetworkLassoBandits

Project for multi-task bandits with the network lasso

## File description

* `bandit.py`: classes representing multi-armed bandit environments.
* `policy.py`: clasees representing policies/agents.
* `experiment.py`: classes representing sequential interactions between an agent/ a policy and a bandit environment
* `Network_Lasso.ipynb`: testing the code for the network lasso optimization problem using ADMM
* `network_lasso.py`: classes defining the network lasso solver, along with other classes used by it.
* `utils.py`: additional miscellaneous classes and functions used by the scripts.
* `main_graph_bandits.py`: script for running experiments
* `Primal_Dual_vs_ADMM.py`: comparing the solutions of both algorithms

## TODO

* PrimalDual solver class:
  
  * [x] Remove "_new", "_sparse" or any suffix that is not anymore needed.
  * [x] Simplify incidence matrix construction since networkx contains a one line function to do it
  * [x] Remove X, y, alpha, lambda from the solver attributes, and add them as arguments when needed. Indeed, they are always changing, and they characterize the optimization problem, and not its solver
  * [x] Remove Theta_true as an attribute as it is not used at all by the class
  * [x] Remove matrix inversion from the class initialization as the matrix changes with each problem, and add lambda and X to its arguments
  * [x] reduce T_i to a scalar for one user (instead of a vector), and to a vector for several ones (instead of a matrix). Indeed, for the moment, it is just the degree vector up to a constant and up to a repetition over dimensions, which is redundant.
  * [x] Remove T_B_tranpose function and replace it with one line as it simply results from the T construction
  * [x] Factorize the construction of T, Sigma and other graph quantities for modularity.
  * [x] Pass second moment matrix and yX tensors to account for different numbers of users, and make the resulting changes (for instance, in the matrix inversion function)
  * [x] Test for different number of observations per user

* Primal dual class solver in the bandit case
  
  * [x] Add initialization as argument
  * [x] Solver takes inverse directly as an argument

* Comparing ADMM and Primal Dual solvers:
  
  * [x] Ensure comparable stopping criteria
  * [ ] Look for literature stopping criterion for Primal Dual
  * [x] option to initialize solvers with a context matrix
  * [ ] Roles of $\tau$ and $\rho$
  * [x] Make sure the same cost is minimized
  * [x] Parallelize repetitions with `joblib`
  * [ ] Test whether PrimalDual stopping earlier is better
  * [ ] Test effect of warm start or not (removing warm start can be done by setting the solver parameters to a random initialization at the end of optimization procedure

* [x] Create script for postprocessing results

* [x] Code acceleration: use single precision

## Comparison to other baselines

* [x] For a fair comparison with methods that start with a graph, give the same graph we start with. For example, we give to the algorithm of Gentile et al. 2014 our graph instead of the complete graph.

## Experimental observations

### Particular case of 1 cluster

* Our algorithm seems to offer an advantage as:
  * the dimensions grows (e.g. 20 users, 20 dimensions),
  * the connectivity of the graph reduces (Erdos-Renyi with low connectivity or even path graph !).
  * For a constant signal over an SBM of 2 clusters that are not densely connected (p=0.8, q=0.1, M=20, d=10)
  * normalized and random walk laplacian (symmetrized) seem to be more robust to the choice of alpha.

### Execution time

```python
dtype= 'float32'
n_users = 200
dim = 100
n_arms_all= 500
n_arms = 50
horizon = 10000
sigma = 0.01
repetitions = 10
n_clusters = 10
random = default_rng(0)
eps = 1e-9
imbalance = 1.0
p = 0.4
q = 0.05
experiment_name = f"u{n_users}d{dim}h{horizon}c{n_clusters}i{imbalance}p{p}q{q}"
```

on i5 10 th Gen, 32 GB ram, 8 processors, 2 tasks in 501 minutes with 8 CPU in parallel

* ```python
  dtype= 'float32'
  n_users = 200
  dim = 20
  n_arms_all= 500
  n_arms = 50
  horizon = 4000
  sigma = 0.01
  repetitions = 10
  n_clusters = 10
  random = default_rng(0)
  eps = 1e-9
  imbalance = 1.0
  ```

312 minutes on 16 GB ram, work computer with i7, parllel 8 cores

Still without "convergence"" of the regret _!!!_

# TODO

* [ ] Add and comment on experimen
  
  * [x] high number of nodes, or high dim, or both
  
  * [x] Specify normalization
  - [ ] Add experimental details in appendix

* [x] comment on definitions of $\iota_G$ and $c_G$,
  
  - [x] $c_G$ fro, Ranjan, van Mieghem and maybe other
  
  - [x] $\iota_G$: Spielman ? that it is at most equal to the isoperimetric ratio for weightless graphs 
- [ ] Mention modularity after cluster assumption ?

- [ ] clear todo blocks
  
  - [ ] Add figures

- [x] Add proof on behavour of $c_G$ for complete graphs, that it is at most number of nodes, with required references. use result from Fontant et al.

- [x] Write decomposition lemma and explain.
  
  - [x] Write the general lemma
  
  - [x] After lemma 4, explain that using it is not different by adding zeros "padding"

- [ ] Fix $exp(-At)$ summation: b must be bounded at least to be able to sum

- [x] $\tilde O$ for regret for hiding logarithmic terms ?

- [ ] Complete writing related work, or balance it
  
  - [ ] Total variation on graphs

- [ ] Re-read
