## Installation and use

To run experiments on XOR, MNIST-Addition, and BDD-OIA -- access the linux terminal and use the conda installation followed by pip3:

```
$conda env create -n rs python=3.8
$conda activate rs
$pip install -r requirements.txt
$pip install -r requirements.dev.txt
```

## BDD-OIA (2048)

We did not include the original dataset since the data dimension exceeds the limits supporte on OpenReview.

## A note on Laplace

In order to make Laplace work properly, you need to:

1. Go to `laplace/utils/feature_extractor.py`
2. Add the following line to `find_last_layer`:
  ```python
  if key != 'original_model.encoder.dense_c':
                continue
  ```
  so that the library takes `dense_c` as the last layer.

3. Go to `laplace/lllaplace.py`
4. Add the following line to `_nn_predictive_samples`:
  ```python
  self.model.model.model_possibilities = [None] * n_samples
  fs = list()
  for i, sample in enumerate(self.sample(n_samples)):
      vector_to_parameters(sample, self.model.last_layer.parameters())
      self.model.model.model_possibilities[i] = sample
      fs.append(self.model(X.to(self._device)).detach())
  ```
  so that the wrapper model knows to start tracking the output predictions.
  Where the added lines are: `self.model.model.model_possibilities = [None] * n_samples` 
  and `self.model.model.model_possibilities[i] = sample`

Moreover in the file `laplace/curvature/curvature.py` change this:

```python
def BCE_forloop(tar,pred):
    loss = F.binary_cross_entropy(tar[0, :4], pred[0, :4])
    
    for i in range(1,len(tar)):
        loss = loss + F.binary_cross_entropy(tar[i, :4], pred[i, :4])
    return loss 

def CE_forloop(y_pred, y_true):
    y_trues = torch.split(y_true, 1, dim=-1)
    y_preds = torch.split(y_pred, 2, dim=-1)

    loss = 0
    for i in range(4):
        
        true = y_trues[i].view(-1)
        pred = y_preds[i]

        loss_i = F.nll_loss( pred.log(), true.to(torch.long) )
        loss += loss_i / 4

        assert loss_i > 0, pred.log() 
    
    return loss

class CurvatureInterface:
    """Interface to access curvature for a model and corresponding likelihood.
    A `CurvatureInterface` must inherit from this baseclass and implement the
    necessary functions `jacobians`, `full`, `kron`, and `diag`.
    The interface might be extended in the future to account for other curvature
    structures, for example, a block-diagonal one.

    Parameters
    ----------
    model : torch.nn.Module or `laplace.utils.feature_extractor.FeatureExtractor`
        torch model (neural network)
    likelihood : {'classification', 'regression'}
    last_layer : bool, default=False
        only consider curvature of last layer
    subnetwork_indices : torch.Tensor, default=None
        indices of the vectorized model parameters that define the subnetwork
        to apply the Laplace approximation over

    Attributes
    ----------
    lossfunc : torch.nn.MSELoss or torch.nn.CrossEntropyLoss
    factor : float
        conversion factor between torch losses and base likelihoods
        For example, \\(\\frac{1}{2}\\) to get to \\(\\mathcal{N}(f, 1)\\) from MSELoss.
    """
    def __init__(self, model, likelihood, last_layer=False, subnetwork_indices=None):
        assert likelihood in ['regression', 'classification']
        self.likelihood = likelihood
        self.model = model
        self.last_layer = last_layer
        self.subnetwork_indices = subnetwork_indices
        if likelihood == 'regression':
            self.lossfunc = MSELoss(reduction='sum')
            self.factor = 0.5
        else:
            self.lossfunc = CrossEntropyLoss(reduction='sum')
            # self.lossfunc = CE_forloop
            self.factor = 1.
```

where for MNIST `self.lossfunc = CrossEntropyLoss(reduction='sum')` and for BDD `self.lossfunc = CE_forloop`.

If you got any problem with Laplace, contact us and we will provide the repository with all the changes required.
