# Sparse Dep Additive Model with Interaction: Enhancing Interpretability and Predictability (SDAMI)

The README.md is used to guide the unser to use the source code for replicating the numerical simulation in SDAMI, which is submitted to ICLR 2026.

In our work, we propose the SDAMI, a framework that combines sparsity-driven feature selection with deep subnetworks for flexible function approximation. Unlike conventional deep learning models, which often function as black boxes, SDAMI explicitly disentangles main effects and interaction effects to enhance interpretability. At the same time, its deep additive structure achieves higher predictive accuracy than classical additive models.

![](./img/SDAM_architecture.png)

## Dependency
You can install the Python dependencies with
```
pip3 install -r requirements.txt
```

## Reproducing numerical experiments
After unzip the SDAMI_ICLR.zip, the user can type following command to navigate to numerical experiments folder.
```
>> cd ./src/script
```
In the manuscript, we provide SDAMI and its variant SDAMI$-p$ which corresponding to `run_SDAMI_p.sh` and `run_SDAMI.sh` respectively.

If you are running the experiment on slurm then 
```
>> bash [run_SDAMI.sh/ run_SDAMI_p.sh]
```
Please remember to revise the command in `.sh` file for different tasks.

For individual cases mentioned in the manuscipt, you can follow the command below to reproduce the results.
```
>> python3 main_SDAMI_p.py --data_name [case] --repeat [number of replication]
>> python3 main_SDAMI.py --data_name [case] --repeat [number of replication]
```

The argument for different cases are shown below

```
Note:
case 1: 'only_main'
case 2: 'weak_main'
case 3: 'inter_no_overlap'
case 4: 'inter_mild_overlap'
case 5: 'inter_strong_overlap'
case 6: 'only_inter'
```

The detailed formulation of each cases is given in section 5 on the manuscipt.

After the program is finished, the simulation results, selected feature from SDAMI, and information of the model will be stored as `{model_name}-{data_name}-n{training_data_size}.json`, `{model_name}-{data_name}-n{training_data_size}-feature.json`, and `{model_name}-{data_name}-n{training_data_size}.txt` under `.\src\output\`.
 
The summary of the model performance, TPR, and visualization can be found in `.\eval\MSE_TPR_FPR_Evaluation.ipynb`.

The data we use for simulation will be stored in `.\data\{data_name}-simulation_results.pt`. The dimension of the data is $(r,n,p)$ where $r$ is number of replication, $n$ is number of training datas, and $p$ is number of features. Thus, we can implement other sparse machine learning model as benchmark in our manuscript.

In the `./Experiment/Notebook/` folder, we provide a `Numerical_Simulation.ipynb` to demonstrate the case 3-6 in the manuscript. Besides, the visualization of identified main effects and interaction also be provided.

## Real Data Application
The real data analyses file can be found in `./Experiment/Notebook/` folder. The `Chip.ipynb, Diabetes.ipynb, V1Cell.ipynb` and `wine_bike_ca.ipynb` execute the corresponding Jupyter Notebook and demonstrate how SDAM works on these these datasets. The corresponding package can be found in [1][2][3].

Due to the restricted size of the uploading file, I store the dataset we use in numerical studies and real data analysis in the following [[link]](https://drive.google.com/drive/folders/141cvllIWb_oZ-EUq4GojLDx8OktCJRlL?usp=share_link).

### Interaction-Term Identification
To simplify the procedure, we have implemented the interaction identification for the provided dataset of Wine quality, BikeShares, and California Housing. The given 2d-list is the index number for interaction corresponding to the dataset. **(You will obtain the same result implementing the provided notebook, more detailed in the Notebook.)**
- Wine interaction: [[3, 7],[1, 7],[5, 6],[2, 5],[1, 4] ,[0, 7]]
- Bike interaction:
  [[3, 6],[3, 8],[1, 3],[3, 10],[3, 7],[0, 3],[2, 3],[6, 10],[8, 9],[6, 9]]
-  CA interaction: [[6, 7],[0, 5],[0, 1],[3, 5]]

The output for SDAMI in table 4 can be reproduced via
```
>> cd ./src/scripts
>> bash SDAMI.sh
```
 ## State-of-the-art comparison
 Thank the reviewers for the constructive comments. We have added Neural Additive Model (NAM), Gami-Net, NODE-GAM, and NODE-GA$^2$M as baselines for comparison. 

 **Numerical Studies** 
 In the `./Experiment/Numerical-Studies/` the `baseline_nam.py`, `baseline_gam.py`, and `baseline_nodegam.py` correspond to the baseline model. Same as the **Reproducing numerical experiment**, we could assign specific [*case*] for implementation. 

```
>> python3 baseline_nam.py --data_name [case]
```


 **Real Data Application** 
 In the `./Experiment/Real-Data-Application/` the `nam_simulation.py`, `gam_simulation.py`, and `nodegam_simulation.py` correspond to the baseline model.

 The *case* options for real data application include `Chip, Diabete, Image, Other` where `Other` includes the dataset besides first three.

```
>> python3 nam_simulation.py --data_name [case]
```


## Reference
[1] R. Agarwal, L. Melnick, N. Frosst, X. Zhang, B. Lengerich, R. Caruana, and G.E. Hinton (2021), Neural additive models

[2] Z. Yang, A. Zhang, Aijun, and A. Sudjianto (2021), Gami-net: An explainable neural network based on generalized additive models with structured interactions.

[3] C.-H. Chang, R. Caruana, and A. Goldenberg (2021), Neural generalized additive model for interpretable deep learning.
