### Changelog

All notable changes to this project will be documented in this file. Dates are displayed in UTC.

#### [v1.3.1](https://github.com/janosh/matbench-discovery/compare/v1.3.0...v1.3.1)

> 11 September 2024

- Fix model detail pages for models with array of training sets and non-string notes in YAML metadata [`#138`](https://github.com/janosh/matbench-discovery/pull/138)
- Move element-projected convex hull distance errors to model detail pages [`#137`](https://github.com/janosh/matbench-discovery/pull/137)
- Show single-model energy parity plots on model detail pages [`#136`](https://github.com/janosh/matbench-discovery/pull/136)
- fix pkg unnecessarily pinning numpy&gt;2 [`8eb916b`](https://github.com/janosh/matbench-discovery/commit/8eb916b73e36417b23dbf925452a696b9c32c1c0)
- fix errors in eda_wbm.py [`c1f34da`](https://github.com/janosh/matbench-discovery/commit/c1f34dac228ef7f0da7598e99fe9be6db36198f5)
- fix /data page MP/WBM element count ptable heatmaps [`1ebad52`](https://github.com/janosh/matbench-discovery/commit/1ebad52cff8e80febf7beadb1fe210b7741802b5)

#### [v1.3.0](https://github.com/janosh/matbench-discovery/compare/v1.2.0...v1.3.0)

> 6 September 2024

- Add metrics table download buttons [`#135`](https://github.com/janosh/matbench-discovery/pull/135)
- Restore MPtrj EDA to working order [`#130`](https://github.com/janosh/matbench-discovery/pull/130)
- Refactor model preds loading to use new required `pred_col` key in YAML metadata files [`#134`](https://github.com/janosh/matbench-discovery/pull/134)
- Orb models [`#133`](https://github.com/janosh/matbench-discovery/pull/133)
- Allow multiple training sets per model in YAML metadata [`#132`](https://github.com/janosh/matbench-discovery/pull/132)
- Correct batch_size of SevenNet-0 [`#129`](https://github.com/janosh/matbench-discovery/pull/129)
- Test more plotting scripts in CI [`#127`](https://github.com/janosh/matbench-discovery/pull/127)
- Add individual model detail pages [`#126`](https://github.com/janosh/matbench-discovery/pull/126)
- Working save trajectory code with energies also being saved for test mace. [`#123`](https://github.com/janosh/matbench-discovery/pull/123)
- Type hint optimizer and filter in `test_mace` and `test_7net` correctly, fix pickled `PatchedPhaseDiagram` and `compile_wbm_test_set.py` [`#122`](https://github.com/janosh/matbench-discovery/pull/122)
- `data.py` add `ase_atoms_(to|from)_zip` [`#120`](https://github.com/janosh/matbench-discovery/pull/120)
- Test `mace_mp_0b` [`#119`](https://github.com/janosh/matbench-discovery/pull/119)
- Update the function names for wren utils given breaking changes in Aviary [`#117`](https://github.com/janosh/matbench-discovery/pull/117)
- Update table caption for landing page [`#116`](https://github.com/janosh/matbench-discovery/pull/116)
- Add `n_estimators` to `model_metadata.yml` [`#115`](https://github.com/janosh/matbench-discovery/pull/115)
- Refactor `Files`, `DataFiles` and `PredFiles` to be `StrEnums` [`#114`](https://github.com/janosh/matbench-discovery/pull/114)
- Type hint optimizer and filter in `test_mace` and `test_7net` correctly, fix pickled `PatchedPhaseDiagram` and `compile_wbm_test_set.py` (#122) [`#121`](https://github.com/janosh/matbench-discovery/issues/121)
- fix site build error [`4d3d33f`](https://github.com/janosh/matbench-discovery/commit/4d3d33f63a963f0d54f5538af04dd74866456c8f)
- fix metrics table proprietary models toggle [`69f25b4`](https://github.com/janosh/matbench-discovery/commit/69f25b43e194f04d34b1f6dd854856fa6859d2b0)
- access pymatviz.io.save_fig from namespace [`db20338`](https://github.com/janosh/matbench-discovery/commit/db20338d3cdbaa72e81170c5060be5384131a23d)
- add data-sort-value attr to metrics table cells in "model params" and [`c989f48`](https://github.com/janosh/matbench-discovery/commit/c989f488a2fcf637b63aa4077372958596a3c5e4)
- fix typos, bump min pymatviz==0.10.1, bump ruff and fix errors [`442abf9`](https://github.com/janosh/matbench-discovery/commit/442abf91b05b318c9fd2b46e6a837e3cace7f87d)
- fix ModelCard not handling multiple training sets correctly [`a2e4c94`](https://github.com/janosh/matbench-discovery/commit/a2e4c946e0715f91c2328b794c829b0934b08eb8)

#### [v1.2.0](https://github.com/janosh/matbench-discovery/compare/v1.1.2...v1.2.0)

> 15 July 2024

- add SevenNet to Matbench Discovery leaderboard [`#112`](https://github.com/janosh/matbench-discovery/pull/112)
- Update mace description [`#113`](https://github.com/janosh/matbench-discovery/pull/113)
- Add new `/models/mace` page with formation energy and raw energy parity plots [`#111`](https://github.com/janosh/matbench-discovery/pull/111)
- Migrate `Key` enum to use `pymatviz.enums.Key` [`#110`](https://github.com/janosh/matbench-discovery/pull/110)
- Add MatterSim predictions [`#104`](https://github.com/janosh/matbench-discovery/pull/104)
- fix: CHGNet MPtrj training set now published and widely used. [`#107`](https://github.com/janosh/matbench-discovery/pull/107)
- Fix `figs/e-form-parity-models` showing convex hull distance instead of formation energy R² in subplot titles [`#106`](https://github.com/janosh/matbench-discovery/pull/106)
- fix: changes to wrenformer test script [`#105`](https://github.com/janosh/matbench-discovery/pull/105)
- refactor eda_mp_trj.py using pymatviz.plot_histogram [`b10e608`](https://github.com/janosh/matbench-discovery/commit/b10e608f5c0d3bec6f77097dcd84e5e2f11b4a58)
- make slurm_submit() time and account optional [`604cb04`](https://github.com/janosh/matbench-discovery/commit/604cb049d96548631e3493d357ecff93bdad79a3)
- fix dead links [`02b2657`](https://github.com/janosh/matbench-discovery/commit/02b2657cbb9feb810e8fce621c0eb5597816dd34)

#### [v1.1.2](https://github.com/janosh/matbench-discovery/compare/v1.1.1...v1.1.2)

> 30 May 2024

- Add `pyright` pre-commit hook and fix possibly unbound variables [`#93`](https://github.com/janosh/matbench-discovery/pull/93)
- Fix landing page vs model page metrics mismatch [`#92`](https://github.com/janosh/matbench-discovery/pull/92)
- Fill 3% missing MACE preds [`#90`](https://github.com/janosh/matbench-discovery/pull/90)
- doc: add prototype description [`#89`](https://github.com/janosh/matbench-discovery/pull/89)
- Rough solution to recreating plots for unique proto set [`#88`](https://github.com/janosh/matbench-discovery/pull/88)
- Pass preds rather than err to v-plots [`#87`](https://github.com/janosh/matbench-discovery/pull/87)
- Extend model metadata schema [`#86`](https://github.com/janosh/matbench-discovery/pull/86)
- Model metadata schema validation [`#85`](https://github.com/janosh/matbench-discovery/pull/85)
- Add GNoME model [`#84`](https://github.com/janosh/matbench-discovery/pull/84)
- Fix test scripts raising `wandb.Table` `AssertionError`: columns argument expects list of strings or ints [`#82`](https://github.com/janosh/matbench-discovery/pull/82)
- Fix landing page vs model page metrics mismatch (#92) [`#91`](https://github.com/janosh/matbench-discovery/issues/91)
- fix CaptionedMetricsTable proprietary models toggle [`64e97af`](https://github.com/janosh/matbench-discovery/commit/64e97af41f6e08e4c51cef394fb9c914b0af3830)
- hide test_set_std_key from &lt;Select /&gt; in Per-Element Model Error Heatmaps on /models page [`07555b5`](https://github.com/janosh/matbench-discovery/commit/07555b54733164090c033bb8c186c7ef1443d605)
- fix pymatviz bin_df_cols util imports + ruff fixes + bump site deps [`a3b4362`](https://github.com/janosh/matbench-discovery/commit/a3b43624b5bba0562bdbf5b0fb9fc15cefb8ffbe)
- bump ruff target-version = "py311" [`e8a4444`](https://github.com/janosh/matbench-discovery/commit/e8a4444741da9996fcc2fa83f73b035ad219fab8)
- add module doc string with installation instructions to train_mace.py [`29eecf2`](https://github.com/janosh/matbench-discovery/commit/29eecf28f756289e4b8a286d00e7c20d219e496a)
- plot script fixes [`202a305`](https://github.com/janosh/matbench-discovery/commit/202a3058d428dc7867ba6b0105d087cf29c7e130)
- change TL;DR to MP non-endorsement disclaimer [`016e71f`](https://github.com/janosh/matbench-discovery/commit/016e71fac0040edaad2b73df8aeea61e5097d453)
- fix and unignore ruff DTZ005 FBT001 FBT002 [`0249327`](https://github.com/janosh/matbench-discovery/commit/0249327b6b5a1eb7578b99ea95613d50d2465392)
- fix email and MP DB version used for convex hull construction in list of WBM processing steps [`1cc9fe6`](https://github.com/janosh/matbench-discovery/commit/1cc9fe614f431fd9fdae96512dc7bbde5137c81c)
- fix model stats JSON export in compile_model_stats.py and use model-stats-uniq-protos.json on landing page to match metrics table [`4d40cd6`](https://github.com/janosh/matbench-discovery/commit/4d40cd61eb529d0242756ccb5a0cd87f732e4b59)
- bump ruff pre-commit to 0.3.0 + auto-fixes [`71a5edc`](https://github.com/janosh/matbench-discovery/commit/71a5edc666a50bcab757bd5211886610677ab24e)
- fix metrics-table horizontal scrolling [`a5a7bda`](https://github.com/janosh/matbench-discovery/commit/a5a7bdaefc781a14453aa604689e5131425ee67a)
- fix copy-paste error of trained_by section in gnome.yml [`0ade5d6`](https://github.com/janosh/matbench-discovery/commit/0ade5d65a3544502649b00f756c984125463fe38)

#### [v1.1.1](https://github.com/janosh/matbench-discovery/compare/v1.1.0...v1.1.1)

> 28 January 2024

- Fix failing `pkg_is_editable` check in PyPI release [`#78`](https://github.com/janosh/matbench-discovery/pull/78)

#### [v1.1.0](https://github.com/janosh/matbench-discovery/compare/v1.0.0...v1.1.0)

> 25 January 2024

- Add metrics analysis excluding WBM materials with duplicate/MP-matching structure prototype [`#75`](https://github.com/janosh/matbench-discovery/pull/75)
- Define `StrEnums` `Key` and `Task` for consistent + easily updatable dataframe columns [`#74`](https://github.com/janosh/matbench-discovery/pull/74)
- Rerun M3GNet with new ASE `FrechetCellFilter` (prev `ExpCellFilter`) [`#72`](https://github.com/janosh/matbench-discovery/pull/72)
- Rerun `CHGNet` with new ASE `FrechetCellFilter` (prev `ExpCellFilter`) [`#71`](https://github.com/janosh/matbench-discovery/pull/71)
- Add flags to CHGNet and MACE test scripts to run in static mode (no relaxation) [`#70`](https://github.com/janosh/matbench-discovery/pull/70)
- UMAP visualization of MP and WBM data [`#65`](https://github.com/janosh/matbench-discovery/pull/65)
- More extensive MP/WBM data analysis for site and SI [`#64`](https://github.com/janosh/matbench-discovery/pull/64)
- add `task_id`, `calc_id`, `ionic_step` metadata to atoms.info [`#63`](https://github.com/janosh/matbench-discovery/pull/63)
- Fix missing stress unit conversion (kBar-&gt;eV/A^3) in `json_to_extxyz.py` [`#61`](https://github.com/janosh/matbench-discovery/pull/61)
- Make distinction between training set `n_structures` and `n_materials` for models trained on DFT trajectories [`#60`](https://github.com/janosh/matbench-discovery/pull/60)
- Update figures to latest MACE checkpoint [`#59`](https://github.com/janosh/matbench-discovery/pull/59)
- Re-test CHGNet v0.3.0 [`#58`](https://github.com/janosh/matbench-discovery/pull/58)
- Add `train_mace.py` for easier reproducibility [`#52`](https://github.com/janosh/matbench-discovery/pull/52)
- Update MACE results [`#54`](https://github.com/janosh/matbench-discovery/pull/54)
- create DATA_DIR and co in PKG_DIR (not /site-packages) if MBD not installed in editable mode (closes #25) [`#25`](https://github.com/janosh/matbench-discovery/issues/25)
- fix outdated pandas checksums in [`#66`](https://github.com/janosh/matbench-discovery/issues/66)
- fix dead pymatgen doc links (closes #55) [`#55`](https://github.com/janosh/matbench-discovery/issues/55)
- clarify element-errors-ptable-heatmap.svelte normalized checkbox purpose [`1f69374`](https://github.com/janosh/matbench-discovery/commit/1f6937483f5880d0ad12a4c7cccdc65044664eab)
- change ModelCard heading color scale from PuOr to Cividis [`2fcf6c2`](https://github.com/janosh/matbench-discovery/commit/2fcf6c2a0b73b78d64361a3797f7f83caaea1c10)
- tweak WBM formation energy and convex hull distance histograms [`1098aa6`](https://github.com/janosh/matbench-discovery/commit/1098aa616ebe3f2e8763bd66a04fc1abdf003962)
- remove df_to_pdf and normalize_and_crop_pdf from matbench_discovery/plots.py [`071174a`](https://github.com/janosh/matbench-discovery/commit/071174a3cd9d267f3d837c61e759466df7133f83)
- add color bar labels to MP/WBM/MPtrj ptable element occurrence heatmaps [`e7d57ee`](https://github.com/janosh/matbench-discovery/commit/e7d57ee004d7a66b424ff6c56926be8949925cc1)
- add site/src/figs/mp-vs-mp-trj-vs-wbm-arity-hist.svelte and site/src/figs/mp-trj-n-sites-hist.svelte to /data page [`46366d1`](https://github.com/janosh/matbench-discovery/commit/46366d1600468ab897ac6c6ae2bc53290b388e58)
- link MPtrj dataset from /contribute page "Direct Download" section [`db999fd`](https://github.com/janosh/matbench-discovery/commit/db999fde0fc939276add8aa039615c6376c8e484)
- add figshare article link to /contribute page [`df4f827`](https://github.com/janosh/matbench-discovery/commit/df4f82781148e541422abdb4e97d319318732bc7)
- change scatter_hull_dist_models colorscale (turbo-&gt;PuOr) to improve outlier visibility on dark bg [`467d777`](https://github.com/janosh/matbench-discovery/commit/467d77746c1f7a445b035a02db97ccc832af7dd1)
- add toggle to switch between formation energy and convex hull distance in scripts/scatter_energy_models.py [`82d07f1`](https://github.com/janosh/matbench-discovery/commit/82d07f1eff5296f66b866039a0f8ef82b7cb4bf5)
- eda_mp_trj.py add code for mp-trj-forces-ptable-hists.pdf showing the distribution of forces for each element in the periodic table [`62a6458`](https://github.com/janosh/matbench-discovery/commit/62a6458c60de26fc6cc9b034b221539bc8cec70a)
- fix trainable params in test_m3gnet.py [`e203f8f`](https://github.com/janosh/matbench-discovery/commit/e203f8f7ab0095b28bf27f773312c4fc99715fdc)
- make landing page best model report dynamic [`2f4c33a`](https://github.com/janosh/matbench-discovery/commit/2f4c33adba77d0009d1f77962617eda01cc1d67f)
- add 2023-03-16-all-mp-tasks.zip figshare url [`9376630`](https://github.com/janosh/matbench-discovery/commit/9376630486911dcb01c4dc58a3bf0bda2dc121d6)
- fix plot function cumulative_metrics() when 'RMSE' in metrics [`71d77f4`](https://github.com/janosh/matbench-discovery/commit/71d77f41242a566484321101f9ae6eb13e33ae66)
- use SymLogNorm for MP+WBM+MPtrj ptable element count heatmaps [`f959ee8`](https://github.com/janosh/matbench-discovery/commit/f959ee8a903148d4d2aa1bf35aa0aaae797ef5e8)
- add per-element MPtrj magmom ptable histogram [`13cbb90`](https://github.com/janosh/matbench-discovery/commit/13cbb90b1e05d2122d4633cb6bf9706b9e83babf)
- update email address, rename (fetch_process_wbm_dataset-&gt;compile_wbm_test_set).py [`d221cee`](https://github.com/janosh/matbench-discovery/commit/d221ceeab7464ab54f7c2664745961785aede43d)
- fix wrong MPtrj training_set URL in chgnet, mace, alignn_ff metadata.yml [`5697156`](https://github.com/janosh/matbench-discovery/commit/56971568665350fd2f240a02eb5163d833c64469)
- preprint restore abstract [`dc81813`](https://github.com/janosh/matbench-discovery/commit/dc81813ec18e90076ae99281894039387d1f8381)
- fix preprint/+page.md missing figure refs and accidentally deleted MetricsTable glossary [`ccd4000`](https://github.com/janosh/matbench-discovery/commit/ccd40003fb92d0c20af1a2af22a6db88a08828ae)
- fix ptable_heatmap_plotly precision keyword renamed to fmt [`4ce353b`](https://github.com/janosh/matbench-discovery/commit/4ce353b6079440b16c8524f2960092012f36c86d)
- mv site/src/routes/(about-the-data-&gt;data) [`6f06b4c`](https://github.com/janosh/matbench-discovery/commit/6f06b4c43e6741ce0666600ef8fd639d3c6be615)
- document both MACE checkpoints tested and update the checkpoint on figshare to the one used for MBD v1 submission [`5d28a57`](https://github.com/janosh/matbench-discovery/commit/5d28a576a5b07d18522b7172ec158c6f7be06c11)

### [v1.0.0](https://github.com/janosh/matbench-discovery/compare/v0.1.5...v1.0.0)

> 13 September 2023

- Update site to match arXiv [`#51`](https://github.com/janosh/matbench-discovery/pull/51)
- Add MEGNet RS2RE to UIP->MEGNet comparison [`#50`](https://github.com/janosh/matbench-discovery/pull/50)
- Add training set URLs in ModelCard and show training set size in metrics tables [`#49`](https://github.com/janosh/matbench-discovery/pull/49)
- Add ALIGNN FF [`#47`](https://github.com/janosh/matbench-discovery/pull/47)
- Add MACE [`#48`](https://github.com/janosh/matbench-discovery/pull/48)
- v0.1.5 [`#44`](https://github.com/janosh/matbench-discovery/pull/44)
- `train_alignn.py` add `wandb` tracking and avoid POSCARs on disk [`#45`](https://github.com/janosh/matbench-discovery/pull/45)
- fix lib/References.svelte mixing authors and titles on page reloads [`3aad858`](https://github.com/janosh/matbench-discovery/commit/3aad8582be68e2b40c8ecea6279e45e10ce69db8)
- use bin counts directly (no KDE) in hull dist density scatter plot [`5df80ef`](https://github.com/janosh/matbench-discovery/commit/5df80efbc23541d38f9f300256cc30d92c61cbce)
- add join_mace_results.py [`eb11ab0`](https://github.com/janosh/matbench-discovery/commit/eb11ab0acdf1f6c25563716af33623d76417610f)
- use different line styles for models in cumulative-precision-recall.svelte plot [`7bd2038`](https://github.com/janosh/matbench-discovery/commit/7bd20386083c119f462dd66b7a3f1732e3ac9e78)
- color hull distance scatter plots by density [`afe6ce9`](https://github.com/janosh/matbench-discovery/commit/afe6ce9aafee8ba4b897ffbf9b2857760a59407f)
- use same colors, line styles and markers for a given model across plots [`d9bb043`](https://github.com/janosh/matbench-discovery/commit/d9bb043f21961f958f0858fa71af86b50a0debcc)
- fix per-model KDE in scatter_e_above_hull_models.py, add color bar since color value is now meaningful [`13b1173`](https://github.com/janosh/matbench-discovery/commit/13b1173cb54876cf56a6267723e7bf6b04ff6a69)
- add marginal hull distribution along rolling_mae_vs_hull_dist_models plot top edge [`e7f9fe8`](https://github.com/janosh/matbench-discovery/commit/e7f9fe8fd4f170ffa3aecfc6e1d112ed21546758)
- delete outdated 'from matbench_discovery import DEBUG' [`a9386fb`](https://github.com/janosh/matbench-discovery/commit/a9386fb3549cef417fe34809280cbe937eefdd95)
- add @pbenner to citation.cff, /models page add color bar for ModelCard bg color [`5f59a90`](https://github.com/janosh/matbench-discovery/commit/5f59a9090febc8870314e1694f328a95af501a2b)
- fix tests/test_models.py::test_model_dirs_have_metadata - AssertionError: Missing metadata file: /home/runner/work/matbench-discovery/matbench-discovery/models/alignn_ff/metadata.yml [`50f5821`](https://github.com/janosh/matbench-discovery/commit/50f5821f2bbea0dbc58c7a6bf16329447cad1e62)
- fix bad column name in join_mace_results.py [`d7f300b`](https://github.com/janosh/matbench-discovery/commit/d7f300b8b73b9c9d7e6e2ff05945139d40420dd9)
- add scripts/model_figs/update_all_model_figs.py [`6696d22`](https://github.com/janosh/matbench-discovery/commit/6696d22d179ad0c85b7babaebf299335501dcb3d)

#### [v0.1.5](https://github.com/janosh/matbench-discovery/compare/v0.1.4...v0.1.5)

> 7 July 2023

- v0.1.5 [`#44`](https://github.com/janosh/matbench-discovery/pull/44)
- [pre-commit.ci] pre-commit autoupdate [`#42`](https://github.com/janosh/matbench-discovery/pull/42)
- Update site to show ALIGNN results [`#38`](https://github.com/janosh/matbench-discovery/pull/38)
- ALIGNN test on WBM data [`#37`](https://github.com/janosh/matbench-discovery/pull/37)
- Add PDF versions of all SI figures [`#36`](https://github.com/janosh/matbench-discovery/pull/36)
- Display `largest-error-scatter-select.svelte` in `/si` [`#35`](https://github.com/janosh/matbench-discovery/pull/35)
- Fix KeyError('wyckoff_spglib') in `fetch_process_wbm_dataset.py` [`#34`](https://github.com/janosh/matbench-discovery/pull/34)
- Fix `compute_struct_fingerprints.py`: cannot insert `material_id`, already exists [`#31`](https://github.com/janosh/matbench-discovery/pull/31)
- Fix `fetch_process_wbm_dataset.py` unnecessarily loading WBM initial structures from disk [`#29`](https://github.com/janosh/matbench-discovery/pull/29)
- Fix loading `2023-02-07-ppd-mp.pkl.gz` [`#26`](https://github.com/janosh/matbench-discovery/pull/26)
- Fix downloading compressed Figshare data [`#21`](https://github.com/janosh/matbench-discovery/pull/21)
- Run scripts in CI [`#19`](https://github.com/janosh/matbench-discovery/pull/19)
- Prompt to download missing data files [`#15`](https://github.com/janosh/matbench-discovery/pull/15)
- Add `upload_to_figshare.py` and replace local data-loading with cached Figshare downloads [`#13`](https://github.com/janosh/matbench-discovery/pull/13)
- DRY workflows [`#8`](https://github.com/janosh/matbench-discovery/pull/8)
- Add `Files`, `DataFiles` and `PredFiles` classes in `matbench_discovery/data.py` [`#7`](https://github.com/janosh/matbench-discovery/pull/7)
- fix path issues and using JSON pandas reader for CSV files in train_(cgcnn|wrenformer).py (closes #43) [`#43`](https://github.com/janosh/matbench-discovery/issues/43)
- Fix KeyError('wyckoff_spglib') in `fetch_process_wbm_dataset.py` (#34) [`#32`](https://github.com/janosh/matbench-discovery/issues/32) [`#33`](https://github.com/janosh/matbench-discovery/issues/33)
- Fix `compute_struct_fingerprints.py`: cannot insert `material_id`, already exists (#31) [`#30`](https://github.com/janosh/matbench-discovery/issues/30)
- Fix `fetch_process_wbm_dataset.py` unnecessarily loading WBM initial structures from disk (#29) [`#28`](https://github.com/janosh/matbench-discovery/issues/28)
- fix AttributeError: 'float' object has no attribute 'split' in fetch_process_wbm_dataset.py (closes #27) [`#27`](https://github.com/janosh/matbench-discovery/issues/27)
- Fix loading `2023-02-07-ppd-mp.pkl.gz` (#26) [`#23`](https://github.com/janosh/matbench-discovery/issues/23)
- Fix downloading compressed Figshare data (#21) [`#20`](https://github.com/janosh/matbench-discovery/issues/20) [`#17`](https://github.com/janosh/matbench-discovery/issues/17)
- Run scripts in CI (#19) [`#16`](https://github.com/janosh/matbench-discovery/issues/16)
- Prompt to download missing data files (#15) [`#14`](https://github.com/janosh/matbench-discovery/issues/14) [`#14`](https://github.com/janosh/matbench-discovery/issues/14)
- Add `upload_to_figshare.py` and replace local data-loading with cached Figshare downloads (#13) [`#10`](https://github.com/janosh/matbench-discovery/issues/10) [`#12`](https://github.com/janosh/matbench-discovery/issues/12) [`#11`](https://github.com/janosh/matbench-discovery/issues/11)
- support model_name as array in metadata.yml files to share metadata between multiple types of a model [`c8eaebd`](https://github.com/janosh/matbench-discovery/commit/c8eaebd673973fccf7db74cec0f8a8a6f0883932)
- include legacy MP energy corrections in data/wbm/2022-10-19-wbm-summary.csv, use them to remove old and apply new corrections in test_megnet.py [`8166801`](https://github.com/janosh/matbench-discovery/commit/81668016b610a6567058a4066b739586e54cf43b)
- fix join_chgnet_results.py by removing code to apply MP2020 corrections [`01658ad`](https://github.com/janosh/matbench-discovery/commit/01658ad73e4c6cb6a320d4f906f19fb443d08c9f)
- add columns wyckoff_spglib and spacegroup_symbol to data/mp/2023-01-10-mp-energies.csv [`01e88f0`](https://github.com/janosh/matbench-discovery/commit/01e88f007faec207ecf7a6f59663f62aecbaf394)
- add model metrics table when only validating the 10k most stable predictions for each model [`c4ca186`](https://github.com/janosh/matbench-discovery/commit/c4ca1860b970bfc469278b915b80fdbf4c2d9626)
- run model combos (m3gnet|chgnet)+megnet through test_megnet.py [`fbe847b`](https://github.com/janosh/matbench-discovery/commit/fbe847bbcb4500277f4e291ce965611667df87d2)
- split analyze_model_failure_cases.py into two scripts, new one is analyze_elements.py [`0fad3bd`](https://github.com/janosh/matbench-discovery/commit/0fad3bd478f2290a26b90351d8b11c5b6cc1491a)
- add scatter plot of largest errors averaged over models vs DFT hull distance [`457abf0`](https://github.com/janosh/matbench-discovery/commit/457abf05476f8835a29e29a1be40e6acb085523b)
- add scripts/compute_struct_fingerprints.py to generate matminer SiteStats fingerprints for all MP+WBM structures [`7946b5e`](https://github.com/janosh/matbench-discovery/commit/7946b5e433713ef8f46b839562f09d889331e63b)
- split model run times into train and test contribs [`b8a18d8`](https://github.com/janosh/matbench-discovery/commit/b8a18d8fd357ea4ec77dae837fdb8317fe978b28)
- add models/chgnet/{test_chgnet,join_m3gnet_results}.py [`da39074`](https://github.com/janosh/matbench-discovery/commit/da390741b0d0f03fbcca5bcbac725c2211a80427)
- git surgery purging all large CSV files from repo history [`4a90dee`](https://github.com/janosh/matbench-discovery/commit/4a90dee85965aebbcfcd547353c548eeb19651ce)
- add scripts/compute_projections.py [`eec1cb4`](https://github.com/janosh/matbench-discovery/commit/eec1cb4073c2db65cd238157f656af963582ffc5)
- make horizontal versions of roc-models-2x4.pdf and model-run-times-bar.pdf [`551050e`](https://github.com/janosh/matbench-discovery/commit/551050e17cc98cf786707448fd304bb06177a914)
- add changelog.md [`0621282`](https://github.com/janosh/matbench-discovery/commit/0621282b0ed6ca397b6293e8d6f2bcbaac736468)
- add models/chgnet/ctk_struct_traj.py and models/wrenformer/analyze_wrenformer.py [`c9fed5a`](https://github.com/janosh/matbench-discovery/commit/c9fed5ae7bddc02c3fbb2160a00b4e8822b1a179)
- add data/mp/get_mp_traj.py to save a snapshot of all MP ionic steps on 2023-03-15 to be released as MBD canonical training set [`76647e3`](https://github.com/janosh/matbench-discovery/commit/76647e37b9fdfbbc77fb2317ad2a0cf3369638a9)
- move load_df_wbm_preds+PredFiles from matbench_discovery/{data->preds}.py [`387184c`](https://github.com/janosh/matbench-discovery/commit/387184caa50c27e2574a86a4bef65a1628434378)
- add site/src/figs/(largest-fp-diff-each-error-models|largest-each-errors-fp-diff-models).svelte [`4b6e83a`](https://github.com/janosh/matbench-discovery/commit/4b6e83a13816f9b8dcc911af5cbe1d79f05bf876)
- move /paper/preprint to /paper [`8a0c3bb`](https://github.com/janosh/matbench-discovery/commit/8a0c3bb47609cf784d95e7499394785487383d82)
- add global STABILITY_THRESHOLD to consistently parametrize across the codebase when materials count as thermodynamically stable [`70b2b1d`](https://github.com/janosh/matbench-discovery/commit/70b2b1d81476e5913802d8286f281af9385a8754)
- plot easy vs hard structures (for all models) norm of SiteStats fingerprint difference before/after relaxation [`a6bfa74`](https://github.com/janosh/matbench-discovery/commit/a6bfa749aba0076a20af9e8fa83422744a4d45ea)
- add missing doc strings [`1ae72c0`](https://github.com/janosh/matbench-discovery/commit/1ae72c0c2f395e90d0a2ba6d1a87f3e7332af805)
- fix direct data file download URLs on /contribute page [`83736ad`](https://github.com/janosh/matbench-discovery/commit/83736ad5a94b823f69d7c61314daf9fbaeac331d)
- add git submodule paper to prepare journal submission in separate repo [`3ac72e6`](https://github.com/janosh/matbench-discovery/commit/3ac72e66df1b2e163c08840fabe41e078d7045f6)
- add figs/cumulative-mae-rmse.svelte displayed in /si [`8798786`](https://github.com/janosh/matbench-discovery/commit/879878603af492888448d59c209a41f74f3fe6f5)
- rename ElemCountInset to PtableInset and add props precision, show_percent, unit [`abfc0ac`](https://github.com/janosh/matbench-discovery/commit/abfc0ac1e6bd956975466426f8760ef9a489be99)
- add scripts/ctk_structure_viewer.py [`fa1a439`](https://github.com/janosh/matbench-discovery/commit/fa1a439b0c73222e583c14b9167c143074d57315)
- +layout.svelte add `<svelte:head>` with `<title>` and `<meta description>` [`5d38918`](https://github.com/janosh/matbench-discovery/commit/5d3891858e88c8efce694389c55dcf597b5464ce)
- refactor data loading in model test scripts [`d564ade`](https://github.com/janosh/matbench-discovery/commit/d564ade5359daeca286ac41c59b02716572c5a40)
- add CmdPalette invoked with cmd+k for keyboard-only site navigation [`59010fa`](https://github.com/janosh/matbench-discovery/commit/59010fa272c29c9d598ca89d81ecc7a116114672)
- improve preprint results + discussion wording [`d27f1c4`](https://github.com/janosh/matbench-discovery/commit/d27f1c466a3c6a8677b51b908f13d60b040c8dfa)
- fix outdated numbers in data/wbm/readme.md [`79c8042`](https://github.com/janosh/matbench-discovery/commit/79c804281c5aa49b223fa7944f8dee8633dc7f44)
- change SI plot of largest model errors: Predicted vs. DFT hull distance colored by model disagreement [`a5b3211`](https://github.com/janosh/matbench-discovery/commit/a5b3211f571f49e5c72d948d971bbd3db19066f0)
- show figs/element-prevalence-vs-error.svelte on /si page [`6771462`](https://github.com/janosh/matbench-discovery/commit/6771462f997fc8cf87e19a7327d349670d2ad44d)
- rm frontmatter.yml after consolidating with citation.cff [`228be00`](https://github.com/janosh/matbench-discovery/commit/228be00c7f4408328c28919cd053d84bf616b5e0)
- rename data/wbm/(analysis->eda).py [`0f2410d`](https://github.com/janosh/matbench-discovery/commit/0f2410de1d619c8318d9a06ce3d693d19d76b223)
- make wide tables horizontally scrollable on mobile screens [`fc20ccf`](https://github.com/janosh/matbench-discovery/commit/fc20ccf5f33dc50ebfa429d4b9ceee95715d7e03)
- toggle site tips on cmd+j, update deps, tweak copy (abstract, metrics-table-first-10k, readme) [`137c414`](https://github.com/janosh/matbench-discovery/commit/137c414badc8a4bd69748396077a46292a09c647)
- add date_published field to model metadata [`cddda31`](https://github.com/janosh/matbench-discovery/commit/cddda31d9395574e63ac3f8a39a9e7df7570fa31)
- fix /contribute [`dfa6c24`](https://github.com/janosh/matbench-discovery/commit/dfa6c24b12273816d3fa30442003e884c64d3b58)
- allow toggling MP+WBM element prevalence ptable heat maps between count modes 'occurrence' and 'composition' [`1fe5bcb`](https://github.com/janosh/matbench-discovery/commit/1fe5bcbf0752e6eb0d04b0e3365bc570237fbf60)
- add material_id to figs/mp-elemental-ref-energies.svelte hover tooltip [`c87f0b2`](https://github.com/janosh/matbench-discovery/commit/c87f0b2087b4d0ed7fb43331cf82e57fbca75ef8)
- bump @rollup/plugin-yaml to v4.1 and refactor citation.cff import using new extensions option [`cc08c78`](https://github.com/janosh/matbench-discovery/commit/cc08c78bcc2f1433c6cc52cb97b16b05a41d0014)
- tweak contributing guide [`b3ea8d3`](https://github.com/janosh/matbench-discovery/commit/b3ea8d3980d62032ed33ae75fb0c7539fafcebe3)
- upload @pbenner's ALIGNN checkpoint to Figshare and link from models/alignn/readme.md [`da56377`](https://github.com/janosh/matbench-discovery/commit/da5637725e3f785cdf8bd82ff6a3be95858b2eae)
- start preprint [`fd8355d`](https://github.com/janosh/matbench-discovery/commit/fd8355dca9fe87ef9196980c06907ef377fdcd97)
- rename pages /about-test-test-set to /about-the-data and /how-to-contribute to /contribute [`e41ca32`](https://github.com/janosh/matbench-discovery/commit/e41ca32e768ea06f404b216353521f89e6bee2e2)

#### v0.1.4

> 19 June 2023

- Add WBM element heatmap to site and data/wbm/readme.md [`#6`](https://github.com/janosh/matbench-discovery/pull/6)
- doc: remove reference to alignn [`#4`](https://github.com/janosh/matbench-discovery/pull/4)
- Initial PyPI release preparation [`#5`](https://github.com/janosh/matbench-discovery/pull/5)
- Start work on website to host docs and display interactive leaderboard [`#3`](https://github.com/janosh/matbench-discovery/pull/3)
- move m3gnet formation energy corrections to join_m3gnet_results.py [`248a79b`](https://github.com/janosh/matbench-discovery/commit/248a79bd755917c9e2f9a00c0f70c3e927321743)
- generate MEGNet formation energy preds for M3GNet-relaxed structures in join_m3gnet_results.py [`e526e62`](https://github.com/janosh/matbench-discovery/commit/e526e627119b537521e5f38d4b34ff28082245d8)
- merge load_model_preds() into load_df_wbm_with_preds() and pass through **kwargs to pandas reader [`1681c38`](https://github.com/janosh/matbench-discovery/commit/1681c38ed21afd119bd271eb9f14a67f6bf1a17c)
- add SI discussion on 'WBM Batch Robustness as a Measure of Extrapolation Prowess' [`705e0d0`](https://github.com/janosh/matbench-discovery/commit/705e0d0c1dc9ff6a17158bae4723eef9d031329d)
- add descriptions to all model metadata.yml [`4013bb1`](https://github.com/janosh/matbench-discovery/commit/4013bb18fd7bf36dcb4765e65d16f7838ff8144d)
- make metrics-table sortable with svelte-zoo action [`13746f4`](https://github.com/janosh/matbench-discovery/commit/13746f40a37b8d03131d2c942398b1141f0a35e9)
- initial commit for paper [`cd4b90f`](https://github.com/janosh/matbench-discovery/commit/cd4b90f9df529b382e47ab623e95cc92c2ea2ce6)
- add stable_metrics() to matbench_discovery/energy.py (formerly part of hist_classified_stable_vs_hull_dist() which gets a big refactor) [`f0363b4`](https://github.com/janosh/matbench-discovery/commit/f0363b4d2bd2e56cfc2db5aa48632ec1b3372611)
- fix WBM e_above_convex_hull values w.r.t. MP PPD [`5dae951`](https://github.com/janosh/matbench-discovery/commit/5dae951de222ccd6e0e6d0d41b6570e4ac1e0d04)
- add matbench_discovery/metrics.py to centralize computing metrics for plotting scripts [`500e670`](https://github.com/janosh/matbench-discovery/commit/500e67060be6e252d77eecae799a9c4032873152)
- add 2023-01-25-rolling-mae-vs-hull-dist-models.svelte [`9fed210`](https://github.com/janosh/matbench-discovery/commit/9fed210cf441e06f62c10ca1db4ae0a64a26b55b)
- add data/wbm/fetch_process_wbm_dataset.py [`6ac75aa`](https://github.com/janosh/matbench-discovery/commit/6ac75aa1c860eb28e6ad806a54c62d0a778aaac9)
- extract ModelCard component from page /models [`f5c3e37`](https://github.com/janosh/matbench-discovery/commit/f5c3e3775f96ffb3835aafe4f2f0a7de4eba9c04)
- add test_hist_classified_stable_as_func_of_hull_dist() [`c855e51`](https://github.com/janosh/matbench-discovery/commit/c855e512bc27a69ad03ed5de0223ba79fc5a49e7)
- add site/src/figs/2023-02-01-mp-elemental-ref-energies.svelte generated in data/wbm/analysis.py [`9f43711`](https://github.com/janosh/matbench-discovery/commit/9f43711c06fcd3d1c9fa018b6ed2835c8e2df8f3)
- allow multiple models per metadata.yml file, add models M3GNet + MEGNet and CGCNN+P to respective model cards [`24f6868`](https://github.com/janosh/matbench-discovery/commit/24f68688f68b6bb4561b03c8d9869074abd3bae5)
- refactor cumulative_clf_metric()->cumulative_precision_recall() to plot both prec and recall for all models [`4972c01`](https://github.com/janosh/matbench-discovery/commit/4972c010523f4fc309019dcf7ece0e921e6b926f)
- remove svelte config prerender.handleHttpError: warn (site now builds error free) [`f84171b`](https://github.com/janosh/matbench-discovery/commit/f84171bba2e88cee4ba9988839bd3cbfe0e06cec)
- add mdsvex custom img component [`2f6fae3`](https://github.com/janosh/matbench-discovery/commit/2f6fae3d6561c3c6201d6fc528f2d6d8aa4dc93f)
- add site/src/routes/paper/iclr-ml4mat/+page.md [`32ed170`](https://github.com/janosh/matbench-discovery/commit/32ed170d210727abafe3c25a9f8d93c0639adfa5)
- add /models page to site [`f28cc6d`](https://github.com/janosh/matbench-discovery/commit/f28cc6d283e1817660ce586fc0151bbe24e097e4)
- add kwarg metrics=("Precision", "Recall") to cumulative_precision_recall() [`4cf5028`](https://github.com/janosh/matbench-discovery/commit/4cf50282d4cc0e7dffcf08c5a5339eca3f8cac74)
- support plotly backend in rolling_mae_vs_hull_dist() [`fbf6a02`](https://github.com/janosh/matbench-discovery/commit/fbf6a02b723b04669070dd06fb67e6a2c7e78985)
- refactor plot func hist_classified_stable_as_func_of_hull_dist() [`967b482`](https://github.com/janosh/matbench-discovery/commit/967b482a10088ebfa946ea8df9a361ede76786b1)
- initial commit [`a111222`](https://github.com/janosh/matbench-discovery/commit/a111222c141474319482f78935cc6da716d678b3)
- add new module matbench_discovery/data.py with func load_wbm() [`0773112`](https://github.com/janosh/matbench-discovery/commit/077311216c62aecc3005da3719d7a16bbde8f555)
- add models/cgcnn/{slurm_train_cgcnn_ensemble,use_cgcnn_ensemble}.py [`a2e3f46`](https://github.com/janosh/matbench-discovery/commit/a2e3f46c3d8b4f31923b4a47529d5aa212dd507d)
- breaking add e_form_per_atm cutoff of +/- 5 eV/atom to WBM test set in fetch_process_wbm_dataset.py [`827c543`](https://github.com/janosh/matbench-discovery/commit/827c543f21a3e92270cbc59edd5eb257d7dd26aa)
- extract plot func rolling_mae_vs_hull_dist() from rolling MAE plot scripts [`8d65a7a`](https://github.com/janosh/matbench-discovery/commit/8d65a7aad620e2718012c03048b5b835a3c8a8db)
- fix WBM index mismatch between CSEs and summary [`20080b5`](https://github.com/janosh/matbench-discovery/commit/20080b5d9621f05ad558e24b92d923b994bdf3e1)
- refactor rolling_mae_vs_hull_dist() to support plotting multiple lines at once [`1b7f056`](https://github.com/janosh/matbench-discovery/commit/1b7f0560dca12724137208d1707530850c4ce02e)
- upgrade /models page with sorting and reporting model metrics (MAE, R2, Recall, ...) and stats (run time, date added, ...) [`8079ca1`](https://github.com/janosh/matbench-discovery/commit/8079ca1645de3a372b5ea2fa18208cd857178533)
- add backend=plotly | matplotlib to hist_classified_stable_vs_hull_dist() [`0fb7550`](https://github.com/janosh/matbench-discovery/commit/0fb755053e9dc36fe27d86fc2be89c33ec47b8f1)
- add route /about-the-test-set/tmi to display per-batch and per-arity WBM element counts [`5d98946`](https://github.com/janosh/matbench-discovery/commit/5d98946eb30a83be1b061d6abdaad74779240d6e)
- add classify_stable() in matbench_discovery/energy.py [`d593ae2`](https://github.com/janosh/matbench-discovery/commit/d593ae2586b059267b041ee29a762c4ecfbb0e13)
- add wandb.log scatter-parity plot in test_{cgcnn,wrenformer}.py [`255429a`](https://github.com/janosh/matbench-discovery/commit/255429a4c75369de633a2cdc41b3fc537e59df32)
- add slurm_submit_python() in new module mb_discovery/slurm.py [`97c6949`](https://github.com/janosh/matbench-discovery/commit/97c6949fcf9a3ec084bf6d4d7ae92f3ac1800825)
- use stable_metrics() from matbench_discovery.energy in scripts/compile_metrics.py and update site/src/routes/models/2023-01-23-model-stats.json [`75fc095`](https://github.com/janosh/matbench-discovery/commit/75fc0951779c10adcdb65fadf6474a8a0f5f6bdc)
- add MP (train set) elem count heatmap for comparison with WBM (test set) heatmap [`3130c89`](https://github.com/janosh/matbench-discovery/commit/3130c89ce408d6d16c3bad8183ec2fb55bce1489)
- crop WBM data to formation energy cutoff of +/-5 eV/atom (drops 524 materials, 502 below -5, 22 above +5) [`ad8349b`](https://github.com/janosh/matbench-discovery/commit/ad8349b17b9b322ad999ca38d4125bbcc4757edb)
- add ml_stability/m3gnet/slurm_array_m3gnet_relax_wbm.py [`1e96458`](https://github.com/janosh/matbench-discovery/commit/1e9645840f2241d5744a2f5290085f6344c37f5f)
- rename plot func precision_recall_vs_calc_count() to cumulative_clf_metric() that plots single metric at a time [`86f85f3`](https://github.com/janosh/matbench-discovery/commit/86f85f36488ec901b92b574f8307f0361bfa6295)
- fix scripts/hist_classified_stable_vs_hull_dist_models.py [`4ae422d`](https://github.com/janosh/matbench-discovery/commit/4ae422d71c548d1c13e8827a59cf3328d8c698b1)
- add models/bowsr/slurm_array_bowsr_megnet_relax_wbm.py [`e6bf955`](https://github.com/janosh/matbench-discovery/commit/e6bf9557daa0ae77116352225ec3fbccaa8cc10b)
- fix scripts/prc_roc_curves_models.py and roc-models.svelte fig [`a42472c`](https://github.com/janosh/matbench-discovery/commit/a42472c35073aa7b1a7b47e65a32c2bd9c66c218)
- add 2023-01-26-model-run-times-pie.svelte to /models page (from scripts/compile_metrics.py) [`0b8426a`](https://github.com/janosh/matbench-discovery/commit/0b8426abddf50100498050b9386a2b5f52828e5c)
- add test_precision_recall_vs_calc_count() [`e54a8c6`](https://github.com/janosh/matbench-discovery/commit/e54a8c6dd8b16f018c2b25e3b7f12b8652317cde)
- add mb_discovery/build_phase_diagram.py [`42a7909`](https://github.com/janosh/matbench-discovery/commit/42a7909bd753adacd67b5bf8d84691ab743f0104)
- fix legend/subplot titles in scripts/scatter_e_above_hull_models.py [`2f795f7`](https://github.com/janosh/matbench-discovery/commit/2f795f7c755865b326ed5cec203b291931249d7b)
- add precision_recall_vs_calc_count() to plot_funcs.py extracted from plot_scripts/precision_recall_vs_calc_count.py [`7ed1b09`](https://github.com/janosh/matbench-discovery/commit/7ed1b09da288dafdc403d1a201d9a8c360e05d5f)
- provisionally last set of fixes/refactors to data/wbm/fetch_process_wbm_dataset.py [`4936079`](https://github.com/janosh/matbench-discovery/commit/493607918bb4b49a6ca611b756e97822a4250660)
- add 2023-01-18-e-form-scatter-models.png to paper [`d1751a3`](https://github.com/janosh/matbench-discovery/commit/d1751a35c0e045f50cd7545d2ecea885d847cbd6)
- remove kwarg stability_crit from hist_classified_stable_vs_hull_dist() and cumulative_clf_metric() [`c5d3496`](https://github.com/janosh/matbench-discovery/commit/c5d34963b035d94d3f81d55ee087838751411bbc)
- add plot_scripts/all_models_scatter.py [`0236c36`](https://github.com/janosh/matbench-discovery/commit/0236c367171651c483b7a963d7342eca06ebfe21)
- add scripts/difficult_structures.py [`db34c09`](https://github.com/janosh/matbench-discovery/commit/db34c092502e557bb017b58e09116e25fd3299ec)
- make 2nd arg elemental_ref_entries to get_form_energy_per_atom() optional [`2517855`](https://github.com/janosh/matbench-discovery/commit/2517855ef971cabeecd0e796eaf27dd58c4d79a0)
- fix load_train_test() caching all data versions to same directory [`5084886`](https://github.com/janosh/matbench-discovery/commit/50848865f8a47dfdd45cad1c8d8b81fed0778f20)
- add citation.cff, add step-by-step guide for adding new models to the leaderboard to site/src/routes/how-to-contribute/+page.md [`4121f49`](https://github.com/janosh/matbench-discovery/commit/4121f4985008775d665b0f60843f27dccdf7e9b0)
- fix voronoi RF metadata.yml [`387722a`](https://github.com/janosh/matbench-discovery/commit/387722a006074c592354e8f1cfe2ad115baae891)
- add slurm submit and wandb logging to train_voronoi_rf.py [`6e58a1b`](https://github.com/janosh/matbench-discovery/commit/6e58a1b36390d4ca7b37b6ef152b76d9b9522bb3)
- eda_wbm_pre_vs_post_m3gnet_relaxation.py add code for 2 new plots [`bf36e42`](https://github.com/janosh/matbench-discovery/commit/bf36e423b9c67ff997a760fa5987abc170e74b39)
- SSoT for projects URLs in site/package.json [`a4b37a3`](https://github.com/janosh/matbench-discovery/commit/a4b37a352fe40f597db4c99ed5b561794321179d)
- add scripts/roc_models.py [`32a02d8`](https://github.com/janosh/matbench-discovery/commit/32a02d8232f888a90efd716b79d6a6a578bdb935)
- add rolling_accuracy kwarg and doc str to hist_classified_stable_as_func_of_hull_dist() [`9c6d61c`](https://github.com/janosh/matbench-discovery/commit/9c6d61c0de88cfc1799639d44fccad21b0b47747)
- add scripts/metrics_table.py [`20ef518`](https://github.com/janosh/matbench-discovery/commit/20ef518e6e7f4b79f01c62fb37efa4cdbc19f863)
- add data/wbm/compare_cse_vs_ce_mp_2020_corrections.py [`c1e55e1`](https://github.com/janosh/matbench-discovery/commit/c1e55e1ef3fad178b9ea949a69451152136032dc)
- add code to generate hist_classify_stable_as_func_of_hull_dist at end of ml_stability/m3gnet/m3gnet_relax_wbm.py [`383b8f4`](https://github.com/janosh/matbench-discovery/commit/383b8f46ec84419dd769066776fb2090605903b1)
- add kwarg show_dft_acc=False to rolling_mae_vs_hull_dist() [`9aebb08`](https://github.com/janosh/matbench-discovery/commit/9aebb08207ffade01a26723bee7163b909838533)
- split model pred loading from CSV into new module matbench_discovery/preds.py [`0e9e5dc`](https://github.com/janosh/matbench-discovery/commit/0e9e5dc6af1e156ad2f0b8a639f58af385eab82a)
- add structure perturbation to train_cgcnn.py [`6dd4398`](https://github.com/janosh/matbench-discovery/commit/6dd43982ac3ca8ae720ccd9d89fc19414f7a793e)
- move mb_discovery/{energy/__init__.py -> energy.py} and add tests/test_energy.py [`aec9933`](https://github.com/janosh/matbench-discovery/commit/aec99338305a54ee121acedd529eee5b971ca6a7)
- make model switching easy in hist_classified_stable_as_func_of_hull_dist_batches.py [`c487910`](https://github.com/janosh/matbench-discovery/commit/c487910787939f887ca24503305bade8b9c1201c)
- convert old data and plot file paths from rhys to new repo [`e61c8c7`](https://github.com/janosh/matbench-discovery/commit/e61c8c73682b7217b706b9d22a430a9897154a47)
- add write_html() and write test for it [`7a642d7`](https://github.com/janosh/matbench-discovery/commit/7a642d7fb818a6139c5495a87d759dcaf84c3128)
- return slurm environment variables from slurm_submit_python() [`444fb7f`](https://github.com/janosh/matbench-discovery/commit/444fb7fe9edbee75024f4b71acb5b41a8bc7440d)
- add data/2022-09-19-wbm-formation-energy+wyckoff.csv [`50b5f28`](https://github.com/janosh/matbench-discovery/commit/50b5f2833f46598d18561fadb6c54fcbb4136964)
- add cumulative F1 score to cumulative precision recall plot [`6e36278`](https://github.com/janosh/matbench-discovery/commit/6e3627801ab1cbf05de80c547bf0aba69736a70e)
- compare IS2RE and RS2RE energies in eda_wbm_pre_vs_post_m3gnet_relaxation.py [`90e1b89`](https://github.com/janosh/matbench-discovery/commit/90e1b89a7ddfda6f3d6527afb181b3f23e187655)
- refactor models/wrenformer/{mp,m3gnet_train_set}/slurm_submit.py to use slurm_submit_python() [`a19ca5d`](https://github.com/janosh/matbench-discovery/commit/a19ca5d8fd9c56eae28b38aedeba21e59d8e8c70)
- don't hydrate pseudo-relaxed structures in join_{bowsr,m3gnet}_results.py [`2073300`](https://github.com/janosh/matbench-discovery/commit/2073300eefce23055d53a443633c9b5ea2729f90)
- append '-debug' to job_name if debugging automatically [`f9c348c`](https://github.com/janosh/matbench-discovery/commit/f9c348ce8b1f3e2879590ef0a76a429828b76177)
- add models/megnet/slurm_array_megnet_wbm.py [`ffa075e`](https://github.com/janosh/matbench-discovery/commit/ffa075e648f08a6da943ca5a673ce40448749d15)
- add models/bowsr/join_bowsr_results.py [`e7f4582`](https://github.com/janosh/matbench-discovery/commit/e7f45826e59233dc50aa232bcd72e23e52225bbb)
- add mb_discovery/energy/slurm_e_above_hull.py [`4f6d853`](https://github.com/janosh/matbench-discovery/commit/4f6d853d4bc3004fbd943e60e0d5c5374cec62f9)
- more refactors in precision_recall_vs_calc_count.py and moving_hull_dist_mae_compare_models.py [`8daf748`](https://github.com/janosh/matbench-discovery/commit/8daf74855a97c330ef4da35b1f737de442dcefb3)
- add ml_stability/pre-vs-post-m3gnet-relaxation.py [`68a370a`](https://github.com/janosh/matbench-discovery/commit/68a370adf57f97faafe3db4ba0a0db25a4b68d51)
- add site Footer, add next/prev page links (with wrap-around) [`98a4289`](https://github.com/janosh/matbench-discovery/commit/98a4289e0830352132f14b35af83622a08736175)
- rename job_id->slurm_job_id in wandb run.config, rename 2022-08-16-m3gnet-wbm-relax-results-IS2RE.json.gz to 2022-08-16-m3gnet-wbm-IS2RE.json.gz [`0473994`](https://github.com/janosh/matbench-discovery/commit/047399422d703c18188d7ba0279eba2e5a3f2cea)
- update all figures with M3GNet + MEGNet and CGCNN=P results [`f9d4d04`](https://github.com/janosh/matbench-discovery/commit/f9d4d04726e718390a87a50b4b0433db45caf597)
- migrate setup.py to pyproject.toml [`81dc5f0`](https://github.com/janosh/matbench-discovery/commit/81dc5f036719d6ee0ee3507a6459583fd0ce65cc)
- rename project from ml-stability to matbench-discovery [`7c30b09`](https://github.com/janosh/matbench-discovery/commit/7c30b094d54bcc31c4e832bf5ef3de49e8a7c404)
- make PNG of WBM e_form hist in data/wbm/readme.md interactive plotly figure [`9223c75`](https://github.com/janosh/matbench-discovery/commit/9223c7556b9e87183c91d098cf18ab083ce8005c)
- fix misnamed variable in rolling_mae_vs_hull_dist() e_above_hull_pred->e_above_hull_error [`f5057ac`](https://github.com/janosh/matbench-discovery/commit/f5057ac0e1bb7e1252d44a90f27bb35cc34728f8)
- compare_cse_vs_ce_mp_2020_corrections.py code for <https://github.com/materialsproject/pymatgen/issues/2730> [`6101995`](https://github.com/janosh/matbench-discovery/commit/6101995c0eb8efd96ff97b2f195c238a95cd3b48)
- add mb_discovery/wrenformer/{mp,m3gnet_train_set} [`3d42214`](https://github.com/janosh/matbench-discovery/commit/3d42214b11a4644c60210e5dd128632958f2b017)
- add wandb scatter plot logging to slurm_array_megnet_wbm.py [`fed968f`](https://github.com/janosh/matbench-discovery/commit/fed968ff2768623574618946b387ad85d4341d13)
- add m3gnet to precision_recall_vs_calc_count.py script and save output pdf [`e776882`](https://github.com/janosh/matbench-discovery/commit/e776882faf33f9db271a03e0984673290b7b924f)
- add code to run test_cgcnn.py with task_type=RS2RE [`a2a99f2`](https://github.com/janosh/matbench-discovery/commit/a2a99f2323dd51e80b882563dbba3bf64466be88)
- fix recursion error when saving voronoi-tesselation to JSON [`38a9d7a`](https://github.com/janosh/matbench-discovery/commit/38a9d7afcef8e608146dcbdb453a2e59b21bae22)
- rename load_wbm() -> load_train_test() and add ability to download MP training files too [`811f581`](https://github.com/janosh/matbench-discovery/commit/811f581e5bfb9743d8245f51d9cb4c12c46bc83d)
- add tests/test_models.py [`ae020a7`](https://github.com/janosh/matbench-discovery/commit/ae020a7240dd6931416256a204d892c15bb8ed83)
- rename enrichment factor to DAF (discovery acceleration factor) [`1ee98fe`](https://github.com/janosh/matbench-discovery/commit/1ee98fe11326431d3cecea981b965f10ff6c4f4d)
- pnpm add -D katex remark-math for equation support in docs [`18be9dc`](https://github.com/janosh/matbench-discovery/commit/18be9dcb95ca3f79e143f6894717a78c382ea501)
- merge all /api/* routes into single page /api [`2908fdf`](https://github.com/janosh/matbench-discovery/commit/2908fdf5255e4cf1f7c02d5ddd07041750f2312b)
- add wandb_log_scatter() to plots.py, use in test_{wrenformer,cgcnn,megnet} [`65172ff`](https://github.com/janosh/matbench-discovery/commit/65172ffcc977bc4a099cd7dae43f4647c7ba638c)
- move model run/eval scripts from mb_discovery/ to new dir models/ [`89976c4`](https://github.com/janosh/matbench-discovery/commit/89976c4d10405999fd217c2ad7855273d2708c1c)
- add usage example code fence and WBM summary column glossary to how-to-use/+page.svx [`5405005`](https://github.com/janosh/matbench-discovery/commit/540500586e7cf753f780fb52a951d29f25234d50)
- add kwarg intersect_lines: str | Sequence[str] = () to precision_recall_vs_calc_count() [`39fa65a`](https://github.com/janosh/matbench-discovery/commit/39fa65a7422fa3554546f33dadeeddd340a9e2ba)
- refactor hist_clf_vary.py [`d5422bd`](https://github.com/janosh/matbench-discovery/commit/d5422bd5d9e4b7d7f6a5c90a58ded1e30ae9371c)
- add models/voronoi/join_voronoi_features.py [`b3c3aba`](https://github.com/janosh/matbench-discovery/commit/b3c3abaf56a12e96fe61bf4c6a8742732deb9fef)
- add models/voronoi/train_voronoi_rf.py [`65d1b54`](https://github.com/janosh/matbench-discovery/commit/65d1b54fd7c9b23b2049bc14bd2f545b8281e5aa)
- file tables and improved wording in data/wbm/readme.md [`d491439`](https://github.com/janosh/matbench-discovery/commit/d491439654a7ab3b2e7a710256546d9c19b20267)
- mv {mb->matbench}_discovery [`9fdab8c`](https://github.com/janosh/matbench-discovery/commit/9fdab8cb304b8d451104a21463ff86605172744d)
- update all plot scripts to use new 2022-10-19-wbm-summary.csv [`8d9e346`](https://github.com/janosh/matbench-discovery/commit/8d9e346d6bc52a509059cffa9fbe19dfe911b5fe)
- improve test_slurm_submit(), add tests/test_init.py [`da2337a`](https://github.com/janosh/matbench-discovery/commit/da2337a80f41eb0d4baf41943c1bab8b53ad892d)
- drop trajectories when joining slurm array m3gnet results [`c3c084b`](https://github.com/janosh/matbench-discovery/commit/c3c084b22677d2058a07360eba8dc6128ef4c5c1)
- add aflow wyckoff labels to 2022-10-19-wbm-summary.csv [`6450ebb`](https://github.com/janosh/matbench-discovery/commit/6450ebb45e3f9922f9628166cdc69bbcd0e0fc28)
- add scripts/hist_classified_stable_vs_hull_dist_models.py [`6030ef3`](https://github.com/janosh/matbench-discovery/commit/6030ef3b37d4aa8786b6bae1c121d42d84c8c6f5)
- add scripts/update_wandb_runs.py [`2779538`](https://github.com/janosh/matbench-discovery/commit/277953877f8e3eac8ee0c4f8ec9ae975de3610d6)
- add figures/2022-11-17-m3gnet-wbm-hull-dist-hist-batches.pdf [`de61b89`](https://github.com/janosh/matbench-discovery/commit/de61b89cde6ffc1e746aa1e88e0da55653ddf939)
- lower dft accuracy cone in hist_classified_stable_vs_hull_dist() [`8777870`](https://github.com/janosh/matbench-discovery/commit/877787046359e15bac05c234965b5d66be5e9f02)
- add test_rolling_mae_vs_hull_dist() to test_plot_funcs.py [`17df9d0`](https://github.com/janosh/matbench-discovery/commit/17df9d07eaada84567649918629ed105f6bab146)
- create no-structure CSVs of m3gnet and bowsr-megnet preds for faster loading used in load_df_wbm_with_preds() [`63b14aa`](https://github.com/janosh/matbench-discovery/commit/63b14aa82338366bd9344a2815ea798bac7e63f1)
- add slurm_submit_python() to use_cgcnn_ensemble.py and use_wrenformer_ensemble.py [`5d9a3b2`](https://github.com/janosh/matbench-discovery/commit/5d9a3b2b93e8df269ae6a4d37a9a1f28ed2cca3a)
- add models/voronoi/featurize_mp_wbm.py [`6932567`](https://github.com/janosh/matbench-discovery/commit/6932567ebb880f1084d1f6a9b5f8dfd21dd4180a)
- add slurm_submit_python() to featurize_mp_wbm.py [`784ee96`](https://github.com/janosh/matbench-discovery/commit/784ee96892c51149527d26339360418907d5f30e)
- make slurm_submit_python() kwargs partition, account non-optional [`d6c9a42`](https://github.com/janosh/matbench-discovery/commit/d6c9a4249f6b45ae019f040df0dd134bb8228a4f)
- record numpy + pytorch versions in train/test scripts [`7988f52`](https://github.com/janosh/matbench-discovery/commit/7988f52c580d220b0abb6b9c85df215e3973ac9a)
- slurm_submit() accept strings for slurm_flags, return timelimit, slurm_flags, pre_cmd as part of slurm_vars [`3a0173b`](https://github.com/janosh/matbench-discovery/commit/3a0173b1c5c2cb17e7f3956c2453f38fa77c7e3c)
- change plotly template to plotly_dark [`856c173`](https://github.com/janosh/matbench-discovery/commit/856c1736fc5ac754c7aeac448af3234e3cbd28e7)
- add GH actions that run pytest and markdown-link-check in CI [`92451da`](https://github.com/janosh/matbench-discovery/commit/92451da7ed00628d3d88616ab0405ec5d211f80a)
- rename e_above_mp_hull->e_above_hull_mp [`f25d91f`](https://github.com/janosh/matbench-discovery/commit/f25d91fd981f303b771dd8e1cee7d094f59b76af)
- centralize plot defaults into mb_discovery/plot_scripts/__init__.py [`f1cc667`](https://github.com/janosh/matbench-discovery/commit/f1cc6674c0847d7e37e1644527c6ee4c7fcda8d3)
- auto-regenerate API docs on production builds [`9a59191`](https://github.com/janosh/matbench-discovery/commit/9a5919149d64a4f301a5c950bf613ef2edc4c1a0)
- add df metadata to slurm_array_{m3gnet,bowsr}_wbm.py run_params [`f127a9e`](https://github.com/janosh/matbench-discovery/commit/f127a9ea60b971c54b61b0a50aca1959d5b0c917)
- set isort known_third_party = wandb [`da42037`](https://github.com/janosh/matbench-discovery/commit/da420373247db9beb5c0e3b96be61bdbd360a3db)
- wrap bowsr.optimize() in try/except [`8219c43`](https://github.com/janosh/matbench-discovery/commit/8219c4321dbe7811fc03d74981146b29720df0ce)
- more refactoring of rhys plotting scripts for clearer variable names [`a16b46a`](https://github.com/janosh/matbench-discovery/commit/a16b46a56b7a5083d4aa12ce12741cf87c252edf)
- fix models/bowsr/test_bowsr.py passing unrelaxed, not bowsr-relaxed structure to megnet [`0a2284a`](https://github.com/janosh/matbench-discovery/commit/0a2284ad14db848c500165c0203b56519a06809d)
- add m3gnet and bowsr-megnet to plot moving_hull_dist_mae_compare_models.py [`0ea1ff3`](https://github.com/janosh/matbench-discovery/commit/0ea1ff3d182e08e4babc95e3062f7d9fd9ea9109)
- add m3gnet/plots/2022-08-16-m3gnet-wbm-hull-dist-hist.pdf [`f50e1b3`](https://github.com/janosh/matbench-discovery/commit/f50e1b3dd9cf98be094f7fa3269c7e548756d1e9)
- fix models/voronoi/featurize_mp_wbm.py featurizer.set_n_jobs(1) [`8508c38`](https://github.com/janosh/matbench-discovery/commit/8508c38dafc07e22774d445fe4f9270a23896799)
- fix landing page clf metrics plot not applying inline styles [`89a33b3`](https://github.com/janosh/matbench-discovery/commit/89a33b3eaf58d8a8b9c1e3b47e8eeff0cef3a101)
- update wrenformer preds CSV path [`612d308`](https://github.com/janosh/matbench-discovery/commit/612d308e776d0d74f0790cb0d98bb0f6e7137a4d)
- fix slurm_array_m3gnet_relax_wbm.py wandb project m3gnet->matbench-discovery [`ba932c2`](https://github.com/janosh/matbench-discovery/commit/ba932c2038730586d68f09a3441a1b950edf86f5)
- matbench_discovery/plot_scripts/{moving_hull_dist_mae_compare_models -> rollng_mae_vs_hull_dist_all_models}.py [`b292e9b`](https://github.com/janosh/matbench-discovery/commit/b292e9b334803b61382adfa57288d1a15e9bfbcd)
- rename data files -all-mp-.../-mp-all-... to just -mp- [`9a41f7b`](https://github.com/janosh/matbench-discovery/commit/9a41f7ba86571935ad3dfd9a07874d507d7e09e1)
- add task_type variable to toggle between relaxing initial and final WBM structures in slurm_array_m3gnet_relax_wbm.py [`7006822`](https://github.com/janosh/matbench-discovery/commit/7006822ed5dcb30fdee1eae4b5c38f6d0219edf9)
- fix predict_from_wandb_checkpoints() didn't return ensemble_metrics due to missing target_col [`5804d13`](https://github.com/janosh/matbench-discovery/commit/5804d13b4b6dd2e4f8cc0a38aee691f9cdca3e43)
- mkdir data/mp and move some PPD pickles and MP CSEs in there [`8a8aac6`](https://github.com/janosh/matbench-discovery/commit/8a8aac65886dfcc2fce89a3b08a35364b89516c9)
- fix bad import in train_cgcnn [`1859e85`](https://github.com/janosh/matbench-discovery/commit/1859e858be8bb6aa2da92a0d906efec161c22669)
- matbench_discovery/__init__.py define CHECKPOINT_DIR = f"{ROOT}/wandb/checkpoints" [`4f1e5b6`](https://github.com/janosh/matbench-discovery/commit/4f1e5b6a8ead955af00730213f8271997100b5be)
- add ml_stability/m3gnet/plots/2022-08-02-m3gnet-wbm-hull-dist-hist.pdf [`1c3cc92`](https://github.com/janosh/matbench-discovery/commit/1c3cc92bf1f7c36c157bb1e1d9d71a0ef2cf2c63)
- slurm_submit_python() print sbatch command at submit time and in slurm job [`7b0918a`](https://github.com/janosh/matbench-discovery/commit/7b0918acd76b19cbeafac61ddadb64430399f0a2)
- remove wandb.login() calls, seems enough to do it once when initially running on new machine [`bac8551`](https://github.com/janosh/matbench-discovery/commit/bac85510f815d1a698e3f253ede9ec9660b95c7e)
- replace figshare wbm-steps-summary.csv imports with data/wbm/2022-10-19-wbm-summary.csv [`73655f6`](https://github.com/janosh/matbench-discovery/commit/73655f6051f4f7ac84f7a69d9fbc025528d8dbc5)
- mv matbench_discovery/plot_scripts -> scripts [`5860a5e`](https://github.com/janosh/matbench-discovery/commit/5860a5e52bec8b117ef708f1d28d18076227a9ab)
- migrate matbench_discovery/build_phase_diagram.py to new data/wbm/2022-10-19-wbm-cses+init-structs.json.bz2 [`f5e755e`](https://github.com/janosh/matbench-discovery/commit/f5e755eb85cc1ce843e421e9e207f029a6879967)
- fix path issue on case-sensitive file systems [`5e9f5df`](https://github.com/janosh/matbench-discovery/commit/5e9f5df44b4e238948e41760db9efc9d1791d954)
- rename col e_above_{->mp_}hull in data/2022-06-11-from-rhys/wbm-e-above-mp-hull.csv [`105e468`](https://github.com/janosh/matbench-discovery/commit/105e468f05afe24c1f8b7cd3049ef3ec1e175fd3)
- fill readme with project goals [`c369d18`](https://github.com/janosh/matbench-discovery/commit/c369d1819f789afbde2674e9ca11bfc4e50f7e49)
- improve package structure [`2be7bc3`](https://github.com/janosh/matbench-discovery/commit/2be7bc399955691b93867e9980ed7a3fc80ba12e)
- more code clean up [`b957855`](https://github.com/janosh/matbench-discovery/commit/b95785543f4c8a5cb24f8f1ba931b040d22724d8)
- rename hist_classified_stable_as_func_of_hull_dist() to hist_classified_stable_vs_hull_dist() [`24cab3d`](https://github.com/janosh/matbench-discovery/commit/24cab3d236fa8a36cc3e473c63d4ee4f970b9830)
- set python_requires>=3.9 so we can use | operator for dict merge [`03c6694`](https://github.com/janosh/matbench-discovery/commit/03c6694243a923645e77071494a6710a8e1457de)
- git rm --cached -r data/2022-06-11-from-rhys/ [`44d5133`](https://github.com/janosh/matbench-discovery/commit/44d5133769b1a572cb331a6048286c4d7804fc60)
- add test dep pytest-markdown-docs to test python code fences in markdown/svx docs [`359dfae`](https://github.com/janosh/matbench-discovery/commit/359dfae52a224f3df12dab9b1529ae32a9107edd)
- slurm_submit_python() don't print cmd if running slurm interactively [`3b2be74`](https://github.com/janosh/matbench-discovery/commit/3b2be7417a9d57fe2807e89fded77249362db8bf)
- mv ml_stability/{stability_plot_scripts->plots} [`c2a45b3`](https://github.com/janosh/matbench-discovery/commit/c2a45b3968a2904e5d3be7668ef76a71b8b8f66d)
- update file paths to CSV files in data/2022-06-11-from-rhys/ [`a9645b7`](https://github.com/janosh/matbench-discovery/commit/a9645b7aadbca26b29d8af0b94a5c3969f62e577)
- remove figures/ dir [`bf7f53a`](https://github.com/janosh/matbench-discovery/commit/bf7f53af2fe47e168b203663995faec332d51f2b)
- tweak plt.rc params [`1fb2149`](https://github.com/janosh/matbench-discovery/commit/1fb214922a7f6d6ef92892d07cf63e095db66fa4)
- dissolve dir models/wrenformer/m3gnet_train_set [`8acba18`](https://github.com/janosh/matbench-discovery/commit/8acba18161ab16487a3506fd8d75f4a11ed61aee)
- big file rename: all model scripts now named train|test_{model_name}.py [`06911dc`](https://github.com/janosh/matbench-discovery/commit/06911dcf151ded05a82205d24a8723158cc0fa90)
- add several new figures [`88dd6b4`](https://github.com/janosh/matbench-discovery/commit/88dd6b426322f669f8fed35ce0c2a54485572d44)
- data/2022-06-11-from-rhys/process_wbm_cleaned.py add warning about baked-in energy corrections in rhys' ComputedStructureEntries [`e5b099c`](https://github.com/janosh/matbench-discovery/commit/e5b099c54e632bffa08d7564bcc518fbbffb786f)
- improve readability of moving MAE scripts [`e13d624`](https://github.com/janosh/matbench-discovery/commit/e13d6244ebf83e5309480b493252eb7c72af118f)
