{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-8hjwN6W0VJz"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Trusted-AI/AIF360/blob/master/examples/sklearn/demo_mdss_bias_scan.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TMH4ugRk0JiK"
      },
      "source": [
        "# Identifying Significant Predictive Bias in Classifiers\n",
        "\n",
        "In this notebook, we attempt to recreate the analysis by Zhe Zhang and Daniel Neill in [Identifying Significant Predictive Bias in Classifiers](https://arxiv.org/pdf/1611.08292.pdf).\n",
        "\n",
        "The analysis is broken down into three steps, starting with a model trained on COMPAS decile scores only. After running bias scan, we add the distinguishing feature, priors count, to the model. We scan again and train a third model with the new subgroups accounted for. Finally, we reproduce Figure 2 from the paper."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cNtZqZph0LDC"
      },
      "outputs": [],
      "source": [
        "# Install AIF360\n",
        "!pip install 'aif360'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "muU69Pg20JiY"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import seaborn as sns\n",
        "sns.set(context='talk', style='whitegrid')\n",
        "\n",
        "from sklearn.metrics import RocCurveDisplay\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "\n",
        "from aif360.sklearn.datasets import fetch_compas\n",
        "from aif360.sklearn.metrics import mdss_bias_scan, mdss_bias_score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p9UbLZxv0Jid"
      },
      "source": [
        "Data loading"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "fGqb0ZGc0Jie",
        "outputId": "f090b129-0d8d-4573-e77d-69489bc16874"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>sex</th>\n",
              "      <th>race</th>\n",
              "      <th>age_cat</th>\n",
              "      <th>priors_count</th>\n",
              "      <th>c_charge_degree</th>\n",
              "      <th>decile_score</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>id</th>\n",
              "      <th>sex</th>\n",
              "      <th>race</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <th>Male</th>\n",
              "      <th>Other</th>\n",
              "      <td>Male</td>\n",
              "      <td>Other</td>\n",
              "      <td>Greater than 45</td>\n",
              "      <td>0</td>\n",
              "      <td>F</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <th>Male</th>\n",
              "      <th>African-American</th>\n",
              "      <td>Male</td>\n",
              "      <td>African-American</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>0</td>\n",
              "      <td>F</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <th>Male</th>\n",
              "      <th>African-American</th>\n",
              "      <td>Male</td>\n",
              "      <td>African-American</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>1 to 5</td>\n",
              "      <td>F</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <th>Male</th>\n",
              "      <th>Other</th>\n",
              "      <td>Male</td>\n",
              "      <td>Other</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>0</td>\n",
              "      <td>M</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <th>Male</th>\n",
              "      <th>Caucasian</th>\n",
              "      <td>Male</td>\n",
              "      <td>Caucasian</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>More than 5</td>\n",
              "      <td>F</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10996</th>\n",
              "      <th>Male</th>\n",
              "      <th>African-American</th>\n",
              "      <td>Male</td>\n",
              "      <td>African-American</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>0</td>\n",
              "      <td>F</td>\n",
              "      <td>7</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10997</th>\n",
              "      <th>Male</th>\n",
              "      <th>African-American</th>\n",
              "      <td>Male</td>\n",
              "      <td>African-American</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>0</td>\n",
              "      <td>F</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10999</th>\n",
              "      <th>Male</th>\n",
              "      <th>Other</th>\n",
              "      <td>Male</td>\n",
              "      <td>Other</td>\n",
              "      <td>Greater than 45</td>\n",
              "      <td>0</td>\n",
              "      <td>F</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11000</th>\n",
              "      <th>Female</th>\n",
              "      <th>African-American</th>\n",
              "      <td>Female</td>\n",
              "      <td>African-American</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>1 to 5</td>\n",
              "      <td>M</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11001</th>\n",
              "      <th>Female</th>\n",
              "      <th>Hispanic</th>\n",
              "      <td>Female</td>\n",
              "      <td>Hispanic</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>1 to 5</td>\n",
              "      <td>F</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>6172 rows × 6 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "                                  sex              race          age_cat  \\\n",
              "id    sex    race                                                          \n",
              "1     Male   Other               Male             Other  Greater than 45   \n",
              "3     Male   African-American    Male  African-American          25 - 45   \n",
              "4     Male   African-American    Male  African-American     Less than 25   \n",
              "7     Male   Other               Male             Other          25 - 45   \n",
              "8     Male   Caucasian           Male         Caucasian          25 - 45   \n",
              "...                               ...               ...              ...   \n",
              "10996 Male   African-American    Male  African-American     Less than 25   \n",
              "10997 Male   African-American    Male  African-American     Less than 25   \n",
              "10999 Male   Other               Male             Other  Greater than 45   \n",
              "11000 Female African-American  Female  African-American          25 - 45   \n",
              "11001 Female Hispanic          Female          Hispanic     Less than 25   \n",
              "\n",
              "                              priors_count c_charge_degree  decile_score  \n",
              "id    sex    race                                                         \n",
              "1     Male   Other                       0               F             1  \n",
              "3     Male   African-American            0               F             3  \n",
              "4     Male   African-American       1 to 5               F             4  \n",
              "7     Male   Other                       0               M             1  \n",
              "8     Male   Caucasian         More than 5               F             6  \n",
              "...                                    ...             ...           ...  \n",
              "10996 Male   African-American            0               F             7  \n",
              "10997 Male   African-American            0               F             3  \n",
              "10999 Male   Other                       0               F             1  \n",
              "11000 Female African-American       1 to 5               M             2  \n",
              "11001 Female Hispanic               1 to 5               F             4  \n",
              "\n",
              "[6172 rows x 6 columns]"
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "cols = ['sex', 'race', 'age_cat', 'priors_count', 'c_charge_degree', 'decile_score']\n",
        "X, y = fetch_compas(usecols=cols)\n",
        "# Quantize priors count between 0, 1-5, and >5\n",
        "X['priors_count'] = pd.cut(X['priors_count'], [-1, 0, 5, 100],\n",
        "                           labels=['0', '1 to 5', 'More than 5'])\n",
        "X"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TNU-3kNO0Jif"
      },
      "source": [
        "### 1. Decile score only"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "V-oGf_Kr0Jig",
        "outputId": "b80bbe35-bae8-47f9-c6a4-0849d2f3994a"
      },
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 45020 characters]",
            "text/plain": [
              "<Figure size 432x432 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "dec = X[['decile_score']]\n",
        "northpointe = LogisticRegression(penalty='none').fit(dec, y)\n",
        "y_prob = northpointe.predict_proba(dec)[:, 1]\n",
        "\n",
        "f, ax = plt.subplots(figsize=(6, 6))\n",
        "RocCurveDisplay.from_estimator(northpointe, dec, y, ax=ax);"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FdGrKX300Jig"
      },
      "outputs": [],
      "source": [
        "df = pd.concat([X, pd.Series(1-y_prob, name='recid_prob', index=X.index)], axis=1)\n",
        "orig_clf = df.groupby('decile_score').mean().recid_prob"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0jy-EJU80Jih"
      },
      "source": [
        "#### Privileged group\n",
        "\n",
        "\"Privileged\" in this case means the model underestimates the probability of recidivism (overestimates favorable outcomes) for this subgroup. This leads to advantage for those individuals."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9okV_MYh0Jii",
        "outputId": "7d43e67c-99f9-4c96-dbe4-4ba7edb9f762"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Function mdss_bias_scan is deprecated; Change to new interface - aif360.sklearn.detectors.mdss_detector.bias_scan by version 0.5.0.\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "({'priors_count': ['More than 5']}, 36.3302)"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "priv_sub, priv_score = mdss_bias_scan(y, y_prob, X=X, pos_label='Survived',\n",
        "                                      penalty=0.5, privileged=True)\n",
        "priv = df[priv_sub.keys()].isin(priv_sub).all(axis=1)\n",
        "priv_sub, priv_score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uwyQpJUq0Jij"
      },
      "source": [
        "Note: we show probabilities of recidivism but bias scanning is done with respect to the positive label, 'Survived'."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7Twg3LBh0Jij",
        "outputId": "5b9dd4fa-cb5a-4f15-fee6-f494e59604ed"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Observed: 71.42%\n",
            "Expected: 60.27%\n",
            "n = 1221\n"
          ]
        }
      ],
      "source": [
        "print(f'Observed: {y[priv].cat.codes.mean():.2%}')\n",
        "print(f'Expected: {df[priv].recid_prob.mean():.2%}')\n",
        "print(f'n = {sum(priv)}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g68mpYg20Jij"
      },
      "source": [
        "#### Unprivileged group\n",
        "\n",
        "\"Unprivileged\" means the model overestimates the probability of recidivism (underestimates favorable outcomes) for this subgroup. This disadvantages those individuals."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TcmWF0oE0Jik",
        "outputId": "4797e9af-8df2-43b4-ae7e-50cd36830a3b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "({'priors_count': ['0']}, 45.1434)"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "unpriv_sub, unpriv_score = mdss_bias_scan(y, y_prob, X=X, pos_label='Survived',\n",
        "                                          penalty=0.5, privileged=False)\n",
        "unpriv = df[unpriv_sub.keys()].isin(unpriv_sub).all(axis=1)\n",
        "unpriv_sub, unpriv_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "i5YcmHkL0Jik",
        "outputId": "15340c08-41c7-4043-9800-75b416891a00"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Observed: 28.63%\n",
            "Expected: 38.06%\n",
            "n = 2085\n"
          ]
        }
      ],
      "source": [
        "print(f'Observed: {y[unpriv].cat.codes.mean():.2%}')\n",
        "print(f'Expected: {df[unpriv].recid_prob.mean():.2%}')\n",
        "print(f'n = {sum(unpriv)}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WyjOYZuK0Jil"
      },
      "source": [
        "### 2. Decile score + priors count"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ECQzsBEJ0Jil"
      },
      "outputs": [],
      "source": [
        "dec = dec.assign(priors_count=X['priors_count'].cat.codes)\n",
        "northpointe = LogisticRegression(penalty='none').fit(dec, y)\n",
        "y_prob_pc = northpointe.predict_proba(dec)[:, 1]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "soKsVOlK0Jim"
      },
      "outputs": [],
      "source": [
        "df = pd.concat([X, pd.Series(1-y_prob_pc, name='recid_prob', index=X.index)], axis=1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O_yFQy-00Jim"
      },
      "source": [
        "#### Privileged group"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LUDfrmiG0Jim",
        "outputId": "3a4c7e5b-ade8-4083-bfb8-2145d5c5cffb"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Function mdss_bias_scan is deprecated; Change to new interface - aif360.sklearn.detectors.mdss_detector.bias_scan by version 0.5.0.\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "({'sex': ['Male'], 'age_cat': ['Less than 25']}, 24.49)"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "priv_sub, priv_score = mdss_bias_scan(y, y_prob_pc, X=X, pos_label='Survived',\n",
        "                                      penalty=1, privileged=True)\n",
        "priv = df[priv_sub.keys()].isin(priv_sub).all(axis=1)\n",
        "priv_sub, priv_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9QT8YdEY0Jin",
        "outputId": "39c3d98d-dbe7-4b9c-e37f-d77171ff4b3a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Observed: 60.04%\n",
            "Expected: 49.76%\n",
            "n = 1101\n",
            "unpenalized score: 26.49\n"
          ]
        }
      ],
      "source": [
        "print(f'Observed: {y[priv].cat.codes.mean():.2%}')\n",
        "print(f'Expected: {df[priv].recid_prob.mean():.2%}')\n",
        "print(f'n = {sum(priv)}')\n",
        "\n",
        "priv_unpen = mdss_bias_score(y, y_prob_pc, X=X, subset=priv_sub,\n",
        "                             pos_label='Survived', privileged=True, penalty=0)\n",
        "print(f'unpenalized score: {priv_unpen:.2f}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TyfxvmaI0Jin"
      },
      "source": [
        "#### Unprivileged group"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ymoBNKwu0Jin",
        "outputId": "c9fa465a-560b-413d-abe2-3155ce2eab69"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "({'decile_score': [2, 3, 6, 9, 10],\n",
              "  'sex': ['Female'],\n",
              "  'c_charge_degree': ['M']},\n",
              " 12.1591)"
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "unpriv_sub, unpriv_score = mdss_bias_scan(y, y_prob_pc, X=X, pos_label='Survived',\n",
        "                                          penalty=0.25, privileged=False, n_iter=25)\n",
        "unpriv = df[unpriv_sub.keys()].isin(unpriv_sub).all(axis=1)\n",
        "unpriv_sub, unpriv_score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zFRvNXCH0Jio",
        "outputId": "2d54ade4-6b77-481f-d21f-59cdeb572580"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Observed: 20.79%\n",
            "Expected: 36.96%\n",
            "n = 202\n",
            "unpenalized score: 13.91\n"
          ]
        }
      ],
      "source": [
        "print(f'Observed: {y[unpriv].cat.codes.mean():.2%}')\n",
        "print(f'Expected: {df[unpriv].recid_prob.mean():.2%}')\n",
        "print(f'n = {sum(unpriv)}')\n",
        "\n",
        "unpriv_unpen = mdss_bias_score(y, y_prob_pc, X=X, subset=unpriv_sub,\n",
        "                               pos_label='Survived', privileged=False, penalty=0)\n",
        "print(f'unpenalized score: {unpriv_unpen:.2f}')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H2PhWmK_0Jip"
      },
      "source": [
        "### 3. Decile score + priors count + top groups"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "sKAh4tzZ0Jip",
        "outputId": "7bdba24a-d496-4dba-e137-22d82ab95b45"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th>sex</th>\n",
              "      <th>race</th>\n",
              "      <th>age_cat</th>\n",
              "      <th>priors_count</th>\n",
              "      <th>c_charge_degree</th>\n",
              "      <th>decile_score</th>\n",
              "      <th>recid_prob</th>\n",
              "      <th>group</th>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>id</th>\n",
              "      <th>sex</th>\n",
              "      <th>race</th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "      <th></th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <th>Male</th>\n",
              "      <th>Other</th>\n",
              "      <td>Male</td>\n",
              "      <td>Other</td>\n",
              "      <td>Greater than 45</td>\n",
              "      <td>0</td>\n",
              "      <td>F</td>\n",
              "      <td>1</td>\n",
              "      <td>0.186358</td>\n",
              "      <td>neither</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <th>Male</th>\n",
              "      <th>African-American</th>\n",
              "      <td>Male</td>\n",
              "      <td>African-American</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>0</td>\n",
              "      <td>F</td>\n",
              "      <td>3</td>\n",
              "      <td>0.265247</td>\n",
              "      <td>neither</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <th>Male</th>\n",
              "      <th>African-American</th>\n",
              "      <td>Male</td>\n",
              "      <td>African-American</td>\n",
              "      <td>Less than 25</td>\n",
              "      <td>1 to 5</td>\n",
              "      <td>F</td>\n",
              "      <td>4</td>\n",
              "      <td>0.448002</td>\n",
              "      <td>under-estimated</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <th>Male</th>\n",
              "      <th>Other</th>\n",
              "      <td>Male</td>\n",
              "      <td>Other</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>0</td>\n",
              "      <td>M</td>\n",
              "      <td>1</td>\n",
              "      <td>0.186358</td>\n",
              "      <td>neither</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <th>Male</th>\n",
              "      <th>Caucasian</th>\n",
              "      <td>Male</td>\n",
              "      <td>Caucasian</td>\n",
              "      <td>25 - 45</td>\n",
              "      <td>More than 5</td>\n",
              "      <td>F</td>\n",
              "      <td>6</td>\n",
              "      <td>0.696115</td>\n",
              "      <td>neither</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                           sex              race          age_cat  \\\n",
              "id sex  race                                                        \n",
              "1  Male Other             Male             Other  Greater than 45   \n",
              "3  Male African-American  Male  African-American          25 - 45   \n",
              "4  Male African-American  Male  African-American     Less than 25   \n",
              "7  Male Other             Male             Other          25 - 45   \n",
              "8  Male Caucasian         Male         Caucasian          25 - 45   \n",
              "\n",
              "                         priors_count c_charge_degree  decile_score  \\\n",
              "id sex  race                                                          \n",
              "1  Male Other                       0               F             1   \n",
              "3  Male African-American            0               F             3   \n",
              "4  Male African-American       1 to 5               F             4   \n",
              "7  Male Other                       0               M             1   \n",
              "8  Male Caucasian         More than 5               F             6   \n",
              "\n",
              "                          recid_prob            group  \n",
              "id sex  race                                           \n",
              "1  Male Other               0.186358          neither  \n",
              "3  Male African-American    0.265247          neither  \n",
              "4  Male African-American    0.448002  under-estimated  \n",
              "7  Male Other               0.186358          neither  \n",
              "8  Male Caucasian           0.696115          neither  "
            ]
          },
          "execution_count": 15,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df['group'] = 'neither'\n",
        "df.loc[priv, 'group'] = 'under-estimated'\n",
        "df.loc[unpriv, 'group'] = 'over-estimated'\n",
        "df['group'] = df.group.astype('category')\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FITyXnKz0Jiq"
      },
      "outputs": [],
      "source": [
        "dec = pd.concat([dec, pd.get_dummies(df.group)], axis=1)\n",
        "northpointe = LogisticRegression(penalty='none').fit(dec, y)\n",
        "y_prob_pcg = northpointe.predict_proba(dec)[:, 1]\n",
        "df['recid_prob'] = 1 - y_prob_pcg"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HlB5K8ps0Jiq",
        "outputId": "262e523a-0ee6-4bf3-e75a-1de1cc8dfb1d"
      },
      "outputs": [
        {
          "data": {
            "image/png": "[encoded data, 73796 characters]",
            "text/plain": [
              "<Figure size 1227.9x360 with 3 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "p = sns.relplot(data=df.groupby(['decile_score', 'priors_count', 'group']).mean(),\n",
        "                x='decile_score', y='recid_prob', hue='priors_count',\n",
        "                style='priors_count', palette=['r', 'g', 'b'],\n",
        "                markers=['o', 's', '^'], col='group', s=250)\n",
        "for ax in p.axes.flatten():\n",
        "    ax.plot(range(1, 11), orig_clf, '--k')\n",
        "plt.ylim([0, 1]);\n",
        "plt.yticks(np.linspace(0, 1., 5));\n",
        "plt.xticks(range(1, 11));"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.9.7 ('aif360')",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.7"
    },
    "orig_nbformat": 4,
    "vscode": {
      "interpreter": {
        "hash": "d0c5ced7753e77a483fec8ff7063075635521cce6e0bd54998c8f174742209dd"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
