{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GFuMaPov5HlV"
   },
   "source": [
    "<!-- ### Time-Dependent Score-Based Model -->\n",
    "\n",
    "<!-- There are no restrictions on the network architecture of time-dependent score-based models, except that their output should have the same dimensionality as the input, and they should be conditioned on time.\n",
    "\n",
    "Several useful tips on architecture choice:\n",
    "* It usually performs well to use the [U-net](https://arxiv.org/abs/1505.04597) architecture as the backbone of the score network $s_\\theta(\\mathbf{x}, t)$,\n",
    "\n",
    "* We can incorporate the time information via [Gaussian random features](https://arxiv.org/abs/2006.10739). Specifically, we first sample $\\omega \\sim \\mathcal{N}(\\mathbf{0}, s^2\\mathbf{I})$ which is subsequently fixed for the model (i.e., not learnable). For a time step $t$, the corresponding Gaussian random feature is defined as\n",
    "\\begin{align}\n",
    "  [\\sin(2\\pi \\omega t) ; \\cos(2\\pi \\omega t)],\n",
    "\\end{align}\n",
    "where $[\\vec{a} ; \\vec{b}]$ denotes the concatenation of vector $\\vec{a}$ and $\\vec{b}$. This Gaussian random feature can be used as an encoding for time step $t$ so that the score network can condition on $t$ by incorporating this encoding. We will see this further in the code.\n",
    "\n",
    "* We can rescale the output of the U-net by $1/\\sqrt{\\mathbb{E}[\\|\\nabla_{\\mathbf{x}}\\log p_{0t}(\\mathbf{x}(t) \\mid \\mathbf{x}(0)) \\|_2^2]}$. This is because the optimal $s_\\theta(\\mathbf{x}(t), t)$ has an $\\ell_2$-norm close to $\\mathbb{E}[\\|\\nabla_{\\mathbf{x}}\\log p_{0t}(\\mathbf{x}(t) \\mid \\mathbf{x}(0))]\\|_2$, and the rescaling helps capture the norm of the true score. Recall that the training objective contains sums of the form\n",
    "\\begin{align*}\n",
    "\\mathbf{E}_{\\mathbf{x}(t) \\sim p_{0t}(\\mathbf{x}(t) \\mid \\mathbf{x}(0))}[ \\|s_\\theta(\\mathbf{x}(t), t) - \\nabla_{\\mathbf{x}(t)}\\log p_{0t}(\\mathbf{x}(t) \\mid \\mathbf{x}(0))\\|_2^2].\n",
    "\\end{align*}\n",
    "Therefore, it is natural to expect that the optimal score model $s_\\theta(\\mathbf{x}, t) \\approx \\nabla_{\\mathbf{x}(t)} \\log p_{0t}(\\mathbf{x}(t) \\mid \\mathbf{x}(0))$.\n",
    "\n",
    "* Use [exponential moving average](https://discuss.pytorch.org/t/how-to-apply-exponential-moving-average-decay-for-variables/10856/3) (EMA) of weights when sampling. This can greatly improve sample quality, but requires slightly longer training time, and requires more work in implementation. We do not include this in this tutorial, but highly recommend it when you employ score-based generative modeling to tackle more challenging real problems. -->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = 'mps'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "cellView": "form",
    "id": "YyQtV7155Nht"
   },
   "outputs": [],
   "source": [
    "#@title Defining a time-dependent score-based model (double click to expand or collapse)\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import numpy as np\n",
    "\n",
    "class GaussianFourierProjection(nn.Module):\n",
    "  \"\"\"Gaussian random features for encoding time steps.\"\"\"\n",
    "  def __init__(self, embed_dim, scale=30.):\n",
    "    super().__init__()\n",
    "    # Randomly sample weights during initialization. These weights are fixed\n",
    "    # during optimization and are not trainable.\n",
    "    self.W = nn.Parameter(torch.randn(embed_dim // 2) * scale, requires_grad=False)\n",
    "  def forward(self, x):\n",
    "    x_proj = x[:, None] * self.W[None, :] * 2 * np.pi\n",
    "    return torch.cat([torch.sin(x_proj), torch.cos(x_proj)], dim=-1)\n",
    "\n",
    "\n",
    "class Dense(nn.Module):\n",
    "  \"\"\"A fully connected layer that reshapes outputs to feature maps.\"\"\"\n",
    "  def __init__(self, input_dim, output_dim):\n",
    "    super().__init__()\n",
    "    self.dense = nn.Linear(input_dim, output_dim)\n",
    "  def forward(self, x):\n",
    "    return self.dense(x)[..., None, None]\n",
    "\n",
    "\n",
    "class ScoreNet(nn.Module):\n",
    "  \"\"\"A time-dependent score-based model built upon U-Net architecture.\"\"\"\n",
    "\n",
    "  def __init__(self, marginal_prob_std, channels=[32, 64, 128, 256], embed_dim=256):\n",
    "    \"\"\"Initialize a time-dependent score-based network.\n",
    "\n",
    "    Args:\n",
    "      marginal_prob_std: A function that takes time t and gives the standard\n",
    "        deviation of the perturbation kernel p_{0t}(x(t) | x(0)).\n",
    "      channels: The number of channels for feature maps of each resolution.\n",
    "      embed_dim: The dimensionality of Gaussian random feature embeddings.\n",
    "    \"\"\"\n",
    "    super().__init__()\n",
    "    # Gaussian random feature embedding layer for time\n",
    "    self.embed = nn.Sequential(GaussianFourierProjection(embed_dim=embed_dim),\n",
    "         nn.Linear(embed_dim, embed_dim))\n",
    "    # Encoding layers where the resolution decreases\n",
    "    self.conv1 = nn.Conv2d(1, channels[0], 3, stride=1, bias=False)\n",
    "    self.dense1 = Dense(embed_dim, channels[0])\n",
    "    self.gnorm1 = nn.GroupNorm(4, num_channels=channels[0])\n",
    "    self.conv2 = nn.Conv2d(channels[0], channels[1], 3, stride=2, bias=False)\n",
    "    self.dense2 = Dense(embed_dim, channels[1])\n",
    "    self.gnorm2 = nn.GroupNorm(32, num_channels=channels[1])\n",
    "    self.conv3 = nn.Conv2d(channels[1], channels[2], 3, stride=2, bias=False)\n",
    "    self.dense3 = Dense(embed_dim, channels[2])\n",
    "    self.gnorm3 = nn.GroupNorm(32, num_channels=channels[2])\n",
    "    self.conv4 = nn.Conv2d(channels[2], channels[3], 3, stride=2, bias=False)\n",
    "    self.dense4 = Dense(embed_dim, channels[3])\n",
    "    self.gnorm4 = nn.GroupNorm(32, num_channels=channels[3])\n",
    "\n",
    "    # Decoding layers where the resolution increases\n",
    "    self.tconv4 = nn.ConvTranspose2d(channels[3], channels[2], 3, stride=2, bias=False)\n",
    "    self.dense5 = Dense(embed_dim, channels[2])\n",
    "    self.tgnorm4 = nn.GroupNorm(32, num_channels=channels[2])\n",
    "    self.tconv3 = nn.ConvTranspose2d(channels[2] + channels[2], channels[1], 3, stride=2, bias=False, output_padding=1)\n",
    "    self.dense6 = Dense(embed_dim, channels[1])\n",
    "    self.tgnorm3 = nn.GroupNorm(32, num_channels=channels[1])\n",
    "    self.tconv2 = nn.ConvTranspose2d(channels[1] + channels[1], channels[0], 3, stride=2, bias=False, output_padding=1)\n",
    "    self.dense7 = Dense(embed_dim, channels[0])\n",
    "    self.tgnorm2 = nn.GroupNorm(32, num_channels=channels[0])\n",
    "    self.tconv1 = nn.ConvTranspose2d(channels[0] + channels[0], 1, 3, stride=1)\n",
    "\n",
    "    # The swish activation function\n",
    "    self.act = lambda x: x * torch.sigmoid(x)\n",
    "    self.marginal_prob_std = marginal_prob_std\n",
    "\n",
    "  def forward(self, x, t):\n",
    "    # Obtain the Gaussian random feature embedding for t\n",
    "    embed = self.act(self.embed(t))\n",
    "    # Encoding path\n",
    "    h1 = self.conv1(x)\n",
    "    ## Incorporate information from t\n",
    "    h1 += self.dense1(embed)\n",
    "    ## Group normalization\n",
    "    h1 = self.gnorm1(h1)\n",
    "    h1 = self.act(h1)\n",
    "    h2 = self.conv2(h1)\n",
    "    h2 += self.dense2(embed)\n",
    "    h2 = self.gnorm2(h2)\n",
    "    h2 = self.act(h2)\n",
    "    h3 = self.conv3(h2)\n",
    "    h3 += self.dense3(embed)\n",
    "    h3 = self.gnorm3(h3)\n",
    "    h3 = self.act(h3)\n",
    "    h4 = self.conv4(h3)\n",
    "    h4 += self.dense4(embed)\n",
    "    h4 = self.gnorm4(h4)\n",
    "    h4 = self.act(h4)\n",
    "\n",
    "    # Decoding path\n",
    "    h = self.tconv4(h4)\n",
    "    ## Skip connection from the encoding path\n",
    "    h += self.dense5(embed)\n",
    "    h = self.tgnorm4(h)\n",
    "    h = self.act(h)\n",
    "    h = self.tconv3(torch.cat([h, h3], dim=1))\n",
    "    h += self.dense6(embed)\n",
    "    h = self.tgnorm3(h)\n",
    "    h = self.act(h)\n",
    "    h = self.tconv2(torch.cat([h, h2], dim=1))\n",
    "    h += self.dense7(embed)\n",
    "    h = self.tgnorm2(h)\n",
    "    h = self.act(h)\n",
    "    h = self.tconv1(torch.cat([h, h1], dim=1))\n",
    "\n",
    "    # Normalize output\n",
    "    h = h / self.marginal_prob_std(t)[:, None, None, None]\n",
    "    return h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PpJSwfyY6mJz"
   },
   "source": [
    "<!-- ## Training with Weighted Sum of Denoising Score Matching Objectives\n",
    "\n",
    "Now let's get our hands dirty on training. First of all, we need to specify an SDE that perturbs the data distribution $p_0$ to a prior distribution $p_T$. We choose the following SDE\n",
    "\\begin{align*}\n",
    "d \\mathbf{x} = \\sigma^t d\\mathbf{w}, \\quad t\\in[0,1]\n",
    "\\end{align*}\n",
    "In this case,\n",
    "\\begin{align*}\n",
    "p_{0t}(\\mathbf{x}(t) \\mid \\mathbf{x}(0)) = \\mathcal{N}\\bigg(\\mathbf{x}(t); \\mathbf{x}(0), \\frac{1}{2\\log \\sigma}(\\sigma^{2t} - 1) \\mathbf{I}\\bigg)\n",
    "\\end{align*}\n",
    "and we can choose the weighting function $\\lambda(t) = \\frac{1}{2 \\log \\sigma}(\\sigma^{2t} - 1)$.\n",
    "\n",
    "When $\\sigma$ is large, the prior distribution, $p_{t=1}$ is\n",
    "\\begin{align*}\n",
    "\\int p_0(\\mathbf{y})\\mathcal{N}\\bigg(\\mathbf{x}; \\mathbf{y}, \\frac{1}{2 \\log \\sigma}(\\sigma^2 - 1)\\mathbf{I}\\bigg) d \\mathbf{y} \\approx \\mathbf{N}\\bigg(\\mathbf{x}; \\mathbf{0}, \\frac{1}{2 \\log \\sigma}(\\sigma^2 - 1)\\mathbf{I}\\bigg),\n",
    "\\end{align*}\n",
    "which is approximately independent of the data distribution and is easy to sample from.\n",
    "\n",
    "Intuitively, this SDE captures a continuum of Gaussian perturbations with variance function $\\frac{1}{2 \\log \\sigma}(\\sigma^{2t} - 1)$. This continuum of perturbations allows us to gradually transfer samples from a data distribution $p_0$ to a simple Gaussian distribution $p_1$. -->"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "LZC7wrOvxLdL"
   },
   "outputs": [],
   "source": [
    "#@title Set up the SDE\n",
    "import functools\n",
    "\n",
    "device = 'mps' #@param ['cuda', 'cpu'] {'type':'string'}\n",
    "\n",
    "def marginal_prob_std(t, sigma):\n",
    "  \"\"\"Compute the mean and standard deviation of $p_{0t}(x(t) | x(0))$.\n",
    "\n",
    "  Args:\n",
    "    t: A vector of time steps.\n",
    "    sigma: The $\\sigma$ in our SDE.\n",
    "\n",
    "  Returns:\n",
    "    The standard deviation.\n",
    "  \"\"\"\n",
    "  t = torch.tensor(t, device=device)\n",
    "  return torch.sqrt((sigma**(2 * t) - 1.) / 2. / np.log(sigma))\n",
    "\n",
    "def diffusion_coeff(t, sigma):\n",
    "  \"\"\"Compute the diffusion coefficient of our SDE.\n",
    "\n",
    "  Args:\n",
    "    t: A vector of time steps.\n",
    "    sigma: The $\\sigma$ in our SDE.\n",
    "\n",
    "  Returns:\n",
    "    The vector of diffusion coefficients.\n",
    "  \"\"\"\n",
    "  return torch.tensor(sigma**t, device=device)\n",
    "\n",
    "sigma =  25.0#@param {'type':'number'}\n",
    "marginal_prob_std_fn = functools.partial(marginal_prob_std, sigma=sigma)\n",
    "diffusion_coeff_fn = functools.partial(diffusion_coeff, sigma=sigma)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "zOsoqPdXHuL5"
   },
   "outputs": [],
   "source": [
    "#@title Define the loss function (double click to expand or collapse)\n",
    "\n",
    "def loss_fn(model, x, marginal_prob_std, eps=1e-5):\n",
    "  \"\"\"The loss function for training score-based generative models.\n",
    "\n",
    "  Args:\n",
    "    model: A PyTorch model instance that represents a\n",
    "      time-dependent score-based model.\n",
    "    x: A mini-batch of training data.\n",
    "    marginal_prob_std: A function that gives the standard deviation of\n",
    "      the perturbation kernel.\n",
    "    eps: A tolerance value for numerical stability.\n",
    "  \"\"\"\n",
    "  random_t = torch.rand(x.shape[0], device=x.device) * (1. - eps) + eps\n",
    "  z = torch.randn_like(x)\n",
    "  std = marginal_prob_std(random_t)\n",
    "  perturbed_x = x + z * std[:, None, None, None]\n",
    "  score = model(perturbed_x, random_t)\n",
    "  loss = torch.mean(torch.sum((score * std[:, None, None, None] + z)**2, dim=(1,2,3)))\n",
    "  # l1_norm = sum(p.abs().sum() for p in model.parameters())\n",
    "  # loss=loss+1e-6*l1_norm\n",
    "  # loss=loss+0.002*(score.norm(p=1,dim=(-1,-2)).sum(dim=(0,1)))\n",
    "  return loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 472
    },
    "id": "8PPsLx4dGCGa",
    "outputId": "6c7b0db8-4f61-453e-8833-a5f3554cf3bf"
   },
   "outputs": [],
   "source": [
    "# #@title Training (double click to expand or collapse)\n",
    "\n",
    "import torch\n",
    "import functools\n",
    "from torch.optim import Adam\n",
    "from torch.utils.data import DataLoader\n",
    "import torchvision.transforms as transforms\n",
    "from torchvision.datasets import MNIST\n",
    "from torchvision.datasets import FashionMNIST\n",
    "# from tqdm.notebook import tqdm\n",
    "import tqdm\n",
    "import functools\n",
    "\n",
    "\n",
    "\n",
    "# score_model = torch.nn.DataParallel(ScoreNet(marginal_prob_std=marginal_prob_std_fn))\n",
    "# score_model = score_model.to(device)\n",
    "# # ckpt = torch.load('ckpt0.0015F.pth', map_location=device)\n",
    "# # score_model.load_state_dict(ckpt)\n",
    "\n",
    "# n_epochs =   100#@param {'type':'integer'}\n",
    "# ## size of a mini-batch\n",
    "batch_size =  256 #@param {'type':'integer'}\n",
    "# ## learning rate\n",
    "# lr=1e-4 #@param {'type':'number'}\n",
    "\n",
    "dataset = FashionMNIST('.', train=True, transform=transforms.ToTensor(), download=True)\n",
    "data_loader = DataLoader(dataset, batch_size=batch_size, shuffle=True, num_workers=4)\n",
    "\n",
    "# optimizer = Adam(score_model.parameters(), lr=lr)\n",
    "# # tqdm_epoch = tqdm.trange(n_epochs)\n",
    "# for epoch in range(n_epochs):\n",
    "#   if epoch% 10 == 0:\n",
    "#   #  print(f'epoch={epoch}')\n",
    "#   #  torch.save(score_model.state_dict(), 'ckpt0.0014F.pth')\n",
    "#   avg_loss = 0.\n",
    "#   num_items = 0\n",
    "#   for x, y in data_loader:\n",
    "#     x = x.to(device)\n",
    "#     loss = loss_fn(score_model, x, marginal_prob_std_fn)\n",
    "#     optimizer.zero_grad()\n",
    "#     loss.backward()\n",
    "#     optimizer.step()\n",
    "#     avg_loss += loss.item() * x.shape[0]\n",
    "#     num_items += x.shape[0]\n",
    "#   # Print the averaged training loss so far.\n",
    "#   # tqdm_epoch.set_description('Average Loss: {:5f}'.format(avg_loss / num_items))\n",
    "#   # Update the checkpoint after each epoch of training.\n",
    "#   torch.save(score_model.state_dict(), 'ckpt0.0014F.pth')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# score_model = torch.nn.DataParallel(ScoreNet(marginal_prob_std=marginal_prob_std_fn))\n",
    "# score_model = score_model.to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "id": "qW1HaPZb9gDM"
   },
   "outputs": [],
   "source": [
    "#@title Define the Predictor-Corrector sampler (double click to expand or collapse)\n",
    "\n",
    "signal_to_noise_ratio = 0.18 #@param {'type':'number'}\n",
    "\n",
    "## The number of sampling steps.\n",
    "# num_steps =  500#@param {'type':'integer'}\n",
    "num_steps =  500\n",
    "\n",
    "def pc_sampler(score_model,\n",
    "               marginal_prob_std,\n",
    "               diffusion_coeff,\n",
    "               batch_size=64,\n",
    "               num_steps=num_steps,\n",
    "               snr=signal_to_noise_ratio,\n",
    "               device='cuda',\n",
    "               eps=1e-3):\n",
    "  \"\"\"Generate samples from score-based models with Predictor-Corrector method.\n",
    "\n",
    "  Args:\n",
    "    score_model: A PyTorch model that represents the time-dependent score-based model.\n",
    "    marginal_prob_std: A function that gives the standard deviation\n",
    "      of the perturbation kernel.\n",
    "    diffusion_coeff: A function that gives the diffusion coefficient\n",
    "      of the SDE.\n",
    "    batch_size: The number of samplers to generate by calling this function once.\n",
    "    num_steps: The number of sampling steps.\n",
    "      Equivalent to the number of discretized time steps.\n",
    "    device: 'cuda' for running on GPUs, and 'cpu' for running on CPUs.\n",
    "    eps: The smallest time step for numerical stability.\n",
    "\n",
    "  Returns:\n",
    "    Samples.\n",
    "  \"\"\"\n",
    "  # t = torch.ones(batch_size, device=device)\n",
    "  # init_x = torch.randn(batch_size, 1, 28, 28, device=device) * marginal_prob_std(t)[:, None, None, None]\n",
    "  # time_steps = np.linspace(1., eps, num_steps)\n",
    "  # step_size = time_steps[0] - time_steps[1]\n",
    "  # x = init_x\n",
    "  # with torch.no_grad():\n",
    "  #   for time_step in tqdm.notebook.tqdm(time_steps):\n",
    "  #     batch_time_step = torch.ones(batch_size, device=device) * time_step\n",
    "  #     # Corrector step (Langevin MCMC)\n",
    "  #     grad = score_model(x, batch_time_step)\n",
    "  #     grad_norm = torch.norm(grad.reshape(grad.shape[0], -1), dim=-1).mean()\n",
    "  #     noise_norm = np.sqrt(np.prod(x.shape[1:]))\n",
    "  #     langevin_step_size = 2 * (snr * noise_norm / grad_norm)**2\n",
    "  #     x = x + langevin_step_size * grad + torch.sqrt(2 * langevin_step_size) * torch.randn_like(x)\n",
    "\n",
    "  #     # Predictor step (Euler-Maruyama)\n",
    "  #     g = diffusion_coeff(batch_time_step)\n",
    "  #     x_mean = x + (g**2)[:, None, None, None] * score_model(x, batch_time_step) * step_size\n",
    "  #     x = x_mean + torch.sqrt(g**2 * step_size)[:, None, None, None] * torch.randn_like(x)\n",
    "\n",
    "  #   # The last step does not include any noise\n",
    "  #   return x_mean\n",
    "  t = torch.ones(batch_size, device=device)\n",
    "  init_x = torch.randn(batch_size, 1, 28, 28, device=device) * marginal_prob_std(t)[:, None, None, None]\n",
    "  time_steps = np.linspace(1., eps, num_steps)\n",
    "  step_size = time_steps[0] - time_steps[1]\n",
    "  x = init_x\n",
    "\n",
    "  with torch.no_grad():\n",
    "    for time_step in time_steps:\n",
    "        batch_time_step = torch.ones(batch_size, device=device) * time_step\n",
    "        \n",
    "        # Corrector step (Langevin MCMC)\n",
    "        grad = score_model(x, batch_time_step)\n",
    "        grad_norm = torch.norm(grad.reshape(grad.shape[0], -1), dim=-1).mean()\n",
    "        noise_norm = np.sqrt(np.prod(x.shape[1:]))\n",
    "        langevin_step_size = 2 * (snr * noise_norm / grad_norm) ** 2\n",
    "        x = x + langevin_step_size * grad + torch.sqrt(2 * langevin_step_size) * torch.randn_like(x)\n",
    "\n",
    "        # Predictor step (Euler-Maruyama)\n",
    "        g = diffusion_coeff(batch_time_step)\n",
    "        x_mean = x + (g**2)[:, None, None, None] * score_model(x, batch_time_step) * step_size\n",
    "        x = x_mean + torch.sqrt(g**2 * step_size)[:, None, None, None] * torch.randn_like(x)\n",
    "\n",
    "    # The last step does not include any noise\n",
    "    return x_mean\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000,
     "referenced_widgets": [
      "9bc9324578c8415a9b26c5ece91c6834",
      "760e743cf93d406980027bd77f93e940",
      "0ab235cfa9334ac38b5a8c3567542405",
      "e1df9f66824947798ccdbd8c6344b4f5",
      "e71f07638f2547a7bad3608002b9d9dd",
      "8f639040c14d457e91d8c7276fda774a",
      "9e3707cfab9d4c01a4803fc1172b88b7",
      "b75a682da6b546c1a2e0f3949355ee38",
      "81f614f73b91433a9aaa0073eb581c6a",
      "70a1e66a26da467780e230d2eaa2aa2a",
      "7b95dbf1175d40fa8e91984f5836130f",
      "a1e3513f993c40da89335f88511c34aa",
      "a72c6b33f0b841e49563484b6c855bbb",
      "0289ce01453a4e56a2b7aeff1758ea58",
      "2a75db1b7139408e9d0738619ac0fc50",
      "9461bce4a5e24cfb8fb5f8126c523a22",
      "ea3944dfec8a4e40be890fc14fb6b311",
      "e2de4bb0e7134d45a6c93312f18875e1",
      "9332ee2ade77412e835d9337804ec095",
      "23770f9c897d4047a3ed7f58d3cf26af",
      "86269cdc86c3494189a27dada14d54d2",
      "46dc3a56e58a435f9933388edb6b6f9b",
      "887a44c27fe54f4fa984570d61852eb7",
      "fe54f5602d034e5398d8284ea2116d3a",
      "2fffe596e6e84e0a801b334114f1be30",
      "49dbf3024984495f9d30436d430b3473",
      "3abaf533ddd4421a885a19a5a87e80c4",
      "7b83678a32314681a7619898bc2b1c3d",
      "a85ac50e6530477dac034dd095f262ff",
      "c713abe7231b450b99809ff53ae63804",
      "7bd57e2af1a9428a9f51fb837921f200",
      "6241852f9a3145cab0a13f1ccff9b9ae",
      "bef901dcc82c48308a8185cc4dbe06b4"
     ]
    },
    "id": "kKoAPnr7Pf2B",
    "outputId": "8ca58072-f43f-413a-ee3a-d9d71afdef57"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6r/vdhtmmgj6hj3r0bc6c274h7r0000gq/T/ipykernel_19068/1680157158.py:16: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  t = torch.tensor(t, device=device)\n",
      "/var/folders/6r/vdhtmmgj6hj3r0bc6c274h7r0000gq/T/ipykernel_19068/1680157158.py:29: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  return torch.tensor(sigma**t, device=device)\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 569580 characters]",
      "text/plain": [
       "<Figure size 600x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6r/vdhtmmgj6hj3r0bc6c274h7r0000gq/T/ipykernel_19068/1680157158.py:16: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  t = torch.tensor(t, device=device)\n",
      "/var/folders/6r/vdhtmmgj6hj3r0bc6c274h7r0000gq/T/ipykernel_19068/1680157158.py:29: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  return torch.tensor(sigma**t, device=device)\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 560836 characters]",
      "text/plain": [
       "<Figure size 600x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#@title Sampling (double click to expand or collapse)\n",
    "\n",
    "from torchvision.utils import make_grid\n",
    "from tqdm.notebook import tqdm\n",
    "import tqdm\n",
    "import random\n",
    "random.seed(42) \n",
    "device = 'mps' #@param ['cuda', 'cpu'] {'type':'string'}\n",
    "\n",
    "score_model = torch.nn.DataParallel(ScoreNet(marginal_prob_std=marginal_prob_std_fn))\n",
    "score_model = score_model.to(device)\n",
    "## Load the pre-trained checkpoint from disk.\n",
    "ckpt = torch.load('ckpt0.002F.pth', map_location=device)\n",
    "# ckpt = torch.load('ckpt0.0001F.pth', map_location=device)\n",
    "\n",
    "score_model.load_state_dict(ckpt)\n",
    "num_steps=500\n",
    "sample_batch_size = 256 #@param {'type':'integer'}\n",
    "sampler = pc_sampler\n",
    "\n",
    "# ## Generate samples using the specified sampler.\n",
    "# samples = sampler(score_model,\n",
    "#                   marginal_prob_std_fn,\n",
    "#                   diffusion_coeff_fn,\n",
    "#                   sample_batch_size,\n",
    "#                   num_steps=num_steps,\n",
    "#                   device=device)\n",
    "\n",
    "# ## Sample visualization.\n",
    "# samples = samples.clamp(0.0, 1.0)\n",
    "# %matplotlib inline\n",
    "# import matplotlib.pyplot as plt\n",
    "# sample_grid = make_grid(samples, nrow=int(np.sqrt(sample_batch_size)))\n",
    "\n",
    "# plt.figure(figsize=(6,6))\n",
    "# plt.axis('off')\n",
    "# plt.imshow(sample_grid.permute(1, 2, 0).cpu(), vmin=0., vmax=1.)\n",
    "# # plt.title(f\"Sampling for regularized score function in T={num_steps} steps\")\n",
    "# plt.show()\n",
    "\n",
    "num_steps=70\n",
    "sampler = pc_sampler\n",
    "\n",
    "samples = sampler(score_model,\n",
    "                  marginal_prob_std_fn,\n",
    "                  diffusion_coeff_fn,\n",
    "                  sample_batch_size,\n",
    "                  num_steps=num_steps,\n",
    "                  device=device)\n",
    "\n",
    "# Sample visualization.\n",
    "samples = samples.clamp(0.0, 1.0)\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "sample_grid = make_grid(samples, nrow=int(np.sqrt(sample_batch_size)))\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "plt.axis('off')\n",
    "plt.imshow(sample_grid.permute(1, 2, 0).cpu(), vmin=0., vmax=1.)\n",
    "# plt.title(f\"Sampling for regularized score function in T={num_steps} steps\")\n",
    "plt.show()\n",
    "\n",
    "num_steps=50\n",
    "sampler = pc_sampler\n",
    "\n",
    "samples = sampler(score_model,\n",
    "                  marginal_prob_std_fn,\n",
    "                  diffusion_coeff_fn,\n",
    "                  sample_batch_size,\n",
    "                  num_steps=num_steps,\n",
    "                  device=device)\n",
    "\n",
    "## Sample visualization.\n",
    "samples = samples.clamp(0.0, 1.0)\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "sample_grid = make_grid(samples, nrow=int(np.sqrt(sample_batch_size)))\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "plt.axis('off')\n",
    "plt.imshow(sample_grid.permute(1, 2, 0).cpu(), vmin=0., vmax=1.)\n",
    "# plt.title(f\"Sampling for regularized score function in T={num_steps} steps\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000,
     "referenced_widgets": [
      "db72ad1c1b324850b63b8d3e6a869de1",
      "fe0bb33b887a4e9abb8ba7f725b3caa2",
      "4ab591c4703e467e9eab4ea0401c530c",
      "5d64005f60b041fa9d64177f20c438e4",
      "995264e33ccb4d22be486db2bf3e293a",
      "951a6d59d2d54544be75a24c69390e44",
      "684bfc28681f49139ef249e72610f3e8",
      "d7a3b1e3e45f408fa8b1f4adf9e1f0d3",
      "5dc6db51965d40898ff1b0cad8b9d766",
      "b85ad7a4c72243da9af06bbfdf131123",
      "785e14ef12754e48a3543e1547928a0c",
      "c7af51326f2345d98023418c91cb3ac8",
      "06f53a896bd347f6adf30f75a95e0862",
      "561b1fcf29c84863b68bab77af299ff5",
      "024b913f921e42c59ac703113d1d343a",
      "789983db1c6846df93c88638c6916997",
      "3c920dea810342fea4978876495f51de",
      "ec685db8f2c14f12af7c449d1e7a12d5",
      "927519a980f94ea486de7b50d2ccdebe",
      "92b0189af44445b8b826225d71eade30",
      "9a5530b2b2bb4ae8997ce9692f77c128",
      "88889801c50c4c68af5fe6939f11dffa",
      "3ad0938c83874a9cadc85966f34cd989",
      "1bc263d9e46b470491aa5e65cbcf9809",
      "8c8060f3481c4a69995ed313d1fd29c5",
      "f953646b0c1e47a0862e3779db5b5b85",
      "e50ce9490e184933b831613b4af4a4c3",
      "a4e2549b24334c49bb96306d1b3cf389",
      "a74e1a0ab5e14bb9824db2cba20e458e",
      "b43846c7cfd64b54ad04350403689c51",
      "5262032e2fef4868a81dbe9c7f64f816",
      "53d0a46d2f6d4939919d5b1a573fb8f6",
      "2dbbab648fd2402c9daf0b217222df40"
     ]
    },
    "id": "4E8XTB1KBcnn",
    "outputId": "66f4694f-ebc0-4111-aa16-09aac3f9ffa8"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6r/vdhtmmgj6hj3r0bc6c274h7r0000gq/T/ipykernel_19068/1680157158.py:16: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  t = torch.tensor(t, device=device)\n",
      "/var/folders/6r/vdhtmmgj6hj3r0bc6c274h7r0000gq/T/ipykernel_19068/1680157158.py:29: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).\n",
      "  return torch.tensor(sigma**t, device=device)\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 362196 characters]",
      "text/plain": [
       "<Figure size 600x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 809488 characters]",
      "text/plain": [
       "<Figure size 600x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 812564 characters]",
      "text/plain": [
       "<Figure size 600x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#@title Sampling (double click to expand or collapse)\n",
    "\n",
    "from torchvision.utils import make_grid\n",
    "import random\n",
    "random.seed(42) \n",
    "\n",
    "## Load the pre-trained checkpoint from disk.\n",
    "ckpt = torch.load('ckptF.pth', map_location=device)\n",
    "score_model.load_state_dict(ckpt)\n",
    "num_steps=500\n",
    "sampler = pc_sampler\n",
    "\n",
    "## Generate samples using the specified sampler.\n",
    "samplesB = sampler(score_model,\n",
    "                  marginal_prob_std_fn,\n",
    "                  diffusion_coeff_fn,\n",
    "                  sample_batch_size,\n",
    "                  num_steps=num_steps,\n",
    "                  device=device)\n",
    "\n",
    "## Sample visualization.\n",
    "samplesB = samplesB.clamp(0.0, 1.0)\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "sample_grid = make_grid(samplesB, nrow=int(np.sqrt(sample_batch_size)))\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "plt.axis('off')\n",
    "plt.imshow(sample_grid.permute(1, 2, 0).cpu(), vmin=0., vmax=1.)\n",
    "plt.title(f\"Sampling for original score function in T={num_steps} steps\")\n",
    "plt.show()\n",
    "\n",
    "num_steps=50\n",
    "sampler = pc_sampler\n",
    "\n",
    "samples = sampler(score_model,\n",
    "                  marginal_prob_std_fn,\n",
    "                  diffusion_coeff_fn,\n",
    "                  sample_batch_size,\n",
    "                  num_steps=num_steps,\n",
    "                  device=device)\n",
    "\n",
    "## Sample visualization.\n",
    "samples = samples.clamp(0.0, 1.0)\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "sample_grid = make_grid(samples, nrow=int(np.sqrt(sample_batch_size)))\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "plt.axis('off')\n",
    "plt.imshow(sample_grid.permute(1, 2, 0).cpu(), vmin=0., vmax=1.)\n",
    "plt.title(f\"Sampling for original score function in T={num_steps} steps\")\n",
    "plt.show()\n",
    "\n",
    "num_steps=20\n",
    "sampler = pc_sampler\n",
    "\n",
    "samples = sampler(score_model,\n",
    "                  marginal_prob_std_fn,\n",
    "                  diffusion_coeff_fn,\n",
    "                  sample_batch_size,\n",
    "                  num_steps=num_steps,\n",
    "                  device=device)\n",
    "\n",
    "## Sample visualization.\n",
    "samples = samples.clamp(0.0, 1.0)\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "sample_grid = make_grid(samples, nrow=int(np.sqrt(sample_batch_size)))\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "plt.axis('off')\n",
    "plt.imshow(sample_grid.permute(1, 2, 0).cpu(), vmin=0., vmax=1.)\n",
    "plt.title(f\"Sampling for original score function in T={num_steps} steps\")\n",
    "plt.show()\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "024b913f921e42c59ac703113d1d343a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9a5530b2b2bb4ae8997ce9692f77c128",
      "placeholder": "​",
      "style": "IPY_MODEL_88889801c50c4c68af5fe6939f11dffa",
      "value": " 50/50 [00:15&lt;00:00,  2.90it/s]"
     }
    },
    "0289ce01453a4e56a2b7aeff1758ea58": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9332ee2ade77412e835d9337804ec095",
      "max": 50,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_23770f9c897d4047a3ed7f58d3cf26af",
      "value": 50
     }
    },
    "06f53a896bd347f6adf30f75a95e0862": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3c920dea810342fea4978876495f51de",
      "placeholder": "​",
      "style": "IPY_MODEL_ec685db8f2c14f12af7c449d1e7a12d5",
      "value": "100%"
     }
    },
    "0ab235cfa9334ac38b5a8c3567542405": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b75a682da6b546c1a2e0f3949355ee38",
      "max": 500,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_81f614f73b91433a9aaa0073eb581c6a",
      "value": 500
     }
    },
    "1bc263d9e46b470491aa5e65cbcf9809": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a4e2549b24334c49bb96306d1b3cf389",
      "placeholder": "​",
      "style": "IPY_MODEL_a74e1a0ab5e14bb9824db2cba20e458e",
      "value": "100%"
     }
    },
    "23770f9c897d4047a3ed7f58d3cf26af": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "2a75db1b7139408e9d0738619ac0fc50": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_86269cdc86c3494189a27dada14d54d2",
      "placeholder": "​",
      "style": "IPY_MODEL_46dc3a56e58a435f9933388edb6b6f9b",
      "value": " 50/50 [00:15&lt;00:00,  2.90it/s]"
     }
    },
    "2dbbab648fd2402c9daf0b217222df40": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2fffe596e6e84e0a801b334114f1be30": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c713abe7231b450b99809ff53ae63804",
      "max": 20,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_7bd57e2af1a9428a9f51fb837921f200",
      "value": 20
     }
    },
    "3abaf533ddd4421a885a19a5a87e80c4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3ad0938c83874a9cadc85966f34cd989": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_1bc263d9e46b470491aa5e65cbcf9809",
       "IPY_MODEL_8c8060f3481c4a69995ed313d1fd29c5",
       "IPY_MODEL_f953646b0c1e47a0862e3779db5b5b85"
      ],
      "layout": "IPY_MODEL_e50ce9490e184933b831613b4af4a4c3"
     }
    },
    "3c920dea810342fea4978876495f51de": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "46dc3a56e58a435f9933388edb6b6f9b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "49dbf3024984495f9d30436d430b3473": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6241852f9a3145cab0a13f1ccff9b9ae",
      "placeholder": "​",
      "style": "IPY_MODEL_bef901dcc82c48308a8185cc4dbe06b4",
      "value": " 20/20 [00:05&lt;00:00,  3.59it/s]"
     }
    },
    "4ab591c4703e467e9eab4ea0401c530c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d7a3b1e3e45f408fa8b1f4adf9e1f0d3",
      "max": 500,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5dc6db51965d40898ff1b0cad8b9d766",
      "value": 500
     }
    },
    "5262032e2fef4868a81dbe9c7f64f816": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "53d0a46d2f6d4939919d5b1a573fb8f6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "561b1fcf29c84863b68bab77af299ff5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_927519a980f94ea486de7b50d2ccdebe",
      "max": 50,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_92b0189af44445b8b826225d71eade30",
      "value": 50
     }
    },
    "5d64005f60b041fa9d64177f20c438e4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b85ad7a4c72243da9af06bbfdf131123",
      "placeholder": "​",
      "style": "IPY_MODEL_785e14ef12754e48a3543e1547928a0c",
      "value": " 500/500 [02:18&lt;00:00,  2.69it/s]"
     }
    },
    "5dc6db51965d40898ff1b0cad8b9d766": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "6241852f9a3145cab0a13f1ccff9b9ae": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "684bfc28681f49139ef249e72610f3e8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "70a1e66a26da467780e230d2eaa2aa2a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "760e743cf93d406980027bd77f93e940": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8f639040c14d457e91d8c7276fda774a",
      "placeholder": "​",
      "style": "IPY_MODEL_9e3707cfab9d4c01a4803fc1172b88b7",
      "value": "100%"
     }
    },
    "785e14ef12754e48a3543e1547928a0c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "789983db1c6846df93c88638c6916997": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7b83678a32314681a7619898bc2b1c3d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7b95dbf1175d40fa8e91984f5836130f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7bd57e2af1a9428a9f51fb837921f200": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "81f614f73b91433a9aaa0073eb581c6a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "86269cdc86c3494189a27dada14d54d2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "887a44c27fe54f4fa984570d61852eb7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fe54f5602d034e5398d8284ea2116d3a",
       "IPY_MODEL_2fffe596e6e84e0a801b334114f1be30",
       "IPY_MODEL_49dbf3024984495f9d30436d430b3473"
      ],
      "layout": "IPY_MODEL_3abaf533ddd4421a885a19a5a87e80c4"
     }
    },
    "88889801c50c4c68af5fe6939f11dffa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8c8060f3481c4a69995ed313d1fd29c5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b43846c7cfd64b54ad04350403689c51",
      "max": 20,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5262032e2fef4868a81dbe9c7f64f816",
      "value": 20
     }
    },
    "8f639040c14d457e91d8c7276fda774a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "927519a980f94ea486de7b50d2ccdebe": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "92b0189af44445b8b826225d71eade30": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9332ee2ade77412e835d9337804ec095": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9461bce4a5e24cfb8fb5f8126c523a22": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "951a6d59d2d54544be75a24c69390e44": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "995264e33ccb4d22be486db2bf3e293a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9a5530b2b2bb4ae8997ce9692f77c128": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9bc9324578c8415a9b26c5ece91c6834": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_760e743cf93d406980027bd77f93e940",
       "IPY_MODEL_0ab235cfa9334ac38b5a8c3567542405",
       "IPY_MODEL_e1df9f66824947798ccdbd8c6344b4f5"
      ],
      "layout": "IPY_MODEL_e71f07638f2547a7bad3608002b9d9dd"
     }
    },
    "9e3707cfab9d4c01a4803fc1172b88b7": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a1e3513f993c40da89335f88511c34aa": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_a72c6b33f0b841e49563484b6c855bbb",
       "IPY_MODEL_0289ce01453a4e56a2b7aeff1758ea58",
       "IPY_MODEL_2a75db1b7139408e9d0738619ac0fc50"
      ],
      "layout": "IPY_MODEL_9461bce4a5e24cfb8fb5f8126c523a22"
     }
    },
    "a4e2549b24334c49bb96306d1b3cf389": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a72c6b33f0b841e49563484b6c855bbb": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ea3944dfec8a4e40be890fc14fb6b311",
      "placeholder": "​",
      "style": "IPY_MODEL_e2de4bb0e7134d45a6c93312f18875e1",
      "value": "100%"
     }
    },
    "a74e1a0ab5e14bb9824db2cba20e458e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a85ac50e6530477dac034dd095f262ff": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b43846c7cfd64b54ad04350403689c51": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b75a682da6b546c1a2e0f3949355ee38": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b85ad7a4c72243da9af06bbfdf131123": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bef901dcc82c48308a8185cc4dbe06b4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c713abe7231b450b99809ff53ae63804": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c7af51326f2345d98023418c91cb3ac8": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_06f53a896bd347f6adf30f75a95e0862",
       "IPY_MODEL_561b1fcf29c84863b68bab77af299ff5",
       "IPY_MODEL_024b913f921e42c59ac703113d1d343a"
      ],
      "layout": "IPY_MODEL_789983db1c6846df93c88638c6916997"
     }
    },
    "d7a3b1e3e45f408fa8b1f4adf9e1f0d3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db72ad1c1b324850b63b8d3e6a869de1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fe0bb33b887a4e9abb8ba7f725b3caa2",
       "IPY_MODEL_4ab591c4703e467e9eab4ea0401c530c",
       "IPY_MODEL_5d64005f60b041fa9d64177f20c438e4"
      ],
      "layout": "IPY_MODEL_995264e33ccb4d22be486db2bf3e293a"
     }
    },
    "e1df9f66824947798ccdbd8c6344b4f5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_70a1e66a26da467780e230d2eaa2aa2a",
      "placeholder": "​",
      "style": "IPY_MODEL_7b95dbf1175d40fa8e91984f5836130f",
      "value": " 500/500 [02:19&lt;00:00,  2.69it/s]"
     }
    },
    "e2de4bb0e7134d45a6c93312f18875e1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "e50ce9490e184933b831613b4af4a4c3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e71f07638f2547a7bad3608002b9d9dd": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ea3944dfec8a4e40be890fc14fb6b311": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ec685db8f2c14f12af7c449d1e7a12d5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f953646b0c1e47a0862e3779db5b5b85": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_53d0a46d2f6d4939919d5b1a573fb8f6",
      "placeholder": "​",
      "style": "IPY_MODEL_2dbbab648fd2402c9daf0b217222df40",
      "value": " 20/20 [00:05&lt;00:00,  3.94it/s]"
     }
    },
    "fe0bb33b887a4e9abb8ba7f725b3caa2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_951a6d59d2d54544be75a24c69390e44",
      "placeholder": "​",
      "style": "IPY_MODEL_684bfc28681f49139ef249e72610f3e8",
      "value": "100%"
     }
    },
    "fe54f5602d034e5398d8284ea2116d3a": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7b83678a32314681a7619898bc2b1c3d",
      "placeholder": "​",
      "style": "IPY_MODEL_a85ac50e6530477dac034dd095f262ff",
      "value": "100%"
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
