pytorch-lightning_ipynb/ordinal/CORAL-light_cement.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "aebb3445",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch            : 1.10.1\n",
      "pytorch_lightning: 1.6.0.dev0\n",
      "torchmetrics     : 0.6.2\n",
      "matplotlib       : 3.3.4\n",
      "coral_pytorch    : 1.2.0\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%load_ext watermark\n",
    "%watermark -p torch,pytorch_lightning,torchmetrics,matplotlib,coral_pytorch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "504da038",
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext pycodestyle_magic\n",
    "%flake8_on --ignore W291,W293,E703"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6282332",
   "metadata": {},
   "source": [
    "<a href=\"https://pytorch.org\"><img src=\"https://raw.githubusercontent.com/pytorch/pytorch/master/docs/source/_static/img/pytorch-logo-dark.svg\" width=\"90\"/></a> &nbsp; &nbsp;&nbsp;&nbsp;<a href=\"https://www.pytorchlightning.ai\"><img src=\"https://raw.githubusercontent.com/PyTorchLightning/pytorch-lightning/master/docs/source/_static/images/logo.svg\" width=\"150\"/></a>\n",
    "\n",
    "# CORAL MLP for ordinal regression and deep learning -- cement strength dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3675d863",
   "metadata": {},
   "source": [
    "This tutorial explains how to train a deep neural network (here: multilayer perceptron) with the CORAL layer and loss function for ordinal regression. \n",
    "\n",
    "**CORAL reference:**\n",
    "\n",
    "- Wenzhi Cao, Vahid Mirjalili, and Sebastian Raschka (2020) \n",
    "[Rank Consistent Ordinal Regression for Neural Networks with Application to Age Estimation](https://www.sciencedirect.com/science/article/pii/S016786552030413X) \n",
    "Pattern Recognition Letters. 140, 325-331"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17767cf8",
   "metadata": {},
   "source": [
    "**Note:**\n",
    "    \n",
    "To keep the notation lean and minimal, this notebook only contains \"Squared-error reformulation\"-specific comments. For more comments on the PyTorch Lightning use, please see the cross-entropy baseline notebook [baseline-light_cement.ipynb](./baseline-light_cement.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10219d24",
   "metadata": {},
   "source": [
    "## General settings and hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c039fb28",
   "metadata": {},
   "outputs": [],
   "source": [
    "BATCH_SIZE = 32\n",
    "NUM_EPOCHS = 200\n",
    "LEARNING_RATE = 0.01\n",
    "NUM_WORKERS = 0\n",
    "\n",
    "DATA_BASEPATH = \".\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec033e03",
   "metadata": {},
   "source": [
    "## Converting a regular classifier into a CORAL ordinal regression model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dddd83f",
   "metadata": {},
   "source": [
    "Changing a classifier to a CORAL model for ordinal regression is actually really simple and only requires a few changes:\n",
    "\n",
    "**1)**\n",
    "We replace the output layer  \n",
    "\n",
    "```python\n",
    "output_layer = torch.nn.Linear(hidden_units[-1], num_classes)\n",
    "```\n",
    "\n",
    "by a CORAL layer (available through `coral_pytorch`):\n",
    "\n",
    "```python\n",
    "output_layer = CoralLayer(size_in=hidden_units[-1], num_classes=num_classes)`\n",
    "```\n",
    "\n",
    "**2)**\n",
    "\n",
    "Convert the integer class labels into the extended binary label format using the `levels_from_labelbatch` provided via `coral_pytorch`:\n",
    "\n",
    "```python\n",
    "levels = levels_from_labelbatch(class_labels, \n",
    "                                num_classes=num_classes)\n",
    "```\n",
    "\n",
    "**3)** \n",
    "\n",
    "Swap the cross entropy loss from PyTorch,\n",
    "\n",
    "```python\n",
    "torch.nn.functional.cross_entropy(logits, true_labels)\n",
    "```\n",
    "\n",
    "with the CORAL loss (also provided via `coral_pytorch`):\n",
    "\n",
    "```python\n",
    "loss = coral_loss(logits, levels)\n",
    "```\n",
    "\n",
    "**4)**\n",
    "\n",
    "In a regular classifier, we usually obtain the predicted class labels as follows:\n",
    "\n",
    "```python\n",
    "predicted_labels = torch.argmax(logits, dim=1)\n",
    "```\n",
    "\n",
    "Replace this with the following code to convert the predicted probabilities into the predicted labels:\n",
    "\n",
    "```python\n",
    "probas = torch.sigmoid(logits)\n",
    "predicted_labels = proba_to_label(probas)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1af9bd66",
   "metadata": {},
   "source": [
    "## Implementing a `MultiLayerPerceptron` using PyTorch Lightning's `LightningModule`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "077b5287",
   "metadata": {},
   "source": [
    "\n",
    "- Given a multilayer perceptron classifier with cross-entropy loss, it is very easy to change this classifier into a ordinal regression model using CORAL as explained in the previous section. In the code example below, we use \"1) the `CoralLayer`\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9e19e4de",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from coral_pytorch.layers import CoralLayer\n",
    "\n",
    "\n",
    "# Regular PyTorch Module\n",
    "class MultiLayerPerceptron(torch.nn.Module):\n",
    "    def __init__(self, input_size, hidden_units, num_classes):\n",
    "        super().__init__()\n",
    "\n",
    "        # num_classes is used by the CORAL loss function\n",
    "        self.num_classes = num_classes\n",
    "        \n",
    "        # Initialize MLP layers\n",
    "        all_layers = []\n",
    "        for hidden_unit in hidden_units:\n",
    "            layer = torch.nn.Linear(input_size, hidden_unit)\n",
    "            all_layers.append(layer)\n",
    "            all_layers.append(torch.nn.ReLU())\n",
    "            input_size = hidden_unit\n",
    "\n",
    "        # CORAL: output layer -------------------------------------------\n",
    "        # Regular classifier would use the following output layer:\n",
    "        # output_layer = torch.nn.Linear(hidden_units[-1], num_classes)\n",
    "        \n",
    "        # We replace it by the CORAL layer:\n",
    "        output_layer = CoralLayer(size_in=hidden_units[-1],\n",
    "                                  num_classes=num_classes)\n",
    "        # ----------------------------------------------------------------\n",
    "        \n",
    "        all_layers.append(output_layer)\n",
    "        self.model = torch.nn.Sequential(*all_layers)\n",
    "        \n",
    "    def forward(self, x):\n",
    "        x = self.model(x)\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbabc3d0",
   "metadata": {},
   "source": [
    "- In our `LightningModule` we use loggers to track mean absolute errors for both the training and validation set during training; this allows us to select the best model based on validation set performance later.\n",
    "- Note that we make changes 2) (`levels_from_labelbatch`), 3) (`coral_loss`), and 4) (`proba_to_label`) to implement a CORAL model instead of a regular classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "bfde200c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from coral_pytorch.losses import coral_loss\n",
    "from coral_pytorch.dataset import levels_from_labelbatch\n",
    "from coral_pytorch.dataset import proba_to_label\n",
    "\n",
    "import pytorch_lightning as pl\n",
    "import torchmetrics\n",
    "\n",
    "\n",
    "class LightningMLP(pl.LightningModule):\n",
    "    def __init__(self, model, learning_rate):\n",
    "        super().__init__()\n",
    "\n",
    "        self.learning_rate = learning_rate\n",
    "        self.model = model\n",
    "\n",
    "        self.save_hyperparameters(ignore=['model'])\n",
    "\n",
    "        self.train_mae = torchmetrics.MeanAbsoluteError()\n",
    "        self.valid_mae = torchmetrics.MeanAbsoluteError()\n",
    "        self.test_mae = torchmetrics.MeanAbsoluteError()\n",
    "        \n",
    "    def forward(self, x):\n",
    "        return self.model(x)\n",
    "        \n",
    "    def _shared_step(self, batch):\n",
    "        features, true_labels = batch\n",
    "        \n",
    "        # Convert class labels for CORAL ------------------------\n",
    "        levels = levels_from_labelbatch(\n",
    "            true_labels, num_classes=self.model.num_classes)\n",
    "        # -------------------------------------------------------\n",
    "\n",
    "        logits = self(features)\n",
    "\n",
    "        # CORAL Loss --------------------------------------------\n",
    "        # A regular classifier uses:\n",
    "        # loss = torch.nn.functional.cross_entropy(logits, true_labels)\n",
    "        loss = coral_loss(logits, levels.type_as(logits))\n",
    "        # -------------------------------------------------------\n",
    "\n",
    "        # CORAL Prediction to label -----------------------------\n",
    "        # A regular classifier uses:\n",
    "        # predicted_labels = torch.argmax(logits, dim=1)\n",
    "        probas = torch.sigmoid(logits)\n",
    "        predicted_labels = proba_to_label(probas)\n",
    "        # -------------------------------------------------------\n",
    "        return loss, true_labels, predicted_labels\n",
    "\n",
    "    def training_step(self, batch, batch_idx):\n",
    "        loss, true_labels, predicted_labels = self._shared_step(batch)\n",
    "        self.log(\"train_loss\", loss)\n",
    "        self.train_mae(predicted_labels, true_labels)\n",
    "        self.log(\"train_mae\", self.train_mae, on_epoch=True, on_step=False)\n",
    "        return loss\n",
    "\n",
    "    def validation_step(self, batch, batch_idx):\n",
    "        loss, true_labels, predicted_labels = self._shared_step(batch)\n",
    "        self.log(\"valid_loss\", loss)\n",
    "        self.valid_mae(predicted_labels, true_labels)\n",
    "        self.log(\"valid_mae\", self.valid_mae,\n",
    "                 on_epoch=True, on_step=False, prog_bar=True)\n",
    "\n",
    "    def test_step(self, batch, batch_idx):\n",
    "        _, true_labels, predicted_labels = self._shared_step(batch)\n",
    "        self.test_mae(predicted_labels, true_labels)\n",
    "        self.log(\"test_mae\", self.test_mae, on_epoch=True, on_step=False)\n",
    "\n",
    "    def configure_optimizers(self):\n",
    "        optimizer = torch.optim.Adam(self.parameters(), lr=self.learning_rate)\n",
    "        return optimizer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71643d31",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "# Note: There Are No Changes Compared To The Baseline Below\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "147de9ed",
   "metadata": {},
   "source": [
    "## Setting up the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ad9a35b",
   "metadata": {},
   "source": [
    "### Inspecting the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "15574f7b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>response</th>\n",
       "      <th>V1</th>\n",
       "      <th>V2</th>\n",
       "      <th>V3</th>\n",
       "      <th>V4</th>\n",
       "      <th>V5</th>\n",
       "      <th>V6</th>\n",
       "      <th>V7</th>\n",
       "      <th>V8</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>4</td>\n",
       "      <td>540.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>162.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1040.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4</td>\n",
       "      <td>540.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>162.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>1055.0</td>\n",
       "      <td>676.0</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>332.5</td>\n",
       "      <td>142.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>228.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>932.0</td>\n",
       "      <td>594.0</td>\n",
       "      <td>270</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>332.5</td>\n",
       "      <td>142.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>228.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>932.0</td>\n",
       "      <td>594.0</td>\n",
       "      <td>365</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>198.6</td>\n",
       "      <td>132.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>192.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>978.4</td>\n",
       "      <td>825.5</td>\n",
       "      <td>360</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   response     V1     V2   V3     V4   V5      V6     V7   V8\n",
       "0         4  540.0    0.0  0.0  162.0  2.5  1040.0  676.0   28\n",
       "1         4  540.0    0.0  0.0  162.0  2.5  1055.0  676.0   28\n",
       "2         2  332.5  142.5  0.0  228.0  0.0   932.0  594.0  270\n",
       "3         2  332.5  142.5  0.0  228.0  0.0   932.0  594.0  365\n",
       "4         2  198.6  132.4  0.0  192.0  0.0   978.4  825.5  360"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "data_df = pd.read_csv(\"https://raw.githubusercontent.com/gagolews/\"\n",
    "                      \"ordinal_regression_data/master/cement_strength.csv\")\n",
    "data_df[\"response\"] = data_df[\"response\"]-1  # labels should start at 0\n",
    "\n",
    "data_labels = data_df[\"response\"]\n",
    "data_features = data_df.loc[:, [\n",
    "    \"V1\", \"V2\", \"V3\", \"V4\", \"V5\", \"V6\", \"V7\", \"V8\"]]\n",
    "\n",
    "data_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "121ac20c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of features: 8\n",
      "Number of examples: 998\n",
      "Labels: [0 1 2 3 4]\n",
      "Label distribution: [196 310 244 152  96]\n"
     ]
    }
   ],
   "source": [
    "print('Number of features:', data_features.shape[1])\n",
    "print('Number of examples:', data_features.shape[0])\n",
    "print('Labels:', np.unique(data_labels.values))\n",
    "print('Label distribution:', np.bincount(data_labels))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d7e50c5",
   "metadata": {},
   "source": [
    "### Performance baseline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f46c09a4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Baseline MAE: 1.03\n"
     ]
    }
   ],
   "source": [
    "avg_prediction = np.median(data_labels.values)  # median minimizes MAE\n",
    "baseline_mae = np.mean(np.abs(data_labels.values - avg_prediction))\n",
    "print(f'Baseline MAE: {baseline_mae:.2f}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c3b99ac",
   "metadata": {},
   "source": [
    "### Creating a `Dataset` class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0a1f7264",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import Dataset\n",
    "\n",
    "\n",
    "class MyDataset(Dataset):\n",
    "\n",
    "    def __init__(self, feature_array, label_array, dtype=np.float32):\n",
    "        self.features = feature_array.astype(dtype)\n",
    "        self.labels = label_array\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "        inputs = self.features[index]\n",
    "        label = self.labels[index]\n",
    "        return inputs, label\n",
    "\n",
    "    def __len__(self):\n",
    "        return self.features.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b092ca9",
   "metadata": {},
   "source": [
    "### Setting up a `DataModule`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b036894c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "\n",
    "class DataModule(pl.LightningDataModule):\n",
    "    def __init__(self, data_path='./'):\n",
    "        super().__init__()\n",
    "        self.data_path = data_path\n",
    "        \n",
    "    def prepare_data(self):\n",
    "        data_df = pd.read_csv(\n",
    "            'https://raw.githubusercontent.com/gagolews/'\n",
    "            'ordinal_regression_data/master/cement_strength.csv')\n",
    "        data_df.to_csv(\n",
    "            os.path.join(self.data_path, 'cement_strength.csv'), index=None)\n",
    "        return\n",
    "\n",
    "    def setup(self, stage=None):\n",
    "        data_df = pd.read_csv(\n",
    "            os.path.join(self.data_path, 'cement_strength.csv'))\n",
    "        data_df[\"response\"] = data_df[\"response\"]-1  # labels should start at 0\n",
    "        self.data_labels = data_df[\"response\"]\n",
    "        self.data_features = data_df.loc[:, [\n",
    "            \"V1\", \"V2\", \"V3\", \"V4\", \"V5\", \"V6\", \"V7\", \"V8\"]]\n",
    "        \n",
    "        # Split into\n",
    "        # 70% train, 10% validation, 20% testing\n",
    "        \n",
    "        X_temp, X_test, y_temp, y_test = train_test_split(\n",
    "            self.data_features.values,\n",
    "            self.data_labels.values,\n",
    "            test_size=0.2,\n",
    "            random_state=1,\n",
    "            stratify=self.data_labels.values)\n",
    "\n",
    "        X_train, X_valid, y_train, y_valid = train_test_split(\n",
    "            X_temp,\n",
    "            y_temp,\n",
    "            test_size=0.1,\n",
    "            random_state=1,\n",
    "            stratify=y_temp)\n",
    "        \n",
    "        # Standardize features\n",
    "        sc = StandardScaler()\n",
    "        X_train_std = sc.fit_transform(X_train)\n",
    "        X_valid_std = sc.transform(X_valid)\n",
    "        X_test_std = sc.transform(X_test)\n",
    "\n",
    "        self.train = MyDataset(X_train_std, y_train)\n",
    "        self.valid = MyDataset(X_valid_std, y_valid)\n",
    "        self.test = MyDataset(X_test_std, y_test)\n",
    "\n",
    "    def train_dataloader(self):\n",
    "        return DataLoader(self.train, batch_size=BATCH_SIZE,\n",
    "                          num_workers=NUM_WORKERS,\n",
    "                          drop_last=True)\n",
    "\n",
    "    def val_dataloader(self):\n",
    "        return DataLoader(self.valid, batch_size=BATCH_SIZE,\n",
    "                          num_workers=NUM_WORKERS)\n",
    "\n",
    "    def test_dataloader(self):\n",
    "        return DataLoader(self.test, batch_size=BATCH_SIZE,\n",
    "                          num_workers=NUM_WORKERS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "cdb18ceb",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.manual_seed(1) \n",
    "data_module = DataModule(data_path=DATA_BASEPATH)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73e75df2",
   "metadata": {},
   "source": [
    "## Training the model using the PyTorch Lightning Trainer class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "bfc57a43",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pytorch_lightning.callbacks import ModelCheckpoint\n",
    "from pytorch_lightning.loggers import CSVLogger\n",
    "\n",
    "\n",
    "pytorch_model = MultiLayerPerceptron(\n",
    "    input_size=data_features.shape[1],\n",
    "    hidden_units=(24, 16),\n",
    "    num_classes=np.bincount(data_labels).shape[0])\n",
    "\n",
    "lightning_model = LightningMLP(\n",
    "    model=pytorch_model,\n",
    "    learning_rate=LEARNING_RATE)\n",
    "\n",
    "\n",
    "callbacks = [ModelCheckpoint(\n",
    "    save_top_k=1, mode=\"min\", monitor=\"valid_mae\")]  # save top 1 model \n",
    "logger = CSVLogger(save_dir=\"logs/\", name=\"mlp-coral-cement\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "caf327e0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py:90: LightningDeprecationWarning: Setting `Trainer(progress_bar_refresh_rate=50)` is deprecated in v1.5 and will be removed in v1.7. Please pass `pytorch_lightning.callbacks.progress.TQDMProgressBar` with `refresh_rate` directly to the Trainer's `callbacks` argument instead. Or, to disable the progress bar pass `enable_progress_bar = False` to the Trainer.\n",
      "  rank_zero_deprecation(\n",
      "GPU available: True, used: True\n",
      "TPU available: False, using: 0 TPU cores\n",
      "IPU available: False, using: 0 IPUs\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "\n",
      "  | Name      | Type                 | Params\n",
      "---------------------------------------------------\n",
      "0 | model     | MultiLayerPerceptron | 636   \n",
      "1 | train_mae | MeanAbsoluteError    | 0     \n",
      "2 | valid_mae | MeanAbsoluteError    | 0     \n",
      "3 | test_mae  | MeanAbsoluteError    | 0     \n",
      "---------------------------------------------------\n",
      "636       Trainable params\n",
      "0         Non-trainable params\n",
      "636       Total params\n",
      "0.003     Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validation sanity check: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:258: PossibleUserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n",
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:258: PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6f2d5ded21124662b9d4ed2d4bb0cd1d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Training: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Validating: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training took 0.60 min in total.\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "\n",
    "trainer = pl.Trainer(\n",
    "    max_epochs=NUM_EPOCHS,\n",
    "    callbacks=callbacks,\n",
    "    progress_bar_refresh_rate=50,  # recommended for notebooks\n",
    "    accelerator=\"auto\",  # Uses GPUs or TPUs if available\n",
    "    devices=\"auto\",  # Uses all available GPUs/TPUs if applicable\n",
    "    logger=logger,\n",
    "    deterministic=True,\n",
    "    log_every_n_steps=10)\n",
    "\n",
    "start_time = time.time()\n",
    "trainer.fit(model=lightning_model, datamodule=data_module)\n",
    "\n",
    "runtime = (time.time() - start_time)/60\n",
    "print(f\"Training took {runtime:.2f} min in total.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b031c637",
   "metadata": {},
   "source": [
    "## Evaluating the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "efb086f5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:xlabel='Epoch', ylabel='MAE'>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABWgklEQVR4nO2dd3hc1bW33z3qvXdZltw77ja2wTbVgOmEEkiAQLhAuMDlQiCk50tuSCMJgdASagKG0HszNraxce+23GTZ6r13zezvj32mSBoZ23g0glnv88wzo1PmrDkz2r+91l57baW1RhAEQQhcbP42QBAEQfAvIgSCIAgBjgiBIAhCgCNCIAiCEOCIEAiCIAQ4wf424FhJTk7Wubm5x3VuS0sLUVFRJ9agE8RgtU3sOjYGq10weG0Tu46N47Vr48aN1VrrFK87tdZfq8e0adP08bJs2bLjPtfXDFbbxK5jY7DapfXgtU3sOjaO1y5gg+6nXZXQkCAIQoAjQiAIghDgiBAIgiAEOF+7wWJBEL55dHV1UVxcTHt7u79NcREXF8fu3bv9bUYfvsyu8PBwsrOzCQkJOer3FCEQBMHvFBcXExMTQ25uLkopf5sDQFNTEzExMf42ow9HsktrTU1NDcXFxeTl5R31e0poSBAEv9Pe3k5SUtKgEYGvK0opkpKSjtmzEiEQBGFQICJwYjie+xgwQrCnvIlX93VS09zhb1MEQRAGFQEjBAeqmnn7QBfVzZ3+NkUQBGFQETBCEBZsPmpHt93PlgiCMNior6/n73//+zGfd+6551JfX3/M51133XW88sorx3yerwgYIQi1hKCz2+FnSwRBGGz0JwR2+5E7ju+99x7x8fE+smrgCJj00dAgEQJB+Drwy7d3squ08YS+57jMWH5+/vh+9993330cOHCAyZMnExISQnR0NMnJyezcuZNdu3Zx0UUXUVRURHt7O3fccQc33XQTALm5uWzYsIHm5mbOOecc5s2bx+rVq8nKyuLNN98kIiLiS21bunQpd999N93d3cyYMYNHH32UsLAw7rvvPt566y2Cg4M566yz+OMf/8h//vMffv7znxMSEkJcXBwrVqw4IfcnYIQgLCQIgA4RAkEQevHAAw+wY8cOtmzZwvLlyznvvPP44osvmDhxIgBPPfUUiYmJtLW1MWPGDC699FKSkpJ6vMe+fft48cUXefLJJ7n88st59dVXueaaa4543fb2dq677jqWLl3KqFGj+O53v8ujjz7Kd7/7XV5//XXy8/NRSrnCT7/61a94/fXXGT169HGFpPojYITA6RGIEAjC4OZIPfeBYubMmXiWu3/ooYd4/fXXASgqKmLfvn19hCAvL4/JkycDMG3aNAoLC7/0Onv27CEvL49Ro0YBcO211/LII49w2223ER4ezo033sh5553H4sWLAZg7dy633HILV111FZdccslX/6AWgTdGYBchEAThyHjW+1++fDmffPIJa9asYevWrUyZMsXrhK2wsDDX66CgILq7u7/0OqY6dF+Cg4NZt24dl156KW+88QaLFi0C4LHHHuMnP/kJRUVFTJ48mZqammP9aN6vd0Le5WuAK2uoS7KGBEHoSUxMDE1NTV73NTQ0kJCQQGRkJPn5+XzxxRcn7LpjxoyhsLCQ/fv3M2LECJ5//nnmz59Pc3Mzra2tnHvuucyePZsRI0YAcODAAWbMmMFpp53G22+/TVFRUR/P5HgIOCEQj0AQhN4kJSUxd+5cJkyYQEREBGlpaa59ixYt4rHHHmPSpEmMHj2a2bNnn7DrhoeH8/TTT/Otb33LNVh88803U1tby4UXXkh7eztaa/785z8DcM8997Bnzx6UUpx++umcdNJJJ8SOgBECSR8VBOFIvPDCCz3+dnoIYWFhvP/++17PcY4DJCcns2PHDtf2u++++4jXeuaZZ1yvTz/9dDZv3txjf0ZGBuvWretz3muvveaTYngBM0YQFixZQ4IgCN4Qj0AQBMFH/OAHP+Dzzz/vse2OO+7g+uuv95NF3gkYIQiyKYKUlJgQBGHgeOSRR/xtwlERMKEhgGCbeASCIAi9ESEQBEEIcAJKCEJsSgaLBUEQehFgQiAegSAIQm8CSgiCbdAhE8oEQfiKREdHA1BaWspll13m9ZgFCxawYcOGft8jNzeX6upqn9h3rASUEITYFB1dIgSCIJwYMjMzB9UCM8dLwKSPgjVYLB6BIAxu3r8Pyref2PdMnwjnPNDv7nvvvZehQ4dy6623AvCLX/yCzs5O1q5dS11dHV1dXfz617/mwgsv7HFeYWEhixcvZseOHbS1tXH99deza9cuxo4dS1tb21Gb9+CDD/LUU08BcOONN3LnnXfS0tLC5ZdfTnFxMXa7nZ/+9KdcccUV/PznP+eDDz7osU7BVyWghMCMEcg8AkEQenLllVdy5513uoTg5Zdf5pVXXuG+++4jNjaW6upqZs+ezQUXXIBSyut7PProo0RGRrJt2za2bdvG1KlTj+raGzdu5Omnn2bt2rVorZk1axbz58+noKCAzMxM3n33XcAUv6utreXtt99m7969PdYp+KoEmBBI1pAgDHqO0HP3FVOmTKGyspLS0lKqqqpISEggPT2d+++/nxUrVmCz2SgpKaGiooL09HSv77FixQpuv/12ACZNmsSkSZOO6tqrVq3i4osvdpW+vuSSS1i5ciWLFi3i7rvv5t5772Xx4sWccsopdHd3e12n4KsSUGMEMo9AEIT+uOyyy3jllVd46aWXuPLKK3n55Zepqqpi48aNbNmyhbS0NK/rEHjSn7dwJPpbk2DUqFFs3LiRiRMn8qMf/Yhf/epXBAcHs2zZsj7rFHxVRAgEQRAw4aElS5bwyiuvcNlll9HQ0EBqaiohISEsW7aMQ4cOHfH8U089lX//+98A7Nixg23bth3VdU899VTeeOMNWltbaWlp4fXXX+eUU06htLSUyMhIrrnmGu6++242bdpEc3MzjY2NnHvuufzlL39hy5YtX/VjA4EWGgqCjnYRAkEQ+jJ+/HiamprIysoiIyODK664gquuuorp06czefJkxowZc8Tzb7nlFq6//nomTZrE5MmTmTlz5lFdd+rUqVx33XWu42+88UamTJnChx9+yD333IPNZiMkJIRHH32UpqYmvvWtb9HV1dVjnYKvSmAJgU2JRyAIQr9s3+7OVkpKSmLNmjVej2tubgbMXADnOgQREREsWbLkqK/luabxXXfdxV133dVj/9lnn83ZZ5/d57zly5fLegRfBUkfFQRB6EuAeQSyZrEgCAPLrFmz6Ojo6LHt+eefZ+LEiX6yqC8BJgSKTrsIgSAMRrTWx5V1M9hZu3btgF6vvyykI+Gz0JBSaohSaplSardSaqdS6g4vxyil1ENKqf1KqW1KqaObgXGcBNugy65xOI79RgmC4DvCw8Opqak5rkZMcKO1pqamhvDw8GM6z5ceQTfwv1rrTUqpGGCjUupjrfUuj2POAUZaj1nAo9azTwixZK/T7iDcFuSrywiCcIxkZ2dTXFxMVVWVv01x0d7efswN6kDwZXaFh4eTnZ19TO/pMyHQWpcBZdbrJqXUbiAL8BSCC4HntOkGfKGUildKZVjnnnCCbcbt7Oh2EB4iQiAIg4WQkBDy8vL8bUYPli9fzpQpU/xtRh98YZcaCFdMKZULrAAmaK0bPba/AzygtV5l/b0UuFdrvaHX+TcBNwGkpaVNO5YULU/e39fMSwcUf10YSVzY4IpFNjc3u0rbDibErmNjsNoFg9c2sevYOF67Fi5cuFFrPd3rTq21Tx9ANLARuMTLvneBeR5/LwWmHen9pk2bpo+L6v36g7/drife+5Iuqm05vvfwIcuWLfO3CV4Ru46NwWqX1oPXNrHr2Dheu4ANup921afzCJRSIcCrwL+11q95OaQYGOLxdzZQ6hNjKnZwdvUzZKlqmVQmCILggS+zhhTwT2C31vrBfg57C/iulT00G2jQPhofICIRgHjVLJPKBEEQPPBl1tBc4DvAdqXUFmvb/UAOgNb6MeA94FxgP9AKXO8zayISAIijRVYpEwRB8MCXWUOrgCOOyFpxqx/4yoYeWEIgHoEgCEJPAqfWUKQJDSXQLB6BIAiCB4EjBCERdKtQ4lSzlJkQBEHwIHCEAOgMjiaeZskaEgRB8CDghCBBNcu6xYIgCB4ElBB0BccQL0IgCILQg8ASgpBo4miR0JAgCIIHASUE3eIRCIIg9CGghMAeGkMCzXTKKmWCIAguAksIQmIIU104Otv8bYogCMKgIeCEAEC11/rZEkEQhMFDQAlBV4ip4R3cUe9fQwRBEAYRASUE3cHGIwjprPevIYIgCIOIgBKCLis0FNzR4GdLBEEQBg8BJQTdwSY0FNolQiAIguAkoITA6RGEiRAIgiC4CCghcASF0UGoCIEgCIIHASUEAC22GAkNCYIgeBBwQtAaHCcegSAIggcBJwSdIbFE2hv9bYYgCMKgIeCEoCs0jihHk7/NEARBGDQEnBDYw+KJo5l2KTwnCIIABKAQ6IhEEmimobXT36YIgiAMCgJOCGyRiYSpLhqbJDwkCIIAASgEwdGJADTXV/nZEkEQhMFBwAlBaHQyAO2N1X62RBAEYXAQcEIQEWeEoEOEQBAEAQhgIehuqfGzJYIgCIODgBOCqLgUAHRrnZ8tEQRBGBwEnBDYosxgMa2yXKUgCAIEoBAQEkEHodhkuUpBEAQgEIUAaLbFyLrFgiAIFgEpBC1BsYR2SeE5QRAECFAh6AiOJbJbSlELgiBAgApBZ2gckVKBVBAEAQhQIegOSyBGN6G19rcpgiAIficghYDwBBJoprm9y9+WCIIg+J2AFAIVmWBVIJUBY0EQhIAUgiBrUlldTaWfLREEQfA/PhMCpdRTSqlKpdSOfvYvUEo1KKW2WI+f+cqW3qRnZALwyHvraWiT8JAgCIGNLz2CZ4BFX3LMSq31ZOvxKx/a0oOkFCMEbXVl/PKtnQN1WUEQhEGJz4RAa70CGJwFfWLSATg13U5+uaSRCoIQ2ChfplAqpXKBd7TWE7zsWwC8ChQDpcDdWmuv3XOl1E3ATQBpaWnTlixZclz2NDc3Ex0djc3ewakrL+fliCt5oO1CHlwQeVzvdyJx2jbYELuOjcFqFwxe28SuY+N47Vq4cOFGrfV0rzu11j57ALnAjn72xQLR1utzgX1H857Tpk3Tx8uyZcvcf/w2R69/+Do99qfvH/f7nUh62DaIELuOjcFql9aD1zax69g4XruADbqfdtVvWUNa60atdbP1+j0gRCmVPGAGxGaS4KiltdNOR7d9wC4rCIIw2PCbECil0pVSyno907Jl4JYNi0knrtssV9nQKplDgiAELsG+emOl1IvAAiBZKVUM/BwIAdBaPwZcBtyilOoG2oArLfdlYIjJJLrEZLbWtXaRGhs+YJcWBEEYTPhMCLTWV33J/oeBh311/S8lNoOwjmpsOKhv7fSbGYIgCP4mIGcWAxCTjtIOkmmgTkJDgiAEMAEsBGZSWbqqpaFNPAJBEAKXABYCM6ksTdWJRyAIQkATuEIQazyCrKB66kUIBEEIYAJXCKJSQAWRE1Ivg8WCIAQ0gSsEtiCITiMrqEE8AkEQApqjEgKlVJRSyma9HqWUukApFeJb0waA2AwyVC114hEIghDAHK1HsAIIV0plAUuB6zFlpr/eJA4jy14iaxIIghDQHK0QKK11K3AJ8Det9cXAON+ZNUCkjiXJXklnS72/LREEQfAbRy0ESqmTgauBd61tPpuVPGCkGi1LaTvoZ0MEQRD8x9EKwZ3Aj4DXtdY7lVLDgGU+s2qgSB0LQJ4+TFunVCAVBCEwOapevdb6M+AzAGvQuFprfbsvDRsQ4nLoCopgdHcR9W2dRIRG+NsiQRCEAedos4ZeUErFKqWigF3AHqXUPb41bQCw2WiJHclIVSwppIIgBCxHGxoap7VuBC4C3gNygO/4yqiBpCNxNKNtxZJCKghCwHK0QhBizRu4CHhTa90FDNzaAT5EpY4lRTXQVFPub1MEQRD8wtEKweNAIRAFrFBKDQUafWXUQBKRNQEAe/kuP1siCILgH45KCLTWD2mts7TW51rrIB8CFvrYtgEhOncqAOHV2/xsiSAIgn842sHiOKXUg0qpDdbjTxjv4GuPik6hjBQS6nf42xRBEAS/cLShoaeAJuBy69EIPO0rowaagtDRZLXs9rcZgiAIfuFohWC41vrnWusC6/FLYJgvDRtIyqLGkmovh5Yaf5siCIIw4BytELQppeY5/1BKzQXafGPSwFOfONG8KN3sX0MEQRD8wNEKwc3AI0qpQqVUIfAw8F8+s2qA6UyZiEMrWgvX863HVrO3osnfJgmCIAwYR5s1tFVrfRIwCZiktZ4CnOZTywaQuIQkDuhM6vauYX1hHZ/vr/a3SYIgCAPGMa1QprVutGYYA9zlA3v8QmpMOBsdI0mo2UAI3ZTUfWOiXoIgCF/KV1mqUp0wK/xMakwYSx1TiXS0MMOWT7EIgSAIAcRXEYJvRIkJgJSYMFY5JtCuQzjTtpHi+lZ/myQIgjBgHLEMtVKqCe8NvgK+MTWbk6PDaCOcVY4JnBG0iYdqRQgEQQgcjigEWuuYgTLEn4QG20iIDOGTjmmcEbSZtPYCWjq6iQr7+i/CJgiC8GV8ldDQN4rUmHCW2qcAsMC2hZJ6GScQBCEwECGwSI0Ng5h0WhPGcIptO8V1Eh4SBCEwECGwuOnUYfzkvLHoYacx3baHPYcruPCRz1l3sNbfpgmCIPgUEQKLU0amcOHkLCLGnEmY6mb76vfYWlTPsj2V/jZNEATBp4gQ9MKWO4cOQpnebeoO7ato9rNFgiAIvkWEoDch4ewJm8R821ZSo0PZVyl1hwRB+GYjQuCF6iFnMdxWxu3j2zhc20pbp93fJgmCIPgMEQIvnHbpTRAUypyWpWgNB6okPCQIwjcXEQJvRCTAyLPIKX0fGw4JDwmC8I3GZ0KglHpKKVWplPK6GLAyPKSU2q+U2qaUmuorW46LSZcT3FrB/OAd7JUBY0EQvsH40iN4Blh0hP3nACOtx03Aoz605dgZeTZEp3N32BvsK2/88uMFQRC+pvhMCLTWK4Ajzca6EHhOG74A4pVSGb6y55gJCYcF9zLenk9K6TJ/WyMIguAzlNa+qyatlMoF3tFaT/Cy7x3gAa31KuvvpcC9WusNXo69CeM1kJaWNm3JkiXHZU9zczPR0dFHb7+jm7Gf30ZdVxBF8x8iOCjouK7rC9sGCrHr2BisdsHgtU3sOjaO166FCxdu1FpP97pTa+2zB5AL7Ohn37vAPI+/lwLTvuw9p02bpo+XZcuWHfM5a15/VOufx+rKta/o0vpW/f720uO+/pE4HtsGArHr2Bisdmk9eG0Tu46N47UL2KD7aVf9mTVUDAzx+DsbKPWTLf3iGHcRhxyphK/9K09+VsDN/9pEY3uXv80SBEE4YfhTCN4CvmtlD80GGrTWZX60xytDkmN5wr6YmJqthBWtAGBPuaSTCoLwzcGX6aMvAmuA0UqpYqXUDUqpm5VSN1uHvAcUAPuBJ4FbfWXLVyEjLpzX9HwawjK4pvrPRNFGflkjLR3drC+UyqSCIHz98dkSXFrrq75kvwZ+4KvrnyiCg2wkxsbyeOJ9/G/Jnfwi+Fk2l4/msc8O8PCy/ay573TS48L9baYgCMJxIzOLj4IhiRE8X5LB4/bz+VbwClqLtrJ0dyVaw9qDNf42TxAE4SshQnAUDEmIpKmjm8e6F9OuwllY/SK7yswks7WycI0gCF9zRAiOguyESACaVTQFOZdxrlpDBjXkJEbKCmaCIHztESE4CoYkRljPkaiTzZj2HyKf5erpGeyvbKa6ucOf5gmCIHwlRAiOAqdHMDwlmrzhY/i141rmOTZwWcnvUDhYL16BIAhfY0QIjgKnRzA8JYrwkCAuuPGntM37EUkHXuNXof9ig5VG2mV3YHf4rmSHIAiCLxAhOArSY8O5fm4uF03JAmDa0EQiTr8XTr6N79g+YNKBxwG4/PE1/PrdXf40VRAE4Zjx2TyCbxJKKX5+/vjeG+GsX/P59n1cWP8sjrUT2V6cRVN7t3+MFARBOE7EI/gqKMXSkT9mmZ6O7f27+Z56iwNVTTRJLSJBEL5GiEfwFclMjOHmjttYNuIl7i9+kZzuSrYXTWfVgXqiwoL5wcIR/jZREAThiIgQfEWyEyLoIJRHk35EZmEwtwS/zf63v8uKmovY6RjKzLxEZuQm+ttMQRCEfpHQ0FfEmVr6eUEdD+pv87vgW8hu2MQ7IT9iSfgDPPjyx7R32f1spSAIXxs6W8GHC4Z5Q4TgK5KdYFJLC6payE6IpGjY5czqeISno29kWtABnmy5nQNr3vCvkYIgfD1ob4A/joJdbw7oZUUIviJxESFEh5kIW05iJCdlx9NANOGn3kHF1cs4rNMYs+wm9Ke/YfdH/8TR2eZniwVBGLTUHoTOJijfNqCXlTGCr4hSiuyECPLLmxiaFMnikzIoqG7hgpMyCQmycU7XT3g3/jGGrPg9Y4H63c8Qv/iX0FINQ2ZBwlB/f4Sjo3ofBIV+fewVhK8jDcXmuXFgF2sUITgBOIUgJzGSjLgIfnvJRNe+yJgk/pL1ICfPDufTd/7Nnxv+Ac9f7D45Zw5MupyYxk5onQSRvQaWHXZoq4eopC83RGszv+FEY++CZy+AoGC49QsIjep7TFcbhESc+GsLQiDhFALn8wAhQnACcA4YD03q20BmJURQ0tDGvtgw3nPMpiNuCo+eHcXjm1oY17SG2XWfEPXOnUwD2Hwf5J4CHU3QWg0Zk6FkIzSWmNdTvwuTr4buNijbBm21MOZ8KN8Kr9wADUWQNx+uehG2vQRVe2DcRbD1BSjdAnFZMPIsSB4F+z6GzMkw4kyo3AmJw8AWAit+b2wYeab7Q+x+C5qsHsryB+Cs/9fzQ1bmw+OnwiWPw/iLEQThOGl0egQlA3pZEYITQFa86QnnJEZ63be5qI7Y8BAAPqsI5+X6Mfxp2w5Cg08jIeIs1t6azfYV7zAxoQ3y34XIZMicAiWbIGUMTL0W9rwL794F790D2iMLafzFULzBeANTvgMbn4Z/ngVlW8z+NQ+bBj5nthGP3W/3NFDZQDsgNNp4I/WHYd0/4IaPoGIHSdWH4cBSSMiF3Hmw5hEYcToMW2CyG0IjYfVDYO+A5b+DsRcar8TpmZRuMSITHmu8m83/Mu+XPMoITmQSLP2lOebMX5nPXFsAEfEw8mzjhfSmfDtJ1euABcf/pQnCYMQzNNTbw2+thYgEn1xWhOAEsPikDBrauhiRGt1nX1ZCBO9tLyMsOIio0CBaOu386aM95CZFsnhSJn9fvh978hhqkitgwQI4/WfeLzL/h1C4CvZ9CFGpkDoWSjfDst+Y2P33PoSsqaZhXflHGHs+nP1b2P+JabQT88wPq2it+bENPw0KlpmGOn0S7H3fvL70n/DBffDYPEDjCnKd/X/GGynZBC9eZURh38cw8ybY9jIkjYCq3fDBvbDjVXNswlB493+NiMy8CTY+C9V7ICbDCNKKP5j3Do2BfR/B1iXQXu/+zKMWwWVPmVDUnvdNiGrcBfDmbYyr2AXt/wXhcSfmSxT8wxePElfvQETdwikE3e2m4XeGhLWGv0yCaddC2Jn9n3+ciBCcADLiIrj77NFe92XFR9Dt0OyvbOayadm8srGYutYubpiXR1xECA4NNUeznoFSkHeKeTgZeaZpVCOTjAgALPyx6bFnz4CgEJh+fc/3yJnt/nvCpeYBMOlb7u0xGbDyTzDz++ze/AVjwyqMtxEeC999E545zwjKsAWw7nFQQfDtl+G5i2DdExA3xHgJYHr9tQfhw/shbQJc/hyMvQC6Wo1I1RwwHs+hz+GLR2HiZUak9n4IH/4IHpkFGSdB/jvGs7nkCSjbQhAYMZlyTd97VbHTiI+3sQzB/yy5Gk66EsYshk9+QUbSbL4Gy5f7jo5m+OQXMPd2aCiBsDjoaDBhIqcQNJaabKLEPGg58SaIEPiYrAT3AOqUnHg2Ha7jYHULl0zNZltxAwCVTR18WNjF7/66kvdun4ey3MGD1S2kxoQRFXaEr2nqd3r+bbPB0DlfzejcueYBVJRFMHbBAve+6FT4rxXmdXA4rP+HCfkkDYeL/g6lm2D2rSYEVLoZzvm9CRvVFRrPw+nqhkbBuAvd7zvuAvNwMvtmSB0DKx+EvR/A9O/BlhfhtZsgJIq2oCgiti7pKwRr/m4EZMKlxpsQzJjTS9cQmXQpdHeYhnjhjyBr2om9Tks1/H02XPGvnh0OAHs32IKMLfnvQFSy6SR0txPaWX9i7RjsaA2dzdDdaRr6T38N65804dCmMtOR2/+JEYWMk8w5VfnmOXk0tJz4CaoiBD4mO94tBHlJUVwzaygl9W1kxkdQ3tgOQFVTB/vq7OyuaORAVQsjUqMpqm3l7D+v4NaFw7nzjFH+Mt87ntlBM7/vfu3psUy/HrC8kZBw9w/6WBi2wDwcDiNwkUkmnDTlasprOsgrXAL1RRA/xBy/7kkjAjEZJjw17y5In3AcH3CQ09kCB5bB2MVHd3zZNihYTqIaDtVTYP/HRgS8CYHW5mHrNcWovQHW/xPm3mEadG9U7oKWKihe31MIujvgz+PhjF+6PdfGMmiuACC0s+7oPsc3hddvhm1LzOthC6DgM/N660uAhiGzjRB4DhhX7THPKWPg0M4TbpJMKPMxnh5BXkoU35uXx08XjwMgNSYMgMqmdqrbzJTyNQU1APx16T467Q4O17YOsMWDEGejNPcOE0aa9z9UpC0wDdJj88wgdVM5LP2VCSvd/Llxr5f9X8/32foSHFo94OafcLa+CC9dbcJq/WHvhrWPm7Te2gIAwtvLjWcGrm192P4f+MNwE64A02sF2PmGGdQv3mA8wB2vGYEGaDG/WeqLzHNDiTlvydVm3Kmh2C0Qzhh4U5n5zvAQgkNrzHyVbyI1B6DuEBxea0Rg4uVw6j1mzC0mA6bfAA2HzbGZU0wYtIcQ5ENEovGkfIAIgY+JDA0mITKE8BAbaTHhPfalOIWgsYPqNvNP9UVBDfsrm3htU7FrX39UNrX7yOpBSlgMXPAQJOTSHpEO3/vIDFov/z/4+8lm3GHR74y7Pee/TaZVyUZzbsFyeP0meOEKd2M42Gksg/z3+m53CkD13r772k24kUOfw/s/NOMoVqMf0VZuxmugfyE48KlJS67YYa79u1wT8qnZb/bXHzK91Veuh8KVpqf6xxFGYOuthqyx2NiW/44J69UfMtvrCk2KMxghcHoEXY0mEeDVG+HTXqnJg4Htr8CHPz7281qqjffWVm8y+R6dA2/fAVEpcP5f4LSfwJ3b4eZVPcOi8UMgNsMIqpPqvcYb8MU8IUQIBoTshEhyk6Kw2Xp+iWHBQcRHhnCwpoXmLvMdry2o4Zdv7yIyNJgZuQmuxn57cQNVTW5ReGtrKbP/bylFgewxZE+DK/8N5/3JNICzboYUK4w2+2bTg/r0N+Yf8c3bICEPUKbBsX8NFhBa8zAs+Ta0Ww2lUwCcQta791y8wTTcFTuh1jq2YifUmcY/oq3c9bpfISjbaj1vMyGkrhbTq3cKQd0htwDV7Ify7Sb9uHiDWwgaSnp6Hs7tdQfdHkFLlVsUwLxuLHYfeyJY+ivjZXxV1j1hvgvLgzkqDq6Eh6bAEwvggx9Baw3EZpnMunl3uRMZIuJNx2XIbDPmBua42CzjEZRsNF5d5W5I8Z6QciIQIRgA7jpzFPf0k1WUGhPG5sP1AMwbkUx1cycr91Xzw0WjGZMeS2VTB1prvvPUWn77/m7Xec+tLsShobDGBykEXzdm3Ah37YYzPXqTYTEw73/MnIWHp5usi0uegHP/YEIUe9/3n71HS81+QJuGd/0/zEBsa627kXU2zk4KV7ob5RoPIXCFhirc29tqjUCCSc1dcrUJBzkHJcu3mvcBM+HQKTr1he73qC3w8E72eHgEJR6Cc9C9vb6opzdW5lFPp9jy3Jy94E3PmZ748dJQbDLfPnvA+/72Bvj8ISOw3s510tVukh7A3KcvY++H8J/r4F+XmMSKxlIzoXPqd+H7n5r0bM9xNSch4abkTHg8hEUbITj0OTx5Grz0HZNWnTLmy69/nIgQDAALx6Ry+tg0r/tSY8I5WG0a88umZQMwNSeea2YNJTUmjPrWLsob26lv7WJtQS0Aeyua2HDIxFXLGwIsPNQfMWl9Bzhn3AjxORCdbibIDZlpsomi02HT88f2/p2tAz7t39XIVuVD0Tqwd5piZP0JQekW6/g97h5/5S7TGIdEYdPdpnEPsXqjdQfNwPAnvzRhnHVPGCEJCjOx7AprULJsm7thrzvk9jbqCt3XqdrjjnE3V7q9Bk+PQNvh8BeAc7LhZrftxevMc0ulGVxe8QcT2uo+itRqbzjHgg6ugKaKvvs3PgMf/xT2LzVjHQdXGi9x/ydmYNsZkivdbO47yghBfZE5Bsx5XR7/f82VxoM7tBomfgtu+Bi+87r5zZ3+M9PAT7zMpHV744xfwOI/m9e584wHO/x045mBeATfZJwDxgBzhifz+8sm8bdvT8VmU6TGmn0brUa/pL6NotpWlqwrIiTI/DNVNHoXgo2Halm+p9LH1g9yQiPhtg1w80rInm62BQXDlKvNP1fvwl5dbbDijyZltTfv3wOPneLuQdYfhn+ebbKTenOsteSbKtyNrhN7t7vxrdztDtnsX2rGQmzBfUNDztnkVfluEWksgY5GdzZXV4v7dW2BGTupsjzNVX8xz2PPh5p9puEOiTT3ytFtJi7WHYKaAvf5PYSgBKLTAO0OybRWm8/mDHs0lrh7tvWHzJwTMELnpPagaXBba8xM+85Wk3bqjbY6+PscM7ahNSz7rble4Spjr3Z4L+m890PzvO9DU47l2cWw7Ncmnx9gy78tu74wz5MuN/fqn2fBvy41mT9PzIffZsP79xo7tvzb3Kdr3zGp1JGJpvNx2VNHN8ibNRUmXGJeT78e7tgClz9rOi4gHsE3mRSrsQ+xQXJ0KJdPH+IqWZEaa/55NhS60+s+za/klY1FnD0+nfjIEMob27E7NO9vL8PhcDdA97+2g1++vcvrNbXWPP7ZAQqrAyCsFBzWd4BtyjWmgVj1F9OYPnsB/H646Ql++v9MdszGZ93Ht9bCtv+YcErpFtMYPn2uaSTW/9McU7gKaguw2dtNJtMTC92N95HoaIanF5kGxrOxqz9kGhUwvXhnL3z3W+Z5yGzTe3YODrfVuT2Fyl3mdcpY9/sNP839ethC81xbYCbxRaWayV0dDaa8yehz3MeOv9h9jaFzTBy/scRMIqw9aOwKCjVio+2Qc7I5tmafERGnPUNmud9zyAz36/RJ5rl8u3tb4UrA+i2vehD+Ns1MYtTahMg+/LFbbLe/YkJXW1407/HZA/DeD01YZdhCSB1vsnRaa022k8NuXh+2Gvh9H7sb/VV/Nu+RPAr2fkhwV5M5LmmkmVBp7zB1vqZdbzK3WqrM/Vn3BDx3ofEyhs51j1OdCJwJEhMug5j0E/e+vRAh8DOpViZRcoRyTSRz7zMiseGQCQnFhAXzhw/30NjezU2nDiM9NpyKxg6W5Vdyy783sXJ/NWAmou2paKK8oR3tpXdaUN3Cb9/P57XNA1vYatCQOMwU41v3uBk/KN0MoxeZnO7vvGEazffuhr9NN1lGq/9mGgGAwhWw/LemMRmz2DQUNQfMrOonT2fcrj+ZHmn9IRPfPbzWpFcuf6BviEJrE/6oLTATjLa95N7n7NHH57h7pSGR7sZ+5BnWcVZ4yBlvzz3FZOTYO0zP3knuPBzKyv9PG2fNtXjd9Ihn3AAnXWX2ZU52N84JuaaIoZPhpxsBRZuZ691tpteeO899jFMIrGu6GDLLCAaY97dZ4ZH4IXQFxxgRCbK844NWXv2oc0zD3FZnRHX9P8zA65qHTaMLsNkK8R1YCnuscM6hVea+5M41ol+yEX6fB79KhN8PMxO4tN3saygywjPvf8zM94zJcPHj4OgivXyZmUGfM8uI4IL74dq3TcbPzavgtvVw6ZNmVn3FTvPdTPOYyX+iGHU2XPZPn2UMgUwo8zvOxj45oq8mp1kewa7SRlJiwjgpO55PdldwyshkJmXHkxYbTkVjO/sqTc731qJ65o9K4cOdJruhrctOY3s3cRE9Y5LrDxphqQq09FNPvvWMcfUPfW56e57rLGScZHqdnU2w9yOTAplzsukZ7/0QyncYF37KNSa2/vp/gaMLbEEk16wzjcqc2+Hx+fDmrWYiXNFa42UsuNeEScLjzOzr/HfglLtNraX1T5kB3PpD7t78mMXwxd/Na2clWZRplD/5Bez7xIRP7Fa+/6QrrB41piFe97jJOkocTkdYChHt5Sb2nJAHh1dDXA6cfJsJNcVkGjFMGm6KEGbPNKIBJgMrc7L7Ho04wy1Qo881oRmAoR5CMGyh+VxgSiPEDzWeQnyOEaKGwxCdRmdoPCHdTSZ8d+hzE68HOOcBSB5pwiTPXmjEOSjMiMqH95vvo2yr+btorfFu0iaaSrmtNTB0ngm3pE8wx3W2mlDehn+aFM7595nvAEwDvuB+IxDB4ZA8ihEHLG9v6DwzZ2XBve7Plu4uNc/IM+Gyp817e4rv1wgRAj/jFoK+ap8YGUqwTdHt0OQkRnLy8CQ+2V3BLQuGA5AWG8buskYOVBkhcJas+GCHO82torG9jxCss4SgvzkK7V121h2s5dRRKV/x0w1ilILhC82jN5GJcPGj5vWhNfD27e7Get3jZvtJV0HWdJPlUbzerCtx0d858M6fGb7gfggONS798xeZ3unpPzO92LfvcF8nKMxkOp38A9M4vn07fGqNFaRPNGKRO88IQWSyaXC2vmAySlJGm8qxyz0mzcUN6VleJGm4CY3UH4aQcNoi0onorIHYTOMVHV5tBifDrGKJd2w1A5lKwdWvmHz2qBQTBkoeaTwEJyNOMzF1MNcMizXhoZQxxu72Bkgbb8JOLZXm8yXmGSGIyzZhjobDEJNOZ2gCUa1FJiRTtceMK0Qmmes5S57Pvd0IweybYdYt8NwF8MnPjZdx4d+NZ9deD3NuM9/J2scgwyppkneqeYAp2vbC5UZI44dA5lRjb+8Fl87/KweXPU/eyReYUu1fRu8SKV8zRAj8jHMcwJsQ2GyK5OgwyhvbyUmM5OpZOYxJj2HOcDPwlB4bTnVzB3srTGx5W3E9ZQ1tbCmqZ8HoFJbvqaK8oZ2wYBtVTR1MzzWL3qwrtISgybsQ/OuLQ/z63d2sutdLIxloDD3ZhADAVIRc97jpReecbLKURpxueoLTr4fEPIpyLmF4sBUCGb7QTHALizbew8m3uTOP2upNYxiXZf6eeJkRmuELTf57+XbTSDkHCDMmuUM2iXlm7CNtghlXOPs3JtwxdI5pPIOscZGYTFh4vxnbAGqSppOYmWd6tyf/wHw2Z4gJjHh5fm4nI84w3kBslvEcwmKtulFBpgedkGca8YZiY1dsthGCxDwjOE4hcApJXLaZMAUuIQBMYxyXbYQgcXjP72Hqteb5pKvM/bz1C+OdKQXJI0zPv2Sj8U7SxntP0bSux3+tcI8xfOc18zn6fO9zOJTbSd7oBd7f5xuGCIGfyUmM5Pq5uYwN8pLihun1lze2MyQxkvCQIOaOcGcfpMWF49Cws7SRsGAblU0dPPypiRnfMC/PCEFjOy9tKGLl3iq2/OwsyhvbKa5rI9im+p2ZvPqAKRlQJqmpPcmda8IGU652p6pOu940ev2FBGbf7H4dHGZ66d4IjTKT48CMD3zxd1PaOyHX9MqHzjGNaliseQa49i0IjjA56GPOc9evTx5lGmibrUe12pLs8xnpLCCYNs4d9vkyrn7Z/Tou29gTFGJ61N2dJjtr2nVmbAKMuFXvNcKROMx4TDGZpuJoaLTpgcdYQhDtIQTxlhCUbTGf3ZPg0J6Nuy0Ixpzr/nv6Dcau1KP8TM54u4/q+3/dECHwM0E2xc/PH8/y5VVe96fEhAMNXhe9Sbe8CbtDs2hCOu9uK+OFdYeZNyKZGVbvv6Khnb3lTTS2d3OgqpldZY0AzBuZzMp91TgcuseMZ7tDu8YQKhrb6bvCAnTbHSilCLL19WK+0UQkwA/WmdCKk96lwU8E028w8e6U0abBu229aUBtNjOY7bx+70bM2bid9pOeixedSObfZzJZwMTmnWMTnlVwh59mChPagsxAdPpEk7brWegufqgJbcVm0BHWSwgAkoYdm11TrjYP4bgQIRjkpFnppd6EwDmYDHD+pEw+2FGO3aH59qwcwkOCSIgMoaS+zTVhbeOhOjYdriMmPJhTRprQUW1rJ8nR7rkMu8saaeowaYsVjR1eheCOJVtwaM2j15zgMsZHwcZDdZyU7cfFaHrHkn1B8gi4cak7DdGzwc8+ins+epFv7AKYfJX79cWPez9m9i3mAWYA2DmHw5Op3zWiEJFAffwEE2pLHWO8COgbGhJ8iqSPDnKcjf2QxL4Lw3sKwfjMWEamRpMcHcaZ49Jc+9cerKXbml+wrrCWj3ZVcMbYNDLizLmVjR3cuWQzn+ab0NQXVvVTm4JKL5PV7A7NZ3urXAPTA0l+eSOXPrqaj3Z5D6N9o8ie5u55D1Y8lyQ9VsKiTVom0BwzAr73gQmPJVvilzb+BBkpHA3iEQxyLpmaRVRYsCsM5ElSlMkqCrIpsuIj+M3FE+i2a0KCjL6nx4WzfI8JOaXFhvH21lK67JpzJqSTGGUGBreX1PPGllKaO+ycNiaNtQdryUmMRKPNrOVejsi+yiaaO7pp7eymy+5wXWsgcM6wLmtoR/qL31BGnW0Ggn1YTkHoi0//i5VSi5RSe5RS+5VS93nZv0Ap1aCU2mI9+lmwN3DJTojkhnl5fSabgckqSo0JIy/ZVDadNjSRWcOSXPs9xeOSqdl02TVRoUGcOirFNZFtWb4RinUHa2jvsrO2oIbZwxJJizGT1Xqz6VA9AA4NpfVtXm3u6LZ7ncgGUFTbyj9WFvS7v7alk8Z2L4XAMPMkAOpbO73u/zIeXX6ATwLBm/g6o5RZj1sYUHwmBEqpIOAR4BxgHHCVUsrbkP5KrfVk6/ErX9nzTeW0samcPd771HNn6CgrPoJTrGyj08emER4S5FoLYZU1G7mxvZt/rCygsb2bs8enm8lqXrKKNh12l7soqu0rBBWN7Uz7f5+4JrX15tnVhfz63d39pq5+559ruf+17V73bS0y4ajalr5C0J+wONlX0cTvPsjn5Q2m9PHzXxzizS0BOrNaEHrhS49gJrBfa12gte4ElgAXfsk5wjHy64sm8j9neq9tkm6NA4xIjWZKTgJzhidx7Rwz2BkRGkRMWDDNHd0kRJoJZ48sO0BMeDDzRiaTGhtGZWMHhxvtnPPXldQ0m4Z70+E6xqSb2HVRXd+1EN7ZVkZzR3e/YwibrV69N2+iqLaVnaWNFFT1rYHU0tHNvkozX6Kul0fQ3mVn3u+WcdsLm2jqx5t4cqUpjuYUkadXHeTZ1YVejxWEQMOXYwRZgMfKExQDs7wcd7JSaitQCtytte6zIKdS6ibgJoC0tDSWL19+XAY1Nzcf97m+xhe2VVaZ7J/wjjrWrl7JTSOh6eA2lltFLaOC7TR1wMhYB4UoKlvtzEsNZs2qlbRUddLc0c2ywi52l7Xzj7dXMj4piIKqVi4eEcI+BZ9vySejtecCJ/9eYxr4DfmFLA/v6RV0OzRbi4x4fLJ6Iw0FPX9+nx42jXhRTWOfe7Gn1o5DmwLGBSWVNMfZXccUNtgpqW+npL6NrQcr+NWc8B6htLp2B69uNHYVV9WzfPlyyutbqG5UJ/yeB9pv7EQgdh0bvrDLl0LgLZ2gt/++CRiqtW5WSp0LvAGM7HOS1k8ATwBMnz5dL3BOijlGli9fzvGe62t8YVtGeRN/3riC06aPZcGMnD77c/euobygloWTR3KwupmXNxRz41lTWDA6lbq4Yl7eu5WttTZA0xaVQVh2MrCByxdOY1PdNlRMPKHZQ6hv6+LciRkcqmmh4IPlAHSExLBgwdwe19t8uI7uj0yd+MTs4SyYl9dj//PPrAcqaeqEk+edQliwe8bn3hUHgHymDU2gqb2b6GiH634tWXcY2M7l07N5eUMxU2bNxe7QfFFQy3mTMvjnqoPY9S7mj0phS1E9c+adSusH7wOamXPmERkaTLfdwdvbSjlvYiahwcfvKAfab+xEIHYdG76wy5ehoWJgiMff2ZhevwutdaPWutl6/R4QopTyzerMAciotGj+euVkLpyc5XW/c8B4fGYsV8zIYfGkDNfMZef6yrXtRrs3H67j0/xKokKDmDY0gSEJkRTVtnL/69v5yRs70FrzzjYzs3T+qBSvS2g6V2KzKSjrFRpq77Lz+YFqV12k3nWQthY3kJ0QwYjU6D6hoV1ljUSHmbkRYLKKlqwv4gcvbOJQTQubD9eRGRfOlJx4Gtq6eqzh4BzneGVjMf/z0lZZw0EISHwpBOuBkUqpPKVUKHAl8JbnAUqpdGX58EqpmZY9NT60KaBQSnHh5CzCQ7zUUsFd8G58ZizThibw8LenutJBUz0yjvKSo9hZ2sjHuyo4dVQKocE2hiRGsK24nsKaVmpbOjlU08qy/EomZccxa1gitS0mtOTJJqtBHpoU1ad8xbqDtbR3Obh4ihGt3gvu7C5rZFxGLPGRodS1dvYYHN5V2sjYjBgy443NpoyGEaK1BbVsLa5nck48SVbK7H6rSB/A4dpW7A7NEytMiMtp1/7Kph7rO3y+v5oHP/ayWPwgR2vtulelzQ4+t5IDBMETnwmB1robuA34ENgNvKy13qmUulkp5SzAchmwwxojeAi4Un9Z+odwwrhixhB+fv44kjxmFjtxzmgGuPGUPLodmurmDteSm0MSI/FoJ1m5v5otRfXMHZHsmgVdVNtKe5c7lXTz4Xqm5CSQERdOaUNPj+CLghqCbYoLJpvyCeUeQtDeZaewuoUx6TEkRoXQZde0WxUUHA7N7rJGxmfGkR5nJt1VNJh6SgDv7SijqLaNyUPiSYwyn2lfhXsBmEM1LXy8q4ICa/Z1eWM7h2taOfPPK3pkPv3xoz08/Ok+uuyOo729g4Ifv7GDG541aw+/XdDJnS9t8a9BwqDEp/MItNbvaa1Haa2Ha61/Y217TGv9mPX6Ya31eK31SVrr2Vrr1b60R+jJyLQYrp+b53VfdFgwkaFBpEQoFlnpqUrBwtEm/DIkwTT2i8anEx0WzD9WFtDt0MwZnuTat6u0kVn/t5R/fXGIwuoWSurbmJGbQEZcBGX1fT2CCVlxDEs26+l6zmHYX9mMQ8OYjFgSIk2vvqlTU1jdwv6qZlo67YzLiCU1JgylTGPuzEpyTqibPCTBNYluT7nbIyiqbeXZ1YXkJEaSERdOeUM7B6qb0Rp2lze5jtl8uB6HHvg1orXW/Oi1bazad+w9+fYuO29uLmFnqcngauzQVDV10Nn99RIzwffIzGLBK0opxqTHkEAzSdFhDE+JIj4y1OU9jM2IQSm4YuYQWjq7WbmvmtAgG9OHJtLeZbrrT68+SENbF69vLnFlCcwfnUpNSyeVTe102x0E2RQd3Q62FTdw/dxc4iJCCA229QgN7bYK5Y1Oj3Etr1nU5OD+Bz8j3kp9HZcZS0iQjaSoMMobTAZRtJUeG2RTTMiKdXkJzjTUnMRIthQ3sL24ntsWjuDzAzWUe3gTzmu9tdU9tFVU18oQL3Wf+uM37+5i8+F6Fo5J5funDDvmgei9Fc28uK6IIJti3shjGz77fH81LZ12OrodaK1ptIZWKpvayU44+s8gfPORWkNCvyy56WS+Pcb0oh//zjT+csVk174RqTGsu/8MFo5OZUqOKYo2JSeeiNAg4iNDiA4LZkeJacA3Ha7n1Y3F5CRGkpsUSUZcBA5tJnVN/tXHvL65hE67gxm5iSilSI8N79Hz3lPeRFiwjdykKBKsXv2uGrsVruok2KYYkWrK42XEhbO7rJH2LgfnTDCezKi0GCJDg10ewf7KZkKDbYzNiGFrkenpnzU+3Vr60z2+UFhjhODtraWu2kxOkfDE7tDc85+tvLe9rMd2rTUvbyhmV1kjf/hwD2sKvA9/rTlQw9Ld3mc8f5pvBq+9zfIGUw/qjx/u8Rqycoa2uh2axrZumjqNHA+0V+ONtQU1XP/0OrrtDhxaU+AxbiMMPCIEQr+EBttcpaZHpMb06Qk7ZydPG2qEwLlgjlLKdeyc4abkxdbiBuaPSkEpRYY1qPvXpftoaOviZ2/uQClcpbPTY8N7jBHsqWhiZFo0QTblCg3l1xqv44UbZ/Hwt6e6BsTTYsPZWWoE6LQxqaTGhDF7mHnfhMhQlILWTjvJUaEMTTJhqMy4cMZnxpJmXbfYyiQ6WN1CUW0r+eVNXD83F5vyLgTPri7kPxuLeaPXGtClDe00tHVx83xTGWlveVOP/VprHvxoD1c9+QX/89IWr7Ojl1lC4K0AIMBP3tjBw8v2s624vsf2bruDT3abLC+AquYOtxD0814Dyaubilm2p4qKpg7Wldk5888r+iQIBDLPrSnkwkc+H7DriRAIX5lZeYlcNTOHy6Znu7YNSTADt7efPpLcJCMK862lLzOtQd361i7GZ8bSZdeMToshzgrzpMWF92gU8subGJMeC5jlOwFKmjWZceHMGZHMognuEhsZceGuaqvZCZG8e/sp/PBss8pXkE0Rb6WnJkWHucTqzHFpxhOJC6O1087uciMkTe3drkqnp4xMIT023OUtOCmqbeUPH+4BcA04O9ltCdLcEUmkxISxp6KnEGwvaeChT/eTFR9BY3s31c0902LrWzvZcMi5NkRfj2BZfqXLvt4Ctb2kgdqWTtfge0FVM3ZLZ8ob2unotrNsTyWPfXaAsgbvNaO2FNXz/ec2uEJ9vfloZ3mPZVGPhQ2FplRJZWM7Fa0O7A7tKpcuwJbD9Wwtqqet00frSvRChED4yoSHBPHbSyaSFe8ulT11aALDkqOYkZvI4kmZRIUGcbLlHTg9ApuCf1w7nfNPyuTKGe4pJ+mxJs7/X89vYP4fllHV1OEqaxETHuzyUkan9y3T7CyrAZCVEEFKTBgRoe70WWd4KDk6lLHW+edONKtlOWszFVS1uLKmXt1YTExYMKPSYshOiKS4ro361k7WW8t9/n35ARxac9HkTA7VtLhECDzHNmIZlRbdI1sJTGor4CoR0js88tneKhzaCElVcwd2h2ZvRZOrTMbvP9zjEtneQrDDEqEzrCyvvR7Xrmhs58GP9nL90+t54P18Xlh7uM99dDg0P359Ox/vqmBfRd+wjcOh+embO/jr0n199gHUtXRy3dPr2GKVFPGkurnDJZpVTR3Ud1jprR5zS5burnCtrR2IVFklXQbKexMhEHzCzfOH88ld8wmyKf779BF88r/ziQozuQmx4SHERYQwf1QKGXER/O2qKVznkb2UFhtOR7eDD3dWEBESRLBNMSvPiIjNply1kUZbXoInzoqr4SE213GeJFkppEnRYUzPTWTlDxe6KrZ6Vmt1TqzbVdbI5Jx4gmyK7IQISuraePDjvVz++BpWH6jmzS0lXHBSJvNGptBl11S3ecxvKGskNymSaEtI9lU295ibsPZgLXnJUa7QVW+PYn1hLTFhwZw5Ng27Q1PT3MEVj6/hgfd309Texe6yRi6Zmk1ydGgfT2VXaSNxESFMzDKL+Oz1aMzLGzvYfLieCVmxZMaFc9jL5L83tpS4QmyHavv21LeVNFDR2NGvN/HTN3ewfE8VH+/q6zFsKHQ38FXNbiEoscSsrqWT/35xM3/4MN/rewcCTu/Qc+JlUW3rlxZXPF4ka0jwGc4lMMOCg8iI67mwzlPXTSczvu9iO+Du1Z88LIl/3zirx3uBifVXN3e6vARv52bFR3gt3e30CJKizbPnuIenN3HysCTe2FyCQ7vHQLITInhjSxsf7ChHa/j+sxto7bRzzeyhOKx/0LIWB/9ee8g1aD02w4jVqLQYWjvtlNS3mTkYDs36wloWjU8nMy6C8BAbByp79rx3ljYyLjOWDOs+bStuoK61i/WFda6B+EnZcWRZnoon5toxrs/r9Ahiw4Mpq28jv7yJi6ZkcqCyxess8IeW7mNMegz55U1eheIjayC6vrWL1s5udpQ0UtvSwaIJGby7rcw1y3yvF29ifWEdocE2uuwOqpo6aHB6BJaoPLumkNZOO/srA3cA2VnksdQa2G/t7OaCh1dx0ZQs5vtgvSLxCAS/MG1oYh9xcDIzN5Ezxqby+8smYbOpHiIAuAaMvYWGnOGd/kQm0RKA5Chvk+jcQjAsJYosa5zDLQRmEl1lUwdzRyTR0mlnYlYcJw2JZ1iKyVoqqHfwy7d2ceu/N3GottVDCMx+Z4O8t7KJhrYuZuQlYrMpcpOiengEdo+Jck67Pj9g5hIcrG5h+V4ziDwxK47shIgeQmB3aPLLGxmXEUdwkPGMnBVdx2XGsr2kgeaObsZnxjEkMYKiXiJS09xBYU0rl00z3sbhGi9CsKvCFaIrrW/nL5/s5YevbKPb7uDZNYUMT4nirHFpXhvzDYW1ZoJfZCiVHqGh4ro2Wju7eWZ1ISFBirrWLleD6InDoXltUzEtvWauf1NwODQ1Vviv3BLHJeuKqGvtYvGkDJ9cU4RAGHSkxobzj2tn9JuvnxAVQpCC4Sl9V1T29Ai8kdTLI/AkPCTINS8hOyGS3KQobAomD4m3tpn3DLIp/nbVVK6cMYS7zzYracVFhJAcHcayoi467Q5sSqE1jLOEYGSaES3ngLEz/j0rz4SFhqdE9xgjKKhqpr3LwfjMWFfIavV+d/rpS+uLyIqPICk6zBWycoadDla30N7lYFxmrPVZw+i00kvHZcTR0e18HUtOYiRVTR09BiV3lxkbx1r7D/USggNVzeyvbObs8Wb8oazBrIvd2N7Nin1VbDxUx9nj0xmTEcuhmpYeg81Fta1sL2ng5GFmAL2ysd3tEdS38fGuCupbu7jp1GHWtfqGpVbur+aul7f2mN/RmyOFUB54P59XNxb3e94nuyro9tEM8v2VTf3uq2rqYG9FE/VtXdgdTi+pnc5uB0+uLGBmXiLThib6xC4RAuFrx/xRqczJDPY6OSs6LJjFkzJcg6S9cYeG+noEYMYJQoNspESHsXhSBlfMGEJMuFscAGYPSyQxKpQHLp3kyoQCGJ4SRXOXKc/x1HUzmJoTz/Rc403EhoeQERfuGnj9NL+SjLhwl7gMT4miqK6Njm7TaDrj8xOy4kiONmmveyqaiAoNIiRIUd/axaTsOJddnXaHa4BxlzVI7RQhp/hFBrvXvg6yKUanu1OCPccYnIPcTiHoHRp6eYOZ4PY9a1ynoKrFVaPpt+/lY3doFoxOZVRaNA5thMPJEysKCLbZ+PasHFJiwthbYbKZwoJtlNS3saGwjuiwYK6YbqrlevMo3tpS2u8+gKc/P8iCPy53NaaeVDV18PiKA7zSjxB8uLOCG5/b0O+62M+uLuTyx9d43Qfm3lV78WIAVu2r5owHV7DZY3EnT37yxnaufWpdj/PLG9p5e2spZQ3t3LrAdwu0ihAIXzu+PSuHGyZ6b8gBHv72VM4Y510InGEWz1pKnmTFR5CdGIHNprhiRg6/vWSSa19GfDhjM2L59syhXs91hocWjU9n9rAkXrt1LvGRbs9jak4CH++q4O2tpSzfU8W3Z+a4xjGGpURjd2hXGGZnaQNhwTaGp0QRHGQj2RKukWkxjM80AjDRJQSmcXc25rvLGgkJck+yS7bme8SEKpd3MTwlivCQIJe4FdW10trZjcOh2VXWSHpsOIlRoeQkRVHW0OYqS9HRbec/G4o5Y2wqk7LjUQpWWyGrIJtiX2UzMWHBTMmJZ2Sq8YKcDXZ1cwcvbyji4ilZpMWGkxId5hKZiVlxtHc5+DS/kpOGmHBXREhQn8a+vcvuGp844GUSWnuXnYc/3c+hmlbXhEBPPtxpxnf6S1V9bZMRiB0l3hdWem5NIesO1noNWbV2dnPZo6v5o5VO3BvnaoBOofakrdPOZ3urKGtod81ojw4LprS+jWV7KkmPDe/R6TjRiBAIAcUZY9N4/DvTGJ3mfcTtvnPG8MdvneR1X0iQjffvOIXz+onTDk8xE9TOmeh9/33njEFrzX+/uJnk6DC+57EewzDrXGfDt7O0kTHpMQRb1WCdwjUiNdo1ZjEpKx5wz9lwjhNsLapneEq0y2NKtjyC2FBFmhU6c3oLzgKBeyuaOfX3y/jjR3vYZQ1SAwy1igs6ReaDHeXUtnRy9ayhhAYbgVpzwISsnDO5541MJiTIRl5yFME25RoXeXHtYTrtDm6ab8I+KR5iPCXHfJaS+jam5iRgsymGpURxoKqZZXsq+djqoS/fU0lTRzfpseFeheA/G4pc8fX8sr5hGOe8h/LGdlo6ull9oNq1/GptSyfLrDLk3hrrfRVNrlDVnvK+7/3J7kpaOu1ezwVcKccHKvuK0Kr91bR3GbHdeMjYMyErlrKGdtYX1jIjL9Fr8sOJQoRACChCg22cPT6933+qkWkxTLVKZhwrl07N5tpxoczM9R7HHZIYyU8Wm2W77zh9hCudFmBkqsnw+evSfVQ0trOjpIFxVs8f3OtDjEiNZvGkDKYNTXA1nlnx7rkEVU0dfFFQw2ljUl3nOsNgsWGKbGvsZGK2OTc5OpSIkCD+9cUhqps7eWZ1Ifurmt1CYc1TcPbcn19ziKFJkcyz0mszrclwADfMy0MpM0EPzL3OTY5yZQ69u72M6UMTXGM7KdGeQuC+5877PyI1mu0lDdz6r03894ubKGto49nVh0iODuVb07MprmvrMf7Q0W3niZUFTMyKI8imyC/v2SDXtXSypqDGJdgHq1u45z/bXGtkv721lC67ZkJWLLtK+zbm7213p8LmexECZ8hqb0VTn7BUe5fdNfvbm4B9vKsc50/SKRgTs+Ks9TM6mJF7fL/Jo0WEQBBOEAlRoSzMCemT5eTJVTNzWPq/87lmds/wUkRoEA9efhL55U2c8rtltHbaOXeie8a0c32IESlm/elXb5njEpKI0CDXXIK3t5bi0LjWdQD3wHhMiCI1NpwlN83m6lkmBq+UcmUdxYQH09ppx+7QrmynoYluIdhSVM+GQ3V89+Rc12fMtDyM9NhwpuQk8Mld83tce2RqNHvKmzhY3UJ+eROLJri9JWeJEnAPyIPbOxieEk1tSyd2h8bhgG8/uZY1BTXccfpIRqXF9AnxPPjxXopq2/jhotEMS45yDXo7eW7NIewOzS0LRgCm9HmJlUpb0djOq5uKGZcRy0WTs6j0SGt18v6OMmbkmiq2vT2ChtYuPttbSVpsGO1djj7jKluK6umyaxKjQvsIgd2hWbq70jWutb2kgSCbcs2mB3f5FV8hQiAIA8zwlGivHsmC0ancfdYoRqRG8+otc1wrrkHP0JA3xmbE8s7WMp5ZXciErFhXlhK4J9HFhJlrzh6W1GOxImd46PLpQ1xlxp2hoZSYMCJCgthd1sQ/VhYQExbMFR6zwJ0pwHlW+fDen23+qBQO17Zy18tbAFyZRs73BogINqVBIkKCGGZVufX8rN+bl8c1s4dysLqFk4clcfWsoS6vwtmobiis5YkVBVw1cwinjExhTEasa9AbTJbWX5fu5YKTMlk8KQOl6DFg/M9VB9lW3MAlU7Ncn72oye1tvLyhiPzyJs6bmMHotBjyrXCXw6H5/nMbWPin5XTZNf99mllpd08vb2TdwVqUgosmZ1FS39YjS2t9YS01LZ1ccFImmXHhdNk1SVGhrhTomHAzIdGXyIQyQRhE3HbaSG47rc+y3Zw3MYOWjm5Xo92b3106iW89tobDta385LyxPfYlR7vHCLzhzBy6cHImUWHBTMoudZWuUEoxd0QyL64zZShuOnUY0R4hLeeqcLmWEPTm8ulDeHVTMesL65iUHdej/LVzhbz4MIVSihl5iYzNcDd4C0encs/Zo7luTi7ddo1Gc+Mpw7DZFHnJUShlxlSKalu5+V8byU6I4MfnmdDbmPQY3t5aSkNrF+/vKOO37+eTkxjJby6eQHhIEFnxEeSXNxEabCMuIoR/rCwgyGZW9AsJMvfpcKPDjFHkV/LA+/mcMjKZq2cP5VBtKy+tL8Lh0KwpqOHjXRWcMTaNOcOTuGRqFj99cwd7yptZNMF9Hz7fX83otBimDo3nqc+NJ+MUnLe2lhIREsTpY1N5cd1hShvaSY4Oc1W8nTY0wTVnw1eIEAjC14CRaTGuRs4bmfERvPD9WTz9eSHfmj6kx76cpEhCg21kRXsPAFw6NZvosGAmZsWhlHLVPnLy96un8o9VBby/vdyVMurE6REM60cIbDbF7y87icUPreyzdnaKNe4Rb3kqz31vZo/9EaFB/GDhCNffPz9/fI99WfERfL6/mre2ltLZ7WDJTbNdIuUUlGutekczcxP53WWTXKnAeclRFNe1MTErjuEpUby8oZjTRqe4vJSs+AjeO9jGy3/6DDChq0euNku5jkk3s8SL6lp5Yd1h4iJCePjbU1xeVk5iJHsq3B7BmgM1rD1Yy72Lxrg8mYLqZsZlxtLZ7eC97WWcOS6NyNBgcpOjWH2ghuSYMNLjwokJD+bUkb7LFnIiQiAI3xCGJkXxiwvG99meGhPOlp+dybrVq7yeNzE7zpWK6o3QYBu3LhjBrQtG9Nk3PNUIgLN364285Ci+uP90okJ7Njex4WYuSFzY8fV2h6dE89neKhIiQ/jHtTMYker2JpxjHFuK6rnj9JHcecbIHiGr4SnRrNxXzdSceKbmJPDyhmIunequnjttaALvbWvjBwuHc/n0IeQkRrrOd9a4+nhXBR/tLOea2UN7hNpGp5nSHK2d3dS2dPKb93aRFR/B9XNzAbPS36f5lXy6u5LEqFDqW7u44CRTJTbPKo2eHB1KeEgQK+5ZSGxE35pZJxoRAkEIACJDffOvPiY9luV3L2Bo0pFXPHP2xD1RSnHTKcMIbig6rmtfNTOHzPhw7jpzdI+BZzCD19OGJjB3eFIfEQB3uu7UnATOHp/Oc9+bySkeK8D93yUTOSOpngvOGtPnuqPSogkLtvHrd3e77PBkTHoMH++uYPIvP3bN6P7LFZNdYpGdEMFrm0qwKXBoMyv9VGuOgDPE5pw34lyIydeIEAiC8JXob3zgaLj77NEsX1725Qd6YdGE9B5rUXiilOLVW+b0e+6CUamcNqaKOSOSsdmUqyF2Eh0W3O+YSmRoMB/eeSrrDtZis6k+A7mnj03jk92VzB6WxJj0GFJjw3pMBps/KoUdJY08dOUUDlQ3ExkS5JrzkZfs9ggGEhECQRACjpykSJ66bsZxn5+bHNWvAJ40JJ737jil33N/fdHEHnZ4Miw5ittPG8E5E3xTXK4/RAgEQRAGCTab4q6zRg/8dQf8ioIgCMKgQoRAEAQhwBEhEARBCHBECARBEAIcEQJBEIQAR4RAEAQhwBEhEARBCHBECARBEAIcpXXfBZ4HM0qpKuDQcZ6eDFSfQHNOJIPVNrHr2BisdsHgtU3sOjaO166hWmuvpUy/dkLwVVBKbdBaT/e3Hd4YrLaJXcfGYLULBq9tYtex4Qu7JDQkCIIQ4IgQCIIgBDiBJgRP+NuAIzBYbRO7jo3BahcMXtvErmPjhNsVUGMEgiAIQl8CzSMQBEEQeiFCIAiCEOAEjBAopRYppfYopfYrpe7zox1DlFLLlFK7lVI7lVJ3WNt/oZQqUUptsR7n+sG2QqXUduv6G6xtiUqpj5VS+6znBD/YNdrjvmxRSjUqpe70xz1TSj2llKpUSu3w2NbvPVJK/cj6ze1RSp09wHb9QSmVr5TappR6XSkVb23PVUq1edy3xwbYrn6/t4G6X0ew7SUPuwqVUlus7QNyz47QPvj2N6a1/sY/gCDgADAMCAW2AuP8ZEsGMNV6HQPsBcYBvwDu9vN9KgSSe237PXCf9fo+4HeD4LssB4b6454BpwJTgR1fdo+s73UrEAbkWb/BoAG06ywg2Hr9Ow+7cj2P88P98vq9DeT96s+2Xvv/BPxsIO/ZEdoHn/7GAsUjmAns11oXaK07gSXAhf4wRGtdprXeZL1uAnYDWf6w5Si5EHjWev0scJH/TAHgdOCA1vp4Z5d/JbTWK4DaXpv7u0cXAku01h1a64PAfsxvcUDs0lp/pLXutv78Asj2xbWP1a4jMGD368tsU0op4HLgRV9dvx+b+msffPobCxQhyAKKPP4uZhA0vkqpXGAKsNbadJvlxj/ljxAMoIGPlFIblVI3WdvStNZlYH6kQKof7PLkSnr+c/r7nkH/92gw/e6+B7zv8XeeUmqzUuozpVT/K637Dm/f22C6X6cAFVrrfR7bBvSe9WoffPobCxQhUF62+TVvVikVDbwK3Km1bgQeBYYDk4EyjFs60MzVWk8FzgF+oJQ61Q829ItSKhS4APiPtWkw3LMjMSh+d0qpHwPdwL+tTWVAjtZ6CnAX8IJSKnYATervexsU98viKnp2OAb0nnlpH/o91Mu2Y75ngSIExcAQj7+zgVI/2YJSKgTzJf9ba/0agNa6Qmtt11o7gCfxoUvcH1rrUuu5EnjdsqFCKZVh2Z0BVA60XR6cA2zSWlfA4LhnFv3dI7//7pRS1wKLgau1FVS2wgg11uuNmLjyqIGy6Qjfm9/vF4BSKhi4BHjJuW0g75m39gEf/8YCRQjWAyOVUnlWr/JK4C1/GGLFHv8J7NZaP+ixPcPjsIuBHb3P9bFdUUqpGOdrzEDjDsx9utY67FrgzYG0qxc9emn+vmce9HeP3gKuVEqFKaXygJHAuoEySim1CLgXuEBr3eqxPUUpFWS9HmbZVTCAdvX3vfn1fnlwBpCvtS52bhioe9Zf+4Cvf2O+HgUfLA/gXMwI/AHgx360Yx7GddsGbLEe5wLPA9ut7W8BGQNs1zBM9sFWYKfzHgFJwFJgn/Wc6Kf7FgnUAHEe2wb8nmGEqAzowvTGbjjSPQJ+bP3m9gDnDLBd+zHxY+fv7DHr2Eut73grsAk4f4Dt6vd7G6j71Z9t1vZngJt7HTsg9+wI7YNPf2NSYkIQBCHACZTQkCAIgtAPIgSCIAgBjgiBIAhCgCNCIAiCEOCIEAiCIAQ4IgSC0AullF31rHZ6wqrVWlUs/TXfQRC8EuxvAwRhENKmtZ7sbyMEYaAQj0AQjhKrPv3vlFLrrMcIa/tQpdRSq4jaUqVUjrU9TZl1ALZajznWWwUppZ606s1/pJSK8NuHEgRECATBGxG9QkNXeOxr1FrPBB4G/mJtexh4Tms9CVPY7SFr+0PAZ1rrkzB173da20cCj2itxwP1mFmrguA3ZGaxIPRCKdWstY72sr0QOE1rXWAVBivXWicppaoxZRK6rO1lWutkpVQVkK217vB4j1zgY631SOvve4EQrfWvB+CjCYJXxCMQhGND9/O6v2O80eHx2o6M1Ql+RoRAEI6NKzye11ivV2Mq2gJcDayyXi8FbgFQSgUNcM1/QThqpCciCH2JUNai5RYfaK2dKaRhSqm1mE7UVda224GnlFL3AFXA9db2O4AnlFI3YHr+t2CqXQrCoELGCAThKLHGCKZrrav9bYsgnEgkNCQIghDgiEcgCIIQ4IhHIAiCEOCIEAiCIAQ4IgSCIAgBjgiBIAhCgCNCIAiCEOD8f7kwprWnOa29AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABJq0lEQVR4nO3dd3hUVfrA8e+ZyaT3hIRAgEDovXchqEjRBQsqtlWURUVX1/JTd9dddcVV13Utuyj2ggV7R8VCQAEpoSbUEAKEkkp6T87vjzOpJIGETCY67+d55snMvXfufefO5L73lHuu0lojhBDCdVmcHYAQQgjnkkQghBAuThKBEEK4OEkEQgjh4iQRCCGEi3NzdgDNFRoaqqOiolr03oKCAnx8fFo3oFbQXuOC9hubxNU8Elfz/BbjiouLy9Bad2hwptb6V/UYMWKEbqmVK1e2+L2O1F7j0rr9xiZxNY/E1Ty/xbiATbqR46pUDQkhhIuTRCCEEC5OEoEQQri4X11jsRDit6GsrAxfX1927drl7FBOEhAQ8KuNy9PTk8jISGw222mvVxKBEMIpUlJSCA8PJzIyEqWUs8OpIy8vDz8/P2eHcZJTxaW1JjMzk5SUFLp3737a65WqISGEUxQXFxMQENDuksCvmVKKkJAQiouLm/U+SQRCCKeRJND6WrJPXS4RLF2XzKdbjjg7DCGEaDdcLhEs23iYL7cfdXYYQgjRbrhcInCzKMor5WY8Qri67OxsnnvuuWa/b+bMmWRnZ7d+QE7kconAalFUSCIQwuU1lggqKiqafN/y5csJDAx0UFTO4XLdR90sFsorJBEI0Z489EUCO4/mtuo6+3fy54HfDWh0/n333cf+/fsZOnQoNpsNX19fIiIi2Lp1K+vXr+fCCy/k8OHDFBcXc/vtt7NgwQIAoqKi2LRpE/n5+cyYMYOJEyeydu1aOnfuzGeffYaXl1eD24uJiWHYsGHExcWRnp7Om2++yaOPPsqOHTu4/PLLWbRoEUCj212xYgX3338/5eXlREdH89prr+Hr69sq+0pKBEIIl/TYY48RHR3N1q1beeKJJ9iwYQOPPPIIO3fuBODVV18lLi6OTZs28eyzz5KZmXnSOvbt28ctt9xCQkICgYGBfPTRR01u093dndWrV3PTTTcxe/ZsFi9eTHx8PK+//nr1+hvabkZGBosWLeLzzz9n8+bNjBw5kv/85z+tti9cr0RgVZSUN130E0K0rabO3NvK6NGj61yE9eyzz/LJJ58AcPjwYfbt20dISEid93Tv3p2hQ4cCMGLECJKTk5vcxqxZswAYNGgQAwYMICIiAoAePXpw+PBhQkJCGtxuRkYGO3fu5LzzzsNisVBaWsq4ceNa42MDLpgIpEQghGhI7XH+f/rpJ77//nvWrVuHt7c3MTExDV6k5eHhUf3carVSVFTU5DaqlrdYLHXea7FYKC8vJzY2tsHtaq2ZOnUqL774okOueHa5qiHpNSSEAPDz8yMvL6/Bebm5uQQFBeHt7c3u3bv55Zdf2iSmnJycBrc7duxY1qxZw/79+wEoLCxk7969rbZdKREIIVxSSEgIEyZMYODAgXh5eREeHl4979xzz+WNN95g8ODB9OnTh7Fjx7ZJTNOnT2fJkiUnbbdDhw68/vrrXH/99ZSXlwOwaNEievfu3SrbdblE4GaxSIlACAHAO++80+B0Dw8Pvv766wbnVbUDhIaGEh8fXz397rvvbnJbsbGx1c9jYmKIiYlpcF5j2z377LNZtWqVVA21BikRCCFEXS5YIlCUV1Y6OwwhxG/ULbfcwpo1a+pMu/3225k3b56TIjo1l0sEVouiQi4oE0I4yOLFi50dQrO5XNWQm1V6DQkhRG0OSwRKqVeVUmlKqfhTLDdKKVWhlJrjqFhqkzYCIYSoy5ElgteB6U0toJSyAo8D3zowjjqk15AQQtTlsESgtV4NZJ1isT8CHwFpjoqjPikRCCFEXU5rLFZKdQYuAs4GRp1i2QXAAoDw8PA6fW6bIz8/n6NHSiktK2/xOhwhPz+/XcVTW3uNTeJqnvYYV0BAABUVFY1e3etMDcUVERHBsWPHOHbsGPfccw9Lly496X0zZ85k0aJFDB8+vM3iakhxcXGzvm9n9hp6GrhXa11xqntsaq1fBF4EGDlypK59IUZzxMbG0r1bR/ShJFq6DkeIjY1tV/HU1l5jk7iapz3GtWvXLqxWq0MukDpTeXl5Dcbl5+eHn58fn376aYPvs1qt+Pj4OOwzNRZXfZ6engwbNuy01+vMRDASWGZPAqHATKVUudb6U0duVMYaEqId+vo+OL6jddfZcRDMeKzR2ffeey/dunVj4cKFADz44IMopVi9ejWZmZlUVFSwaNEiZs+eXed9ycnJXHDBBcTHx1NUVMS8efPYuXMn/fr1O+Wgc76+vtxyyy18//33BAUF8c9//pN77rmHQ4cO8fTTTzNr1iySk5O55pprKCgoAOB///sf48ePB+CZZ57hs88+o6SkhIsuuoiHHnroTPZQNad1H9Vad9daR2mto4APgYWOTgIAVosFrZF2AiFc3Ny5c3nvvfeqX7///vvMmzePTz75hJ9++omVK1dy1113oXXjx4rnn38eb29vtm/fzl//+lfi4uKa3GZBQQExMTHExcXh5+fH/fffz3fffccnn3zC3//+dwDCwsL47rvv2Lx5M++99x633XYbYG5Ms3//fjZs2MDWrVuJi4tj9erVrbAnHFgiUEq9C8QAoUqpFOABwAagtV7iqO2eipvVVEOVV1ZitVidFYYQorYmztwdZdiwYaSlpXH06FHS09MJCgoiIiKCO+64g9jYWNzc3Dhy5Aipqal07NixwXWsXr26+kA9ePBgBg8e3OQ23d3dmT7ddKYcNGgQHh4e2Gw2Bg0aVD2GUVlZGbfeeitbt27FarVWjzK6YsUKfvzxx+oqn/z8fPbt28ekSZPOeF84LBFora9oxrLXOSqO+qwWkwikRCCEmDNnDh9++CHHjx9n7ty5vP3226Snp7N69WqCg4OJiopq8D4EtZ2qjbM2m81WvXztexJU3Y8A4KmnniI8PJxt27ZRWVmJp6cnAFpr7rzzTm6//faWfNQmud6VxZaqEoEkAiFc3dy5c1m2bBkffvghc+bMIScnh7CwMGw2GytXruTgwYNNvn/SpEm8/fbbAMTHx7N9+/YzjiknJ4eIiAgsFgtLly6losLcUXHatGksXbqU/Px8AI4cOUJaWuv0vHfJsYYAGW9ICMGAAQPIy8ujc+fOREREcNVVV/G73/2OyZMnM3z4cPr27dvk+2+++WbmzZvH4MGDGTp0KKNHjz7jmBYuXMgll1zCBx98wJQpU6rvnHbeeeexZcuW6ltU+vr68tZbbxEWFnbG23S5ROBmNYUgKREIIQB27KjprRQaGsq6desa7KZZdSYeFRVVfR8CLy8vli1bdtrbqloHmF5KDc3r1atXnZLFo48+Wv184cKF3Hvvvae9vdPlslVD0kYghBCGy5UIrJaaXkNCCOEIY8aMoaSkpM60pUuXMmjQICdF1DSXSwRSIhCi/Wiqj/6v2fr165227ZbsU5erGrJKryEh2gVPT09ycnJ+s8nAGbTWZGZmVnc5PV0uWCIwuU9KBEI4V2RkJNu2bavTgNpeFBcXN/tg2hZOJy5PT08iIyObtV6XSwTVJQLpPiqEU9lsNvLz8xk5cqSzQzlJbGxsswZtayuOisvlqoakjUAIIepyuURgtUqvISGEqM3lEoGUCIQQoi6XSwTSa0gIIepyuUQgvYaEEKIul0sEUiIQQoi6XC4R1LQRSGOxEEKACyYCuY5ACCHqcrlEUHWrSmkjEEIIw2GJQCn1qlIqTSkV38j8q5RS2+2PtUqpIY6KpTa5Q5kQQtTlyBLB68D0JuYfACZrrQcDDwMvOjCWalbpNSSEEHU48ub1q5VSUU3MX1vr5S9A80ZJaqGqEkFZhTQWCyEEgHLkELD2RPCl1nrgKZa7G+irtZ7fyPwFwAKA8PDwEc25NVxt+fn5lFi9uWtVEfMGuDO5i61F62lt+fn5+Pr6OjuMBrXX2CSu5pG4mue3GNeUKVPitNYNj/CntXbYA4gC4k+xzBRgFxByOuscMWKEbqmVK1fq1Jwi3e3eL/XSdcktXk9rW7lypbNDaFR7jU3iah6Jq3l+i3EBm3Qjx1WnDkOtlBoMvAzM0FpntsU2rTLWkBBC1OG07qNKqa7Ax8A1Wuu9bbXdqiEmpNeQEEIYDisRKKXeBWKAUKVUCvAAYAPQWi8B/g6EAM8ppQDKdWP1V63IapUri4UQojZH9hq64hTz5wMNNg47klxHIIQQdbnelcVVbQQyxIQQQgAumAhk9FEhhKjL5RKBUgqrRUmvISGEsHO5RACmVCAlAiGEMFwyEbhZlPQaEkIIO5dMBFIiEEKIGi6ZCNykjUAIIaq5ZCKwWixSIhBCCDuXTARuFiXXEQghhJ1LJgJpIxBCiBoumQjcrNJrSAghqrhkIpASgRBC1HDJRCC9hoQQooZLJgLpNSSEEDVcMhFIiUAIIWq4ZCKQNgIhhKjhkolAxhoSQogaLpkIrBZFmVxQJoQQgAMTgVLqVaVUmlIqvpH5Sin1rFIqUSm1XSk13FGx1GeuI5BEIIQQ4NgSwevA9CbmzwB62R8LgOcdGEsd0mtICCFqOCwRaK1XA1lNLDIbeFMbvwCBSqkIR8VTm7QRCCFEDaW1486MlVJRwJda64ENzPsSeExr/bP99Q/AvVrrTQ0suwBTaiA8PHzEsmXLWhRPfn4+vr6+PLO5mIwizcMTvFq0ntZWFVd71F5jk7iaR+Jqnt9iXFOmTInTWo9scKbW2mEPIAqIb2TeV8DEWq9/AEacap0jRozQLbVy5UqttdY3Ld2kz30ytsXraW1VcbVH7TU2iat5JK7m+S3GBWzSjRxXndlrKAXoUut1JHC0LTbsZrVIY7EQQtg5MxF8Dvze3ntoLJCjtT7WFht2kwvKhBCimpujVqyUeheIAUKVUinAA4ANQGu9BFgOzAQSgUJgnqNiqc8qQ0wIIUQ1hyUCrfUVp5ivgVsctf2mmBKB9BoSQghw4SuLpUQghBCGSyYCaSMQQogaLpkIrBaL3LxeCCHsXDIRuFmlRCCEEFVcMhFIG4EQQtRwyUQgvYaEEKKGSyYCq0VRqaFSSgVCCOGaicDNogCocOCAe0II8WvhkonAajEfW9oJhBDCRRNBVYlAeg4JIYSLJgJrVdWQXEsghBCumQjcrFUlAuk5JIQQLpkIqksEUjUkhBCumQiq2gjKJBEIIUTTiUAp5d/EvK6tH07bqO41JG0EQghxyhJBbNUT+83la/u0tYNpKzW9hqSNQAghTpUIVK3nwU3M+1WRNgIhhKhxqkSgG3ne0OtfDbmOQAghapzqVpVhSqk7MWf/Vc+xv+7g0MgcSEoEQghR41QlgpcAP8C31vOq1y+fauVKqelKqT1KqUSl1H0NzA9QSn2hlNqmlEpQSrXJDextbuZjl5RLG4EQQjRZItBaP9TYPKXUqKbeq5SyAouBqUAKsFEp9bnWemetxW4Bdmqtf6eU6gDsUUq9rbUuPe1P0AIBXjYAcovLHLkZIYT4VWjWdQRKqf5KqX8opfYBz59i8dFAotY6yX5gXwbMrreMBvyUUgpTysgCypsTU0sE2hNBdqFD840QQvwqKH2KoZiVUt2AK+yPcqAbMFJrnXyK980Bpmut59tfXwOM0VrfWmsZP+BzoC+myulyrfVXDaxrAbAAIDw8fMSyZctO9/PVkZ+fj6+vL/mlmlt/LOSqvu5MjbK1aF2tqSqu9qi9xiZxNY/E1Ty/xbimTJkSp7Ue2eBMrXWjD2AtkAD8Dehln3agqffUeu+lwMu1Xl8D/LfeMnOApzCNzz2BA4B/U+sdMWKEbqmVK1dqrbUur6jUUfd9qZ9csafF62pNVXG1R+01NomreSSu5vktxgVs0o0cV09VNZSOOVMPp6aX0Ol2tUkButR6HQkcrbfMPOBje5yJ9kTQ9zTX3zxH4uiz+xkoysZqUfh72siRqiEhhGg6EWitZwODgM3AQ0qpA0CQUmr0aax7I9BLKdVdKeUOzMVUA9V2CDgHQCkVDvQBkpr3EU5TYRYRx3+ENNNWHeht40ShNBYLIcQpG4u11jla61e11lOBscADwNNKqcOneF85cCvwLbALeF9rnaCUukkpdZN9sYeB8UqpHcAPwL1a64wz+DyNCx9g/qYmABDo7U52kSQCIYQ41QVldWitU4FngWftjcinWn45sLzetCW1nh8FzmtODC3mF0GZmy+21HjA9Bw6IVVDQgjRdCJQStWvyqlvVivG4lhKUeATRWCqqRoK8rZxIKPAyUEJIYTznapEMA44DLwLrOdXPNAcQL5vNwLTVkFlJYHe7lIiEEIITt1G0BH4CzAQeAZzlXCG1nqV1nqVo4NrbQU+UVCaD9kHCfS2kVdcTnmFDDMhhHBtp+o1VKG1/kZrfS2moTgRiFVK/bFNomtl+b72Zo3UhOqri3OkwVgI4eJO2VislPIAzsdcWRyFaSz+2LFhOUahd1dAQdpOggKGAZBdVEaIr4dzAxNCCCc6VWPxG5hqoa+Bh7TW8W0SlYNUuHlBUBSkxhPQsWq8ISkRCCFc26lKBNcABUBv4DYzNhxgGo211rrRexq3Wx36QEYiQd7ugAw8J4QQpxqGulmjk/4qhPSEpFgCvayAlAiEEOK3d6A/lZBoKC8mqMJcwCxdSIUQrs4FE0FPAHzzkrEo6TUkhBAumAh6AWDJSpSLyoQQAldMBH4dweYDmfsJ9LJJG4EQwuW5XiJQyrQTZCbah6KWEoEQwrW5XiIA006QmUjHAE+O5RQ7OxohhHAq100E2Qfp7OfG8ZziqttmCiGES3LdRKAr6eOeTmFpBblF5c6OSAghnMY1E4F/JwAi3fIAOJpT5MxohBDCqVwzEXgHAxBmMzemOS7tBEIIF+bQRKCUmq6U2qOUSlRK3dfIMjFKqa1KqQSlVNvc48DLJIIQlQ9IiUAI4dqadc/i5lBKWYHFmJvZpAAblVKfa6131lomEHgOmK61PqSUCnNUPHXYSwR+Og+rRXEsW0oEQgjX5cgSwWggUWudpLUuBZYBs+stcyXwsdb6EIDWOs2B8dRw8wCbD5aiE4T7eUiJQAjh0pSjuk4qpeZgzvTn219fA4zRWt9aa5mnARswAPADntFav9nAuhYACwDCw8NHLFu2rEUx5efn4+vrC8DYdfPJDhzI1Tk3YrPAvaO9WrTO1lA7rvamvcYmcTWPxNU8v8W4pkyZEqe1HtnQPIdVDdHwje7rZx03YARwDuAFrFNK/aK13lvnTVq/CLwIMHLkSB0TE9OigGJjY6l+7+4IOvq50ycgnJ1Hc2npOltDnbjamfYam8TVPBJX87haXI6sGkoButR6HQkcbWCZb7TWBVrrDGA1MMSBMdXwDoaiLDoFeHI0u0guKhNCuCxHJoKNQC+lVHellDswF/i83jKfAWcppdyUUt7AGGCXA2Oq4RUMhVlEBHhRUl4pg88JIVyWw6qGtNblSqlbgW8BK/Cq1jpBKXWTff4SrfUupdQ3wHagEni5ze6LbC8RRAR4AqYLaZCPe5tsWggh2hNHthGgtV4OLK83bUm9108ATzgyjgZ5BUNRNl2DPAA4kFHAgE4BbR6GEEI4m2teWQz2awk0Pf3LcbModh3LdXZEQgjhFC6cCEIA8CjNoWeYLzuPSiIQQrgm100E9mEmKMyiX4Q/u47lOTceIYRwEtdNBN5B5m9RFv0i/DieW0xWgdytTAjhelw3EdQqEfSPMI3E0k4ghHBFrpsI7APPVZUIQBKBEMI1uW4i8PAHixsUZhHi60G4v4c0GAshXJLrJgKlwCsIirIA6Bfhz04pEQghXJDrJgKoHmYCTCLYn55PaXmlk4MSQoi25dqJwC8cclIA6B/hT1mFZl+adCMVQrgW104EnYbD8R1QVky/CH8AuZ5ACOFyXDsRdBkNlWVwbCvdQ33wtFmk55AQwuW4diKIHG3+Hl6P1aLo09Ffeg4JIVyOaycC3w4QFAWHNwDQP8KPXcdz5SY1QgiX4tqJAEypIGUjaE3/CH+yC8s4nlvs7KiEEKLNSCLoMhryUyH7EP07mQbjrYeynRuTEEK0IUkE3cabvwdWMTgykAAvG9/tSnVuTEII0YYkEYT1B/9I2PstNquFc/uF8/3OVLmwTAjhMhyaCJRS05VSe5RSiUqp+5pYbpRSqkIpNceR8TSycegzHfavhLJiZgzsSG5xOb8kZbZ5KEII4QwOSwRKKSuwGJgB9AeuUEr1b2S5xzE3uXeO3tOhrACSf2Zir1C83a18k3DcaeEIIURbcmSJYDSQqLVO0lqXAsuA2Q0s90fgIyDNgbE0LeossHnD3m/wtFmZ0jeMFQmpVFRKN1IhxG+fclSfeXs1z3St9Xz762uAMVrrW2st0xl4BzgbeAX4Umv9YQPrWgAsAAgPDx+xbNmyFsWUn5+Pr69vg/MGxD9GQE4Cv4x9mXVpVp7fVsJfxnjSO8jaom21VlzO1l5jk7iaR+Jqnt9iXFOmTInTWo9scKbW2iEP4FLg5VqvrwH+W2+ZD4Cx9uevA3NOtd4RI0bollq5cmXjMw/8pPUD/lpvek3nFZfpXn9Zrh/6PKHF22q1uJysvcYmcTWPxNU8v8W4gE26keOqI6uGUoAutV5HAkfrLTMSWKaUSgbmAM8ppS50YEyN6zYBIobAusX42iyc1SuUbxOOy1XGQojfPEcmgo1AL6VUd6WUOzAX+Lz2Alrr7lrrKK11FPAhsFBr/akDY2qcUjDuVsjYC4nfM21gR45kF7HjSI5TwhFCiLbisESgtS4HbsX0BtoFvK+1TlBK3aSUuslR2z0jAy4Cv06w7r+c1z8cd6uFjzcfcXZUQgjhUA69jkBrvVxr3VtrHa21fsQ+bYnWekkDy16nG2goblNWG4y5EQ6sJjBnN9MGduTjzSkUl1U4NSwhhHAkubK4vhHXgs0HPl7A38qeIbjkMN/EyzUFQojfLkkE9XkFwZS/gK6kQ8p3vOv5GF+u2SKNxkKI3yxJBA0ZfyvcugF17ReEWvK5Oe1Blm8/5uyohBDCISQRNKXzcCzTHmGEZR9fffE+ucVlzo5ICCFanSSCU7AOu5IyzxAuKfmUs/8dyzvrDzk7JCGEaFWSCE7F5oVtzB84x7qFRR5vEvfZYhavTHR2VEII0WokEZyO0X+AoCimlX7Pk+5LyPz+aR75aiel5ZUcyiykvELuXSCE+PVyc3YAvwo+oXD7NlRlBZXvX8vfdy/l1jWBDFo3gZLyShbGRHPP9L7OjlIIIVpESgTNYbFiueQl6DqOZzyXcFevNEZ3D+b1tclkFZSyIuE4qXLje9GelRaAdIUW9UgiaC6bF1zxLtaQaBYc/Rv/OstKUVkFlzy/lgVL41jw5iapKhLtU2kBPNkPtr/v7EhEOyOJoCW8guCqD8Hdh6ivf8+/O/9Ez6xVTOrdgW0pOTz+zW5e+fkAmw+dcHak4teqrBj2fG2el5fCri/NmXxlBez8rGVn9dmHoCQHjm1t1VDFr5+0EbRUYBe4+kN4YxaXZDzPJe6gL7icBd9YeOmnA9WLXTYykpmDIugc6FU9rUuwN542K8t3HCPU18MZ0Yv2LuFj+PRmuG0LHN8B7/8ebvoZco+Z5/O+hm7jm7fOXPsAiicOtn684ldNEsGZCB8Ad+6CvGPwv1Go9c/z5GX/Zn1SFn3C/Xh9bTJLf0nm/U0pdd7WOdCL8dEhfBCXgp+nG/8Ya2vduLSG7/4GUZMgegp8+1dI3wV+ETDjX5CVBBtfhqkPm9h/fBjKCqH/bBg1v3VjES1TddDOS4U8+1hXOUdqpmcl1U0EJfnw9b1wzt/BKxC+ugsm32tOWKrk2N+bLYmgXUqKhUPrIebeNt+0JIIz5eYOQd1gyFzYtgz/SfcwtW9HsFj5++/6c/e03mw5lM2JwlIAissqWbJqPx/EHWbWkM58m3CcN3eWcuE0jVIKoHpco6rXJ9Ha3D+htrJiM83NA/b/CGv/C+tfhKgJ5nXkKIj/GLIOQGYiFGVB2k7IPQqV5eDTwRw8rO4w/Pent82WqKyAilKw2MDqdvK6W2s7v3b56eZvYQYUZNinHYf8VPO8/ln9wTWw9S2ImgihvWHLUggfCGNrjfhelUSyD8l+bo82LzUlwQm3mbbINiRtBK1l3C1QXgxP9Ycn+0LqTgC83d2Y0DOUCwZ34oLBnZgzIpKvz0pib/CdPDPVj7vP68OWtApmL17DJ1tS+GlfOjOe+Ylz/7OKX5IyT95OQQY8MxjWLa6ZtvoJeCQcFoXDD/8w83zDTYLa/yNM+SvM/x4ufB5SNoCywLRH4dg2E/N1X5lqh+hz4Is/maqI2vZ9B//qAVvfPbN9VF4Ki0fDIx3h373g6BZY/wI80RMO/QJxb5jt7F95Ztv5LSioSgSZ5gGmZJBnH/Mqu94V7qnx5m/uEci1l0AzE6EoG57sR1DW5ppEUJILRfb2qw0vwTNDoaLcnCg8NciULnZ/ZRqWS/LqbufIZvN95aWe5ufIhH/3Nt+vaFr2QdCVkL67zTctJYLW0qEPXPYmZO43B7e358DUf0BAJHQdW7Pc7q+wLb/TfOG/LGb++f/h2KH9rDhSyp3vbSHGso3h3hqrBd58+QfU+CjGdA82Z+o9p8Leb8xB4Nu/gE8Y9J0Ja/8HkaPBvxP89KTZztl/g2FXm3/A/rPNtMGXgncwBHaD0J6mass3DML6mflzXoGnBprSxIVLIOlHOh5bCWteNWfxn90CxTmmuqHnVFMaao6ET8zBaczN5kDz5myzPosbvH0plOabJPXeNTDjMXD3Me/rPLJuFUdTik5AYRaERNedXpJnSkMRg5sX8+nIPWZiD+11ZuspLzWltE5DaxJBQYYpFYA9EdgPwPWrd1IT7LEcqdlvmftMw3DeUYKztoF7rbvtZR8Ed19Y/W9T0shKMicNOYdMFcX2ZZB3FNL3QGSt+50f3mBiS98FfuGn/kzpu00p5tAvdf8PAI5tx1JReho7xkVUJffUBOg0rE03LYmgNVUdcHtNhdfOh49uMK/Pf9LUvR9aDx9eDxFDzYFq67uoKfczKdLGX6+czJGfltJ15RNQbl+fO7DJ/gBTp5+ywdxFLbiHaUwccBEUZ8N5i8w/7AeVcGA1jLzeHPQHXFg3xp7n1DzvMbnuPK8gGHYNbHzJVB3seJ++YBLHNZ/AB9fBN/fWxDLhttPfN1rDuv9Bh74w/VEYdQO8Os3cK/qCp+CNWeZAetELsPQik3Sq4wqGG74zyasphVnwynnmrPmOePN5qnz7F9jytml8Dep2+nGfjuV3m1LMdV+c2Xo2vwFf3wN37YH8NDOtMNOcVYNJBPn29oL6VUP2Eii5R8HmbZ5n7q+e7lOQDMWFEBQFJ5LNQSd9T8360hJMEgLY9XlNqSwzsW4iqEpAp1siyLXfprx+CaYwC16aQqfuvwfOO711/ZaVFdVU+1V9l21IqoYcoeMg+NM2uHkd9J4BX91tEsC7l5uz9qs+gLPugvIi2PACAFYFXXe/bOp3F/4CC38h69rVXOH2NHPdnqIwYjSsX2L+QftMh7lvmwPnjvfNGXOX0WCxmlLJHfEmCbTEmBtNaWXH+zDmZjaM+q+JJyQa5v9gnncdb0o9xbnw4yOQUWvspUPr4eenobLetRRJK+H4dhi70NRNh/aC27fB7z83pak/xsH135rpt6yv3gfM+9qUEt66qOmDT0UZvHO5OVCV5kPc6zWx5KXCtvdAV5i4wSSmtf+Fg2vN81+WmDPhlji2DcoK4O1L6bvradM205LunUe3mn2flQQF9kRQu0SQf7ym4TjvmDmYfv+gSRoZe8302g3KOYfhiDmL8M1PNvO6TTDzThw0JcnQ3qCspjowbZeZt+MDsw/BJIL8NFj5qGnfqUpA+cdNKevHReYgVlZknpfk1/1MVdVU9UswJw5AZTl+eTJuF1A3UabGm9/CuufabPMOTQRKqelKqT1KqUSl1H0NzL9KKbXd/lirlBriyHjalFcQhPeHOa9C3/MhZaM5i7/6IzNkRVg/6H8hrH6CkIwNprHv2DZzoAzrB2H9CO4+hAfnzyHZ0o17jsaYf/DSfAqjzuWrfUV81O9pKrpPhnMfrGn4Uwo8/Foed3B3mHiHiWPaPyn06Qru9jNMN3cT24TbzT/4i5Nh9b9g6YWmeuToVnjrYvj+AdNrqUrmfvjIjNfE4Mtqpnv41TQYe/iaW4WCqdqw7wO6jYer3jcHxLfnnFxnXSXhU1NamvU/6BFj2kmqYnltOlSUQNdxsPlNUx31w0Ow4n546xL4/I+mpPP2ZXBwXfP2V1G2OeAOvRoCuxJ0Yht8/X+w5pnmrQdq6vkzE2vq8Gs3FuccMdUyAV0AeyL7+SnTrqMrwDvE3kZwtGade1cA4F6WbZJVWD/wDIBt70LqDhh/m0m+u78yPceizjLJyM0T/DubWLYshVWPmTadqgNW3nFI/N60TyXF1jxP/K7uZ6qKpX4Jxv7ap0B6MAE1+ye0j/kdLL8bvv0z5KQ0/b5W4rBEoJSyAouBGUB/4AqlVP96ix0AJmutBwMPAy86Kh6ncfc2Z+9/2gF/+NEkgyqzF0PEEAbGPwpvzTFVIEPm1nl7n45+fLxwPBXR53FYdaJIuzPqnTJueWczd32bwdSMO9nAALTWHMgooKw1rmo+5++m+sbSyM+j13kQ0tOcuY69xRy0nh0Gr0wFz0AYepWpBno8Ch7vDs+PNweXqz5qWW+IziNMSSc1oW6V0fF4WDLRXGy17r/m7HbQpTDuVnPArIolK8mUzKY/CqV58J8B5gA6eK5JyluWwoCLIbCrKbWl7YY938DLU83ZcPpeeH4CZB8+Obaq6pT+s2FBLOvGvQID55gE9Hj3kx9P9DINtPVVVtQ0Eh6Jq5lekG56eKFMKUFXmtIfmFIPwJ6vzN+eU82ymYkQZv9XK80zvYeq+HcynzNtp+kpNuhSs2zVtif+yZTAesSYNqSMRDi80cxLja9VNXS85uCVGl9TnZG5v+7nyqlVOinJg5fOMe0F9oTiXZhirnh+7XyTjLSG12aafbVkollu23vwwuST1526E5acZXpYZe6H5yeeXAXVkDXPwAfzTr1cW6rar31mmOrAFPs+3/ttm2zekW0Eo4FErXUSgFJqGTAbqK4A01qvrbX8L0CkA+Npfzx84coPOPzePXTtGGIOsA0cKDsFevH870fB4TdITtrLhVnRjOkRgp+nG3/7NJ7LXlhHVIg3yZmFnNsvnBeuGYHV4sCugRYLXPyS+ecbfCkMvBi2v2cafUfNN2f+HfrW/FNarKbh+lR1/E3pNRUm3Q2rHof0PXgUp8PbN5kqkvd/b86IL3jaxBZ9jmmX6THFxBIxFHpPM20D0x83B8qASJMwcg7Dri9gjH1dr0w1JZyibFN1t+sLc1abGg87P4XxfzQJqbTQJPWqRtrwAeavssCFz5nPn99AVVZqgjnbAxMXgE+ISQTl9nGqqg4CHv6QmWQO/iE9TdwAXcZA/EfmoN9xkKnWsXpA97NMI29hpjnAVyWpwZfBd/bShn+kafM5vgNG/QFsnib2hI8BZar9LnzeJI+tb8OBn2p6Kh1ca3ocgflsVQev1AQTP9TEmJNi9nFV1VB5sdmXRzaZDg/FpuHaoitMt8mDP5vG5w59Tek4+hxI2WQSRO4R8/2+dbFpK/INM+vc9YWpbkyKNVVVqTtMh4QJt5vqwIYas4tzYNUTJkGe97DpXVeS1/Kq1PqKTpjk6elvqjzBJEP/Tqa03tjz7IPmO4yeAmueNqU2jwCTCEbd0DqxNcGRiaAzUPsUKgUY08TyNwBfOzCe9sm3A0nR19I1JubUy3YZTVSX0TxSa9KKOybxzPf72J6Sw7joEN7dcJhb39nMgE7+nNs/nL4d/Skpr8BmP7v/JuE4ZRWVjI8OpYPfGVzV3Hm4eYBpTKzdoAjNa0g+XaMXmLO52EcZfGAjVBSarq9f/Mk0mFeVpiyWuhfGjVlQ87x2v3owVWFVsQZ1M0OHvDYT/CNMu8Peb0y1F5hSQmgfeOdS87pDP3N27hlo/qGruHnA5P9r+DOUFZsDWlUyAHNNxTl/N889/GvOrsP6weH15nn4gJqDbKdh5j2VZaZ3WOyjJgkH1moED+1tLiDMOwbdJ1NqCzTVQ/6dzMF2/481B5iqEkNItCnBVu3HkGiTDMuLzOuqs1Obt1mvm6d5nbrTXIsCJsbUBFOCuuoDk0T9OpkeSDs+sC+fYJKbh79JLGufNdMPbzS9ksB0fijKMh0HwgeYHnjLrjS9y677ypxEpdiXTdlQ03ay91uz/FuXQMyfIaZejfTmpSYJVC17ZLNJHvO+OvOeOieSTWeFqhOAqQ+bz/n9AyYW7xDzvY+/zSSJz26BkTeYk5YTB01JreNg892Omm9ONuJeM3+rqmcdRDnqpuxKqUuBaVrr+fbX1wCjtdZ/bGDZKcBzwESt9Umd55VSC4AFAOHh4SOWLVvWopjy8/Px9fVt0XsdqTXj+mhvKV8mlaExDdD9gq3sPlGBv7siwENxIMdUHVkV3DDIg/Gdmj4XaG/7rPee5+h07FsqlBvbhzxETuBArOWFWCuKKPUIaZVteBSnU2H1Iip5GZ2Ofo1Fl1NqC8BWlkeeXw/cS09wLOI8uie/S7nVk3zfnmwdZtLz6ewvS0UpATk7UboSS2UxAxKeoMLqjrWilIzQMXTIMO0URyOm0emYOfgeiLqS7snvALBu7MsM3Xo/7qVZrJnwFtaKEpQux1pRwpgNJtHtGHg/kSmfEpi9k5/Oeo9+2x4iNHcnqyd9gNLluJfmUOwVXv15x/0yn7QO49k5oOaq1sAT2xm6zbT15Pj3JSDXVB+dCByIf24iJR7BeBcdRWMBzHGk3M2X5KjL6ZX4MimdzyfyyFekhk0iPG01GguKSoo9Qqi0eFLoHUlQVhxWXY5GodBkhIwmMDuenye+DcqCZ9FxymwBVLh5EZy5iUE7HuFE0BB2DPob49deg628gDzfaNxLT+BeegJQ5PlF45+3D4Ddff7I8YhzAVCVZYxZfzPFnmF4lGRS7uaDT0EyFl1BqS2AzcMfp9izI+GpKzlmi8IS3J3w1JV4F9Zqc6klvcN48v16EJKxEf/cPXRIX4OtLJe9vRcSlvZT9fdY4h6ER+kJNIpS98CTnid1v5oO6esos/mzfciDeBekUOTVkcDseIZsf4Dj4WdT4hHCge5Xkl9Q2OL/xylTpsRprUc2NM+RJYIUoHbn70jgpD2qlBoMvAzMaCgJAGitX8TefjBy5Egdczpnzw2IjY2lpe91pNaMKyYGnqqoJKeojMe+3s2axAwuH9WZwyeKSErP51+X9KJvhB//XL6LF7dnUeHfiVun9CTIp+FrAtrdPhvUBd7Zz+7wixl24a2O3dZ+H1hquoS6z/wnfHYL/nmJMPVhuo+5CZ5ZhVveUQL7jK/eR6e/v2p1mVT7cYv/EDr0pUPfCfCTOYB0GnoO2BNB97EXgD0RjJt6IVRugMoKJp0zrWY9ZcVgTwSDJk6H/e5weAOTzjmPvce/o0NYRyaffe7JoWgNKS8SNvIawobWij2nF2z7G7j7ETB+XnXX4aBB0+CneLyLUyGwK6qqCrDrOGyH1tGr0tTlR+ZuBiB85CxYvhpFJVhseJZkgsUN72EXk7c9Db/8A6jBl8H29wjN3Ag9YoiZcnYD+ywGugQQvPxuJls3Q3kB+Efil5sEaBh8OWx/zySBs/8GSbH0TX6TvhffZ0oxn9wIJel4zlliGrXXLzFVeVd/hPtH8xm771/Qezrsfo7uHh3wHHkl7H7GLEO9qlZdSbfUFaaqMH6RWcYrGK75kAHdxkF5iamyROEx5xX4dCGqOAePy9+C5Xejsg/hccUy+Ooueux4y/TcGnFt3d9O+SRIeZOOaasA6HbdC8Su/skh/4+OTAQbgV5Kqe7AEWAucGXtBZRSXYGPgWu01nsdGItLcbNaCPH14IlLG++E9cb1o3ngswReXXOApesO0iXYiy7B3kSF+DBzUASjooIaH+LCmUKi4Y9xpMfGOn5b3SaYi658OsCQK+C7B0xd9/Dfmx5UYxaY7ptV7QMtNe4WiP/QNNoGdjXTbN41z6Gm8dc71PSumvnEyeuxeZrqh8JM0+Nnwu3Vs452Pp/ejR1AlIL535083S/CxBE5suZCPI8AU7UEpt6+z0xzQAXTC+7QuppuuFVtCyE9zcWPBWnQf5Zp36gsh8BuFPh0wy//AEy6x1TRVJTWNIY3ZOT1piPCqsfN6zE31vRQG3MjJP5gvqNR86H7ZHjlXNPWkXvEdIk++37oda5pt1q/BPrNgp7nwpXvwxu/g1+egz7n45b4g6mGHDjHtIfV7ziRfdi0J61cZH4nV39s9n8VNw+48r2a15e9UfP8oiU1z2cvNvslKbZu1R6Y39htmxvfF63IYYlAa12ulLoV+BawAq9qrROUUjfZ5y8B/g6EAM/ZDzrljRVdROvycLPy2CWDuX5idz6KS+FgZiEp2YWsT8ri9bXJhPt7MCE6lFE+TfdC0lq3z4TRGtzcYdo/zSBuFitMewRQ5jWY+t3co9Dn/DPbTufh5gy223hzIASTfLxrVXX5dzJnnH4dm16Xf2dTp1z7YrqWslhM1+QOfWquPg/qahpYq/SIMb2XlLXWxYoauoyFw/ZhJQIiTftLQRqMmGcSAUBgN452mknHARNNR4KIoaa+P7KJRGCxmq7NX99jPuPQK00icPMy9evT/mm27xUIXUaZdf3wD3NdxKj5cJa9bSZqornCffQfzOsuo+GKZeZizCl/ZccXLzDM+7hJHA31ngvsYi6y3PQaTPlz3STQHG7ucNlSM/Bjv9+1bB2twKFXFmutlwPL601bUuv5fECGu3Si3uF+/Hlmv+rXhaXlfL3jOCv3pPHdzlQ+Ly3nq+PrKSgt586pvTmrVwcAyisqeW1NMs/+uI9rx0Vx13m9201CKCwtZ+Hbm1lwVg/G9wxtdLmKSo1FNTG4H8CIa2ue1+vai6d/w2fmLTHJfoDKsg9hXjsRuPuZM8zAruYs3a6gpJxnf9zHrVN64udpvwYjKMo0crfWdzHmxprngd0gOLpuMgruYRqbLVazbYubOdsfd0tNIvDvZN6XdcAcgL2CTO+aoG7kBrjDpBizXNexputs5IimYxp6Fax8xPSe8gk1DeO+4aakNOTyusuOv9VU0fS9wIy8W7VfrDYzjElt0VPMA8gJHAAxt9CksH4w819NL3M6WvN31EIyxISow9vdjUtGRHLJiEjS8oq547VVZBeVkltUzrzXNvLgrAFcMDiCm9/azLqkTHp08OF/KxPZdSyXC4ZEMLV/R3w9HPezyi0u48ttxwgoq9vJQWtN/JFcenf05bU1ycTuSefIiSK++dOk6q60GfklPLdyPyXlFRSXVfJ1/DGuHN2V+y+of3mLEwVEmvpm3zBzkAPTvRRMFYVbTU+vFTuP88KqJHp28OXSkfbmuGn/JC8vh/kvrOPBWQPoF+FfZ/Vfbj/KhOjQRtuEmnTFMnPQqn3BYmBXuNh++Y/VZpJB9iHTXTegi+ma6e4D5z5g6tOVMokj+Sf7hXHHatZ11l3mgH2q0oyHr+k55BlgXl/+Vp39Uke/WXDtF6ZkYLE2/zO7CEkEolFhfp78YbAHMTFnkVtcxs1vxXH/p/H844udaDRPzBnMnBGRPL9qPy+sSuKH3WkE++zi5snRXDchCotS7E/P53hOMYM6B1QffLTWJBzNZWNyFgM6BTC6e00f7tLySv774z4UcP3E7gR6u5NfUs5bvxwkp6iMj+JSSMsroZOvoiDoMGsSM+gW4sOOlGxW7klndFQwu4/n0inAk31p+byz4RCRgV6sS8rk/U2HKSgpx9fDjbIKTVSID6+uOcC0gR35Nv44+9Pz8XZ3Y8agjkztH46HmzlwbErO4qEvdnLfjL5MaKKEUV9BSTkZ+SV0C/E5/Z1utZkqkvCBpnulxUaRLYi5i9fw6rUjCal1I6P1SVkAxB08UZMIAruwJsWN9QcO8tnWo/Tt6McHm1LwLtUczCzg1ne2cN34KB6cNYDC0nI83axYTveak3B7wtTaVMV4+JnrXmoP8NdlrDn7d/MwySDD9N7Bv1NNF9vuk8zwGPW7RHoFQtemepjX0nFQzfOq/voNUcpsTzRJEoE4Lf6eNpZeP4ZvE47z3qbD/OGsHtUHxYUxPblpUjSbD53g2R8TeWT5Lt7fdJj8knKO5ZiLpIJ93Lljam+iQ31YHJvImsSaDmKzh3ZiZLcgSis0n287yrbD2QC8tiaZO8/rzdc7jrMhOQurRTGgkz9/Orc3D3+xg3s+3E6wjztfbDuKh5uVq8d25b2Nhymr0Ly7YCz3fLidv31qLqRyt1oYFx3C/ef3o1e4H1prcovLOefJVVz2gumlM6CTP+l5uXy14xgd/T25OSaaEd2CuPntzaTnlXDdaxt47OLBXDIikspKzTsbDrF4ZSIFJeWMjw7l+auHV38mrTULlm4i7uAJvrtjMl2CzUEvt7gMX3e3pg++838wBzClwDuE4+U+bDuazep96UyIDuXRr3fztwv6s/6ASQSbDprhKLILSwn0dmfDAfN6XVImcQdPcM9H25kdbaMw2Cz/5faj3Di5B9OeWs1t5/Ri/lk9Go6jMUqZi7V8Opw8b/b/asZZmvFEw1VUk/4PzrqbLYdOUFKh0Vrz0Bc7mTW0E8O7tkLbhmg2SQTitFksihmDIpgxKKLBeSOjgnnz+tF8m3CcJ1fsoU9HP+46rw+hvu4888O+6oOyn4cbf7+gP+f2C+edDYd4fe0BPttqehYH+7iz+MrhRIf5sOjLXTz0xU6sFsXiK4dz/uCa7VozEunWbwijo4LJLzUXM/l72rhoWCQpJwoZ0CmAf186hBUJqYyMCmJEtyA8bTVVA0opArxsLLpwAP/6dg8PzRrAWb06UFGp+WlfOv/9MZEHPjdXDXvZrHx40zieXLGXuz7Yxg+7U9mfVsCe1DxGdw8m0MvGNwnHWbU3nV2ZFRy0N7ZXJbuHvtjJU5cPYYm95PR/0/pw4+R6w2TX3ZkUl1WwMTmLs0bPZ8V202C/PimL4zklfLLlCG4WxYGMAsL8PEhMy+eHXan84c1NLLl6BBuTzQE//kgO728y13TGZ1Tgbk8cGfml3PD6JnKLy/l485FmJYKcwjLe3nCQBcOvx82ngYO2Umhg9d50JkSH4FYr4eUUlXHJ82t54Hf9iQrx4eLn1zKrh40u/fJ4fW0yqbnFPH/1KdoHXMS+1Dw2JGdx1ZhWHim3EZIIRKubNqAj0wbU7d0yqVcH9qblcTS7iIGdAwjzM70s7pvRl3um9SE9vwR3q4VAb1t14+3SG0azfMdxfD3dmNy77tlnuI+FsT1M3bl/VUMpMKKbOegD9IvwP6mOvL7pAyOYPrBWgrEoYvqEMbl3B3Yey+WXpCz6R/gzMiqYpTeM5okVe3hxdRKDOwfw1OVDuHBoZ8oqNOf8J5b7P43neHYx5dokkB6hPlw8vDP/XrGXIQ+toFKDv6cbyzYeZsGkHic1Un+65QjP/rCPN28Yzetrknn55wN8dPN8lsRuBMpYfyCLxDQzuucHcWbohgWTerDoq13c/cE2KjUsjt1PwtEcRnYLYtPBE3wQl4JSkJRTSd7edCb37sCWQyfYeSyXEB93dh7L5WBmwUnVV4Wl5cQdPMHEnqF14lz6SzL/XrGXrldeygWDO9GQn/ZlcO2rG3j04kFcMbqmC+yqvekkpuWzdN1BRncPRmvYklbBqr3m3guxe9IpLquok7Bd1Qurk/gwLoWIAE/O7nsa9304QzIMtWgTFouib0d/zu4bXp0Eas8L9/ckyMe9zkFHKcX5gyNOSgJtQSnFgE4B3DCxO+OiTcJxs1r484x+7Hl4Bp/dOpGLhkWilMLdzcId5/Ym5UQRnf0sPDN3KEMiA1h04UAWTIrmmrHduGVKTz5ZOJ6/nt+PAxkFbE/JYUXCcV5ancQHmw5zIKOA+z+NJynD1OO/sS4ZgMUrEzlRWEZ0Bx8OZBQQd+gEZ/c1Y+34uFuZO7orNqviRGEZPTr4sO1wNpUabpwcjbvVgtZw9ZhuaCAtr4QJPUOYNbQTnjYLL/7e9NT+Ov74SZ//gc8SuOaVDXywKYWsglKW7zhGZaXmy+2mcffLbcfQWnM02ww/UV5RyWdbj1BSXsHn20zp7ottR6ms1Hy3M5Wi0gpW7jZDa8fuTeeTLWYwukN5lXwYdxgPNwtFZRX8tC+DH3alcjir0DFf7K9EVfXog5/vJCO/hNLyVhhMsglSIhCimdzdTj5/unBoZ9zdLKjUPZw/tDOzh3aunvfwhTWjf/bo4MvfPkvgT+9t5UBGQfV0izLXdtw4qQcvrE7C293KsC4B/Gg/eN44KZp7PtqO1nDHub2xKPD1cMPXw42BnQNIzijgjXmjOefJVZRXVjIuOoRhXQPZfOgEd07tzcdxBykog1FRwVw9thsLzoqma4g3QyIDeG3NAb7cfpRQXw8m9+7AoM4BfLg5BS+blQe/SOA/3+3leG4x142PYvfxPEJ83Fm5J42nvt/Hsz/s460bxpCWV8yd72/jxkk9+Db+OO5uFtYlZfLfHxN56vu9XD22K6v2ptMzzJfEtHwSjuZyweAIvtx+jP3pBdwwsTvvbzrMg58ncCS7iGAfd169bhRDuwSe1neitSYpo4AeoT7tphtzS+UVl5GYns/EnqH8nJjByEXfE+rrzo93xzhsm1IiEKIVWCyKCwZ3wsfW9EEowMvGOX3DOJBRwIVDO7H9wfN49w9jiekTxqILB3Lv9L5cNaYrD88eyNXjTP2wj7uV2cM64evhRri/BwM7+/PS70fy9FwzSNrjlwzmzevH0CXYmyvHdOXsvmH4erhx59TePHLhIIJ83BkYYsXb3crAzgF4u7vRNcQ0Xs8d3ZWCkgr8PW0czirkoS92MmfJOgK9bHx6ywQ83Cx42CyM6BbE62uTsSh45KKBlJRX8uwPpkfQ2+sP8qG9quqF1UnklZRzz7Q+aA1Pfb8Xm1Xx1i+HyCoo5Y9n96RzoBlh9+aYaMK9zf46t184Z/cN40h2EdMHmC7Ilzy/lvlvbKo+O27KOxsOcc6Tq3hz3a///gY7UnLQ2lT7vTZvFHec25uM/FLeWX8aQ2y3kJQIhGhj907vy4huQVw3Pgo3e2+mquongEcuMl0jC0vL8bJZGRwZiIeblYVTognwsp10xts7vKZf/4Ozaoa7GNMjhDH2dpS5fd3584Dh2Kx1z/2uGN21Tj3+5kMneOWnA8wcFEGfjn78cFcM3u5WMvJLmPbUaoZ0CeS8/h3pHOhFRaXmrF6hfLLlCBVac/XYrny25ShuVsW146P4YFMK+9PzefP6MSx4cxMFpeVM7t2BtNwSvtuVSv8If0Z3dGNtqmJEtyA6BXoyqHMA142PIruojJdWJ/HR5hTmLFnLX2b247rxUaScKOLbhONcNaYbXu6mLSG3uIz/rNhrktTyXRzMLCR2TxqPXTK4TtfkM1FUWsEf3tzEuOgQbplyBsOpn4Yt9sQ3ODKAQG93pvQJY2NyFq/+fIBHxjmm/UQSgRBtLCrU57R66ni7u/HU5UMJ9zfXDiyMafkBKMjTclrVLMO7BjH8qpreQMH2az8ig7z58Obx+HvZsFgUb1w/CpvVQlmFrm64vnFSNNMGdKSsohKb1cKiiwaSnlfCuOgQFl00kP1p+QR6u/OHST34wyTz+Wf3tLHo6om4u1noFlKzX0J9PfjzzH4sjOnJne9v5aEvdvLFtqMkpuWTW1zO8h3H+OM5vTiaXcTK3WlkFpTy2rxR/N8H23h1zQH8Pd1Y+PZmli0Yi9aaEF8PDmcVsnJPGpN6d6jupppyopDkjELG9AgmOaOAtLwSxvUIIbOosrrH1fCugby2JpmfEzP4OTGD6A6+TB94iqE+zsC2w9l0D/Uh0Lvmor8bJ/fgmlc2sO6oO1MdsE1JBEK0Y4484DRX7R5YPcNqSiFn9QrFzaLoEuxdfb0EmPaIKrXbTGpzsygCvG0NzgMI8Lbx8rUjeX/TYR75ahfdQny4bFQXHv5yJ/NeMzfwsVkVN07uwZQ+YXx88wRKys1NcmYvXsO5/1l10jqf/n4fo7sH0yfcj/c3HaakvBIfdysFpeZ93UN9OJRZRIXeXud9142PYsvhbG57dwujuwczoJM/3u5ubDl8gmAfd343pFN1D7bM/BK+2H4ML5uFm2N60j20bq+s/en5vLk2mf3pBZSUV9Ap0AtPNyte7lY2JmcR0yeszvITe4baO03kNLqvzoQkAiHEGXn1ulEOXb9SistHdWXWENMgb7UoJvfqwLGcIroEexPu71k9jEhV2wfAsgVjWZ+URZi/Bxn5pfh6WInpE8ZHm1P4fOtR3lp/kKn9wpk9tDOr9qbRO9yPIG933tlwiG5eJfxlznhsVgs/7ErlUFYhf5nZj+yiUp6P3c+6/Zm8tiaZ0opKeob5Epd8go83H6kTd5C3jaKyCt7flEKorzuB3u4ozJ0bktLzcXez0KejPx5uFuIOnqC8QlNQUk5eSTkT613BrpTijetHE+ugUXclEQghzkj9dgdHqWoTAHPAr33Qb8jgyEAGRwaeNH1hTE8WxvSktLyyugdY7YsVLxkRSWxsbHXbS+1qvDA/Tx74nWmHqazUFJVV4OPhRlFpBZsPnaC80lxV7eFmYXjXIHKKyvh4cwrJmQXkFJVVr+e8/uFcP7E7ob4nj5FUVFpR57O2BUkEQgiX1FA34OawWBQ+9gEWvdytDY5D1cHPo+mryBvQ1kkApPuoEEK4PEkEQgjh4iQRCCGEi5NEIIQQLk4SgRBCuDiHJgKl1HSl1B6lVKJS6r4G5iul1LP2+duVUsMbWo8QQgjHcVgiUEpZgcXADKA/cIVSqv7NYWcAveyPBcDzjopHCCFEwxxZIhgNJGqtk7TWpcAyYHa9ZWYDb2rjFyBQKXXy7a+EEEI4jCMvKOsMHK71OgWof2fqhpbpDByrvZBSagGmxACQr5Ta08KYQoGMFr7XkdprXNB+Y5O4mkfiap7fYlyN3vfSkYmgoYHZdQuWQWv9IvDiGQek1Cat9cgzXU9ra69xQfuNTeJqHomreVwtLkdWDaUAXWq9jgSOtmAZIYQQDuTIRLAR6KWU6q6UcgfmAp/XW+Zz4Pf23kNjgRyt9bH6KxJCCOE4Dqsa0lqXK6VuBb4FrMCrWusEpdRN9vlLgOXATCARKATmOSoeuzOuXnKQ9hoXtN/YJK7mkbiax6XiUlqfVCUvhBDChciVxUII4eIkEQghhItzmURwquEu2jCOLkqplUqpXUqpBKXU7fbpDyqljiilttofM50QW7JSaod9+5vs04KVUt8ppfbZ/wadaj2tHFOfWvtkq1IqVyn1J2fsL6XUq0qpNKVUfK1pje4fpdSf7b+3PUqpaW0c1xNKqd32oVs+UUoF2qdHKaWKau23JW0cV6Pfm5P313u1YkpWSm21T2/L/dXYscHxvzGt9W/+gWms3g/0ANyBbUB/J8USAQy3P/cD9mKG4HgQuNvJ+ykZCK037V/Affbn9wGPO/l7PI65MKbN9xcwCRgOxJ9q/9i/022AB9Dd/vuztmFc5wFu9ueP14orqvZyTthfDX5vzt5f9eY/CfzdCfursWODw39jrlIiOJ3hLtqE1vqY1nqz/XkesAtzNXV7NRt4w/78DeBC54XCOcB+rfVBZ2xca70ayKo3ubH9MxtYprUu0VofwPSMG91WcWmtV2ity+0vf8Fco9OmGtlfjXHq/qqilFLAZcC7jth2U5o4Njj8N+YqiaCxoSycSikVBQwD1tsn3Wovyr/a1lUwdhpYoZSKsw/rARCu7dd22P+GOSGuKnOp+w/q7P0Fje+f9vSbux74utbr7kqpLUqpVUqps5wQT0PfW3vZX2cBqVrrfbWmtfn+qndscPhvzFUSwWkNZdGWlFK+wEfAn7TWuZiRV6OBoZixlp50QlgTtNbDMaPC3qKUmuSEGBqkzEWJs4AP7JPaw/5qSrv4zSml/gqUA2/bJx0DumqthwF3Au8opfzbMKTGvrd2sb+AK6h7stHm+6uBY0OjizYwrUX7zFUSQbsaykIpZcN80W9rrT8G0Fqnaq0rtNaVwEs4qFjcFK31UfvfNOATewypyj4irP1vWlvHZTcD2Ky1TrXH6PT9ZdfY/nH6b04pdS1wAXCVtlcq26sRMu3P4zD1yr3bKqYmvrf2sL/cgIuB96qmtfX+aujYQBv8xlwlEZzOcBdtwl4H+QqwS2v9n1rTaw+/fREQX/+9Do7LRynlV/Uc09gYj9lP19oXuxb4rC3jqqXOmZqz91ctje2fz4G5SikPpVR3zD03NrRVUEqp6cC9wCytdWGt6R2UuVcISqke9riS2jCuxr43p+4vu3OB3VrrlKoJbbm/Gjs20Ba/sbZoDW8PD8xQFnsxGf2vToxjIqb4th3Yan/MBJYCO+zTPwci2jiuHpgeCNuAhKp9BIQAPwD77H+DnbDPvIFMIKDWtDbfX5hEdAwow5yN3dDU/gH+av+97QFmtHFciZj646rf2BL7spfYv99twGbgd20cV6PfmzP3l33668BN9ZZty/3V2LHB4b8xGWJCCCFcnKtUDQkhhGiEJAIhhHBxkgiEEMLFSSIQQggXJ4lACCFcnCQCIepRSlWouiOettpotfbRLJ11zYMQDXLYrSqF+BUr0loPdXYQQrQVKREIcZrs49Q/rpTaYH/0tE/vppT6wT6Q2g9Kqa726eHK3Atgm/0x3r4qq1LqJfuY8yuUUl5O+1BCIIlAiIZ41asaurzWvFyt9Wjgf8DT9mn/A97UWg/GDO72rH36s8AqrfUQzPj3CfbpvYDFWusBQDbm6lUhnEauLBaiHqVUvtbat4HpycDZWusk++Bgx7XWIUqpDMxQCWX26ce01qFKqXQgUmtdUmsdUcB3Wute9tf3Ajat9aI2+GhCNEhKBEI0j27keWPLNKSk1vMKpK1OOJkkAiGa5/Jaf9fZn6/FjGgLcBXws/35D8DNAEopaxuP+y/EaZMzESFO5qXsNy+3+0ZrXdWF1EMptR5zEnWFfdptwKtKqf8D0oF59um3Ay8qpW7AnPnfjBn1Uoh2RdoIhDhN9jaCkVrrDGfHIkRrkqohIYRwcVIiEEIIFyclAiGEcHGSCIQQwsVJIhBCCBcniUAIIVycJAIhhHBx/w9XkZdD0n+UjgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "metrics = pd.read_csv(f\"{trainer.logger.log_dir}/metrics.csv\")\n",
    "\n",
    "aggreg_metrics = []\n",
    "agg_col = \"epoch\"\n",
    "for i, dfg in metrics.groupby(agg_col):\n",
    "    agg = dict(dfg.mean())\n",
    "    agg[agg_col] = i\n",
    "    aggreg_metrics.append(agg)\n",
    "\n",
    "df_metrics = pd.DataFrame(aggreg_metrics)\n",
    "df_metrics[[\"train_loss\", \"valid_loss\"]].plot(\n",
    "    grid=True, legend=True, xlabel='Epoch', ylabel='Loss')\n",
    "df_metrics[[\"train_mae\", \"valid_mae\"]].plot(\n",
    "    grid=True, legend=True, xlabel='Epoch', ylabel='MAE')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "c113e6b0",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Restoring states from the checkpoint path at logs/mlp-coral-cement/version_0/checkpoints/epoch=114-step=2529.ckpt\n",
      "LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n",
      "Loaded model weights from checkpoint at logs/mlp-coral-cement/version_0/checkpoints/epoch=114-step=2529.ckpt\n",
      "/home/jovyan/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:258: PossibleUserWarning: The dataloader, test_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 4 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.\n",
      "  rank_zero_warn(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "651ce18e66064ff996bf3f1a691d2959",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Testing: 0it [00:00, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃<span style=\"font-weight: bold\">        Test metric        </span>┃<span style=\"font-weight: bold\">       DataLoader 0        </span>┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│<span style=\"color: #008080; text-decoration-color: #008080\">         test_mae          </span>│<span style=\"color: #800080; text-decoration-color: #800080\">           0.25            </span>│\n",
       "└───────────────────────────┴───────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃\u001b[1m \u001b[0m\u001b[1m       Test metric       \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m      DataLoader 0       \u001b[0m\u001b[1m \u001b[0m┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│\u001b[36m \u001b[0m\u001b[36m        test_mae         \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m          0.25           \u001b[0m\u001b[35m \u001b[0m│\n",
       "└───────────────────────────┴───────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[{'test_mae': 0.25}]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainer.test(model=lightning_model, datamodule=data_module, ckpt_path='best')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}