Predecir_el_precio_de_la_vivienda_Solución.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.5.4"
    },
    "colab": {
      "name": "Predecir el precio de la vivienda - Solución.ipynb",
      "provenance": [],
      "collapsed_sections": []
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kCF7FB8dWoaS",
        "colab_type": "text"
      },
      "source": [
        "# Predicción del precio de casas de Boston\n",
        "\n",
        "Intentaremos predecir el precio mediano de las viviendas en un barrio de Boston a mediados de la década de 1970, partiendo de una variedad de datos sobre el el barrio en el momento, como la tasa de criminalidad, la tasa de impuesto a la propiedad, etc.\n",
        "\n",
        "El conjunto de datos que usaremos es pequeña, 506 muestras en total. Debemos estar atentos, ya que hay valores discretos y continuos (como la criminalidad)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zfrSOiiiDFKl",
        "colab_type": "text"
      },
      "source": [
        "Los datos contienen 13 características a partir de las cuales deberíamos ser capaces de poder predecir el precio de cada vivienda. Las características son:\n",
        "\n",
        "1. Ratio de crimen per cápita.\n",
        "\n",
        "2. Proporción de tierra residencial dividida en zonas de más de 25.000 pies cuadrados.\n",
        "3. Proporción de acres de negocios no minoristas por ciudad.\n",
        "4. Variable ficticia de Charles River (= 1 si limita con el río; 0 en caso contrario).\n",
        "5. Concentración de óxidos nítricos (partes por 10 millones).\n",
        "6. Número promedio de habitaciones por vivienda.\n",
        "7. Proporción de unidades ocupadas por sus propietarios y construídas antes de 1940.\n",
        "8. Distancias ponderadas a cinco centros de empleo de Boston.\n",
        "9. Índice de accesibilidad a carreteras radiales.\n",
        "10. Tasa de impuesto a la propiedad de valor total por $10,000.\n",
        "11. Relación alumno-profesor por localidad.\n",
        "12. 1000 * (Bk - 0.63) ** 2 donde Bk es la proporción de personas negras por pueblo.\n",
        "13. % de población de un estatus bajo."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KmKVDNcfj9XK",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "! pip install analytics-zoo\n",
        "! sudo apt install openjdk-8-jdk\n",
        "! echo \"2\" | sudo update-alternatives --config java"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M86KWgPnlMmF",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        },
        "outputId": "30af2f93-001b-44ba-9ef5-ed47b1b2dd6c"
      },
      "source": [
        "from zoo.common.nncontext import *\n",
        "sc = init_nncontext(init_spark_conf().setMaster(\"local[4]\"))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Prepending /usr/local/lib/python3.6/dist-packages/bigdl/share/conf/spark-bigdl.conf to sys.path\n",
            "Adding /usr/local/lib/python3.6/dist-packages/zoo/share/lib/analytics-zoo-bigdl_0.10.0-spark_2.4.3-0.8.1-jar-with-dependencies.jar to BIGDL_JARS\n",
            "Prepending /usr/local/lib/python3.6/dist-packages/zoo/share/conf/spark-analytics-zoo.conf to sys.path\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "q1fTd4IuWoaZ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "40882e14-2048-4dac-a12f-4112aa8efd30"
      },
      "source": [
        "from zoo.pipeline.api.keras.datasets import boston_housing\n",
        "(train_data, train_targets), (test_data, test_targets) =  boston_housing.load_data()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://s3.amazonaws.com/keras-datasets/boston_housing.npz\n",
            "24576/57026 [===========>..................] - ETA: 0sSuccessfully downloaded boston_housing.npz 57026 bytes.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "T-szVO5mYyr9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "921f1505-ac46-41ba-9bc6-964f31287855"
      },
      "source": [
        "train_data.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(404, 13)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ozUvTVrCEXVs",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "9eae59a3-6bc1-4fa0-ecf2-137623c4b261"
      },
      "source": [
        "test_data.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(102, 13)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vk2TIF6wEY83",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 675
        },
        "outputId": "2ac2445f-5a39-4dc6-d286-7472474470c9"
      },
      "source": [
        "train_targets"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([22.6, 50. , 23. ,  8.3, 21.2, 19.9, 20.6, 18.7, 16.1, 18.6,  8.8,\n",
              "       17.2, 14.9, 10.5, 50. , 29. , 23. , 33.3, 29.4, 21. , 23.8, 19.1,\n",
              "       20.4, 29.1, 19.3, 23.1, 19.6, 19.4, 38.7, 18.7, 14.6, 20. , 20.5,\n",
              "       20.1, 23.6, 16.8,  5.6, 50. , 14.5, 13.3, 23.9, 20. , 19.8, 13.8,\n",
              "       16.5, 21.6, 20.3, 17. , 11.8, 27.5, 15.6, 23.1, 24.3, 42.8, 15.6,\n",
              "       21.7, 17.1, 17.2, 15. , 21.7, 18.6, 21. , 33.1, 31.5, 20.1, 29.8,\n",
              "       15.2, 15. , 27.5, 22.6, 20. , 21.4, 23.5, 31.2, 23.7,  7.4, 48.3,\n",
              "       24.4, 22.6, 18.3, 23.3, 17.1, 27.9, 44.8, 50. , 23. , 21.4, 10.2,\n",
              "       23.3, 23.2, 18.9, 13.4, 21.9, 24.8, 11.9, 24.3, 13.8, 24.7, 14.1,\n",
              "       18.7, 28.1, 19.8, 26.7, 21.7, 22. , 22.9, 10.4, 21.9, 20.6, 26.4,\n",
              "       41.3, 17.2, 27.1, 20.4, 16.5, 24.4,  8.4, 23. ,  9.7, 50. , 30.5,\n",
              "       12.3, 19.4, 21.2, 20.3, 18.8, 33.4, 18.5, 19.6, 33.2, 13.1,  7.5,\n",
              "       13.6, 17.4,  8.4, 35.4, 24. , 13.4, 26.2,  7.2, 13.1, 24.5, 37.2,\n",
              "       25. , 24.1, 16.6, 32.9, 36.2, 11. ,  7.2, 22.8, 28.7, 14.4, 24.4,\n",
              "       18.1, 22.5, 20.5, 15.2, 17.4, 13.6,  8.7, 18.2, 35.4, 31.7, 33. ,\n",
              "       22.2, 20.4, 23.9, 25. , 12.7, 29.1, 12. , 17.7, 27. , 20.6, 10.2,\n",
              "       17.5, 19.7, 29.8, 20.5, 14.9, 10.9, 19.5, 22.7, 19.5, 24.6, 25. ,\n",
              "       24.5, 50. , 14.3, 11.8, 31. , 28.7, 16.2, 43.5, 25. , 22. , 19.9,\n",
              "       22.1, 46. , 22.9, 20.2, 43.1, 34.6, 13.8, 24.3, 21.5, 24.4, 21.2,\n",
              "       23.8, 26.6, 25.1,  9.6, 19.4, 19.4,  9.5, 14. , 26.5, 13.8, 34.7,\n",
              "       16.3, 21.7, 17.5, 15.6, 20.9, 21.7, 12.7, 18.5, 23.7, 19.3, 12.7,\n",
              "       21.6, 23.2, 29.6, 21.2, 23.8, 17.1, 22. , 36.5, 18.8, 21.9, 23.1,\n",
              "       20.2, 17.4, 37. , 24.1, 36.2, 15.7, 32.2, 13.5, 17.9, 13.3, 11.7,\n",
              "       41.7, 18.4, 13.1, 25. , 21.2, 16. , 34.9, 25.2, 24.8, 21.5, 23.4,\n",
              "       18.9, 10.8, 21. , 27.5, 17.5, 13.5, 28.7, 14.8, 19.1, 28.6, 13.1,\n",
              "       19. , 11.3, 13.3, 22.4, 20.1, 18.2, 22.9, 20.6, 25. , 12.8, 34.9,\n",
              "       23.7, 50. , 29. , 30.1, 22. , 15.6, 23.3, 30.1, 14.3, 22.8, 50. ,\n",
              "       20.8,  6.3, 34.9, 32.4, 19.9, 20.3, 17.8, 23.1, 20.4, 23.2,  7. ,\n",
              "       16.8, 46.7, 50. , 22.9, 23.9, 21.4, 21.7, 15.4, 15.3, 23.1, 23.9,\n",
              "       19.4, 11.9, 17.8, 31.5, 33.8, 20.8, 19.8, 22.4,  5. , 24.5, 19.4,\n",
              "       15.1, 18.2, 19.3, 27.1, 20.7, 37.6, 11.7, 33.4, 30.1, 21.4, 45.4,\n",
              "       20.1, 20.8, 26.4, 10.4, 21.8, 32. , 21.7, 18.4, 37.9, 17.8, 28. ,\n",
              "       28.2, 36. , 18.9, 15. , 22.5, 30.7, 20. , 19.1, 23.3, 26.6, 21.1,\n",
              "       19.7, 20. , 12.1,  7.2, 14.2, 17.3, 27.5, 22.2, 10.9, 19.2, 32. ,\n",
              "       14.5, 24.7, 12.6, 24. , 24.1, 50. , 16.1, 43.8, 26.6, 36.1, 21.8,\n",
              "       29.9, 50. , 44. , 20.6, 19.6, 28.4, 19.1, 22.3, 20.9, 28.4, 14.4,\n",
              "       32.7, 13.8,  8.5, 22.5, 35.1, 31.6, 17.8, 15.6])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TTl5mwsEWoa0",
        "colab_type": "text"
      },
      "source": [
        "## Preparamos los datos\n",
        "\n",
        "Estandarizamos los datos para centrarlos alrededor del 0"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KgYI-BRYEiQG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "mean = train_data.mean(axis=0)\n",
        "train_data -= mean\n",
        "std = train_data.std(axis=0)\n",
        "train_data /= std\n",
        "\n",
        "test_data -= mean\n",
        "test_data /= std"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZH4Ahm456GjQ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 479
        },
        "outputId": "183604e3-ac5c-466b-f6d5-f6b39593e63d"
      },
      "source": [
        "print(train_data)\n",
        "print(test_data)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[-0.40241083 -0.48917404 -1.17629455 ... -0.7163786   0.20831263\n",
            "  -0.75004698]\n",
            " [ 0.61863723 -0.48917404  1.04403852 ...  0.82605672  0.1090616\n",
            "  -0.44090876]\n",
            " [-0.39717361 -0.48917404  0.41499396 ... -0.95008092  0.41130618\n",
            "  -0.30398452]\n",
            " ...\n",
            " [-0.36395996 -0.48917404 -0.73402093 ... -0.48267628  0.17402392\n",
            "  -0.8883828 ]\n",
            " [-0.15021417 -0.48917404  1.26517533 ... -1.74466881  0.00871508\n",
            "   2.21146895]\n",
            " [ 0.04657554 -0.48917404  1.26517533 ... -1.74466881  0.44592354\n",
            "   1.94326684]]\n",
            "[[-0.40573743 -0.48917404 -0.88493186 ...  0.82605672  0.36376209\n",
            "  -0.83333079]\n",
            " [-0.40256235 -0.48917404 -1.29283962 ... -0.29571442  0.43124403\n",
            "  -0.718992  ]\n",
            " [ 0.59085911 -0.48917404  1.04403852 ...  0.82605672  0.23591887\n",
            "   0.69824442]\n",
            " ...\n",
            " [-0.39308165 -0.48917404 -0.16474307 ... -0.29571442  0.41974142\n",
            "  -0.33221632]\n",
            " [-0.38435962 -0.48917404 -0.62793503 ... -0.24897396  0.40035132\n",
            "   0.86763522]\n",
            " [-0.3944007  -0.48917404 -1.05526697 ... -0.85659999  0.44592354\n",
            "   0.28747171]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T_wBZssQWoa5",
        "colab_type": "text"
      },
      "source": [
        "**Tengamos en cuenta que las cantidades que usamos para estandarizar los datos de test se han calculado utilizando los datos de entrenamiento. Nunca debemos usar en nuestro\n",
        "flujo de trabajo cualquier cantidad calculada en los datos de prueba, incluso para algo tan simple como la estandarización de datos.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mMcvvu_eWoa7",
        "colab_type": "text"
      },
      "source": [
        "## Creamos nuestra red\n",
        "\n",
        "Debido a que tenemos pocas muestras, usaremos una red pequeña con 2 capas ocultas de 6 neuronas cada una. En general, cuanto menos muestras de entrenamiento se tiene, más fácil es hacer overfitting. Para evitarlo, utilizar una red pequeña suele ser una buena opción."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NiUr2P-wEjpt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from zoo.pipeline.api.keras import models\n",
        "from zoo.pipeline.api.keras import layers\n",
        "\n",
        "def build_model():\n",
        "    # Because we will need to instantiate\n",
        "    # the same model multiple times,\n",
        "    # we use a function to construct it.\n",
        "    model = models.Sequential()\n",
        "    model.add(layers.Dense(64, activation='relu',\n",
        "                           input_shape=(train_data.shape[1],)))\n",
        "    model.add(layers.Dense(64, activation='relu'))\n",
        "    model.add(layers.Dense(1))\n",
        "    model.compile(optimizer='rmsprop', loss='mse', metrics=['mae'])\n",
        "    return model"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YzhiErAGWoa_",
        "colab_type": "text"
      },
      "source": [
        "Nuestra red termina con una sola unidad y sin activación (es decir, será una capa lineal).\n",
        "Esta es una configuración típica para la regresión escalar (es decir, regresión en la que estamos tratando de predecir un solo valor continuo). Aplicar una función de activación restringiría el rango que puede tomar la salida; por ejemplo si aplicamos una función de activación sigmoide a nuestra última capa, la red solo podía aprender a predecir valores entre 0 y 1. Aquí, porque la última capa es puramente lineal, la red es libre de aprender a predecir valores en cualquier rango.\n",
        "\n",
        "Estamos compilando la red con la función de pérdida mse - Error cuadrático medio, el cuadrado de la diferencia entre predicciones y valores reales, una función de pérdida ampliamente utilizada para problemas de regresión.\n",
        "\n",
        "También estamos trabajando con una nueva métrica durante el entrenamiento: mae. Esto significa Error absoluto medio. Es simplemente el valor absoluto de la\n",
        "diferencia entre las predicciones y los objetivos. Por ejemplo, un MAE de 0.5 en este problema significaría que nuestras predicciones se desvían $500 de media."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "m-ott6NlWobA",
        "colab_type": "text"
      },
      "source": [
        "## Validando nuestro enfoque usando K-fold cross-validation\n",
        "\n",
        "Para evaluar nuestra red mientras seguimos ajustando sus parámetros (como el número de epochs utilizados para el entrenamiento), simplemente podríamos dividir los datos en un conjunto de entrenamiento y un conjunto de validación. Sin embargo, debido a que tenemos tan pocos datos, el conjunto de validación terminaría siendo muy pequeño (por ejemplo, alrededor de 100 ejemplos). Una consecuencia de esto es que nuestros puntajes de validación pueden cambiar mucho dependiendo de qué muestras elegimos usar para la validación y cuáles elegimos para el entrenamiento, es decir, las etiquetas de entrenamiento pueden tener una alta varianza con respecto a las etiquetas de validación. Esto nos daría un modelo entrenado poco fiable.\n",
        "\n",
        "La mejor práctica en tales situaciones es usar K-fold cross-validation. Consiste en dividir los datos disponibles en particiones K (típicamente K = 4 o 5), luego instanciar K modelos idénticos y entrenar a cada uno en particiones K-1. El score de validación para el modelo utilizado sería el promedio de K scores obtenidos."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rht55dq8Ewsb",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 586
        },
        "outputId": "46a47e7b-50cc-49d4-a85f-7d621f6f941a"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "k = 4\n",
        "num_val_samples = len(train_data) // k\n",
        "num_nb_epoch = 50\n",
        "all_scores = []\n",
        "for i in range(k):\n",
        "    print('processing fold #', i)\n",
        "    # Prepare the validation data: data from partition # k\n",
        "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
        "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
        "\n",
        "    # Prepare the training data: data from all other partitions\n",
        "    partial_train_data = np.concatenate(\n",
        "        [train_data[:i * num_val_samples],\n",
        "         train_data[(i + 1) * num_val_samples:]],\n",
        "        axis=0)\n",
        "    partial_train_targets = np.concatenate(\n",
        "        [train_targets[:i * num_val_samples],\n",
        "         train_targets[(i + 1) * num_val_samples:]],\n",
        "        axis=0)\n",
        "\n",
        "    # Build the model (already compiled)\n",
        "    model = build_model()\n",
        "    # Train the model\n",
        "    model.fit(partial_train_data, partial_train_targets,\n",
        "              nb_epoch=num_nb_epoch, batch_size=16) # verbose=0 si no queremos logs\n",
        "\n",
        "    # Evaluate the model on the validation data\n",
        "    val_mse, val_mae = model.evaluate(val_data, val_targets) # verbose=0 si no queremos logs\n",
        "    all_scores.append(val_mae)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "processing fold # 0\n",
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n",
            "processing fold # 1\n",
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n",
            "processing fold # 2\n",
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n",
            "processing fold # 3\n",
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YVs0LyzlE3IF",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "f43639a8-2245-43c2-b5be-4e1b3b0afdf3"
      },
      "source": [
        "all_scores"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[3.3052000999450684, 2.496307849884033, 2.425664186477661, 2.7723302841186523]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ODCj26zwE5jU",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "43c8a99f-eac6-4060-a5cb-33cd1ffcfc79"
      },
      "source": [
        "np.mean(all_scores)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2.7498756051063538"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gm8A4fsFWobM",
        "colab_type": "text"
      },
      "source": [
        "Como puede observar, las diferentes ejecuciones muestran scores de validación bastante diferentes, de 3.3 a 2.4. Su media (2.7) es un valor más fiable que cualquiera de los scores de la lista, esa es la ventaja de K-fold cross validation. En este caso, estamos desviándonos \\\\$2.700 de media, que sigue siendo significativo considerando que los precios oscilan entre 10.000 y 50.000$ de media.\n",
        "\n",
        "Entrenemos la red un poco más: 500 epochs. Para mantener un registro de cómo funcionó el modelo en cada epoch, modificaremos nuestro ciclo de entrenamiento para guardar el registro de score de validación por epoch."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1Z_DvE4oE6yl",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 554
        },
        "outputId": "08b2b31a-ccf6-40c4-c827-56c4f6596686"
      },
      "source": [
        "num_epochs = 500\n",
        "all_mae_histories = []\n",
        "for i in range(k):\n",
        "    print('processing fold #', i)\n",
        "    # Prepare the validation data: data from partition # k\n",
        "    val_data = train_data[i * num_val_samples: (i + 1) * num_val_samples]\n",
        "    val_targets = train_targets[i * num_val_samples: (i + 1) * num_val_samples]\n",
        "\n",
        "    # Prepare the training data: data from all other partitions\n",
        "    partial_train_data = np.concatenate(\n",
        "        [train_data[:i * num_val_samples],\n",
        "         train_data[(i + 1) * num_val_samples:]],\n",
        "        axis=0)\n",
        "    partial_train_targets = np.concatenate(\n",
        "        [train_targets[:i * num_val_samples],\n",
        "         train_targets[(i + 1) * num_val_samples:]],\n",
        "        axis=0)\n",
        "\n",
        "    # Build the model (already compiled)\n",
        "    model = build_model()\n",
        "    # Train the model (in silent mode, verbose=0)\n",
        "    import time\n",
        "    dir_name = '3-7 ' + str(time.ctime())\n",
        "    model.set_tensorboard('./', dir_name)\n",
        "    history = model.fit(partial_train_data, partial_train_targets,\n",
        "                        validation_data=(val_data, val_targets),\n",
        "                        nb_epoch=num_epochs, batch_size=16)\n",
        "    \n",
        "    #mae_history = history.history['val_mean_absolute_error']\n",
        "    mae_history = model.get_validation_summary(\"Loss\")\n",
        "    all_mae_histories.append(mae_history)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "processing fold # 0\n",
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n",
            "processing fold # 1\n",
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n",
            "processing fold # 2\n",
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n",
            "processing fold # 3\n",
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mdmc_JVCWobQ",
        "colab_type": "text"
      },
      "source": [
        "Calculamos el MAE de cada epoch."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KBdsPXFjE-O6",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "all_mae_histories = np.array(all_mae_histories)\n",
        "average_mae_history = np.mean(all_mae_histories, axis=0)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NiY23M6IdGUA",
        "colab_type": "text"
      },
      "source": [
        "### Mostramos resultados"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SWrBYhIBFBFb",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 279
        },
        "outputId": "aaa88a14-64fb-41c5-9eeb-465eaed689df"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "plt.plot(average_mae_history[:,0],average_mae_history[:,1])\n",
        "plt.xlabel('Steps')\n",
        "plt.ylabel('Validation MAE')\n",
        "plt.ylim((14, 25))\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2dd5hcdfX/32fq9t1sSS+bhCSQQhIS\nQmICJNRQFFRE+UkVv0FBpCmCgiAIKCAIiopSBQRpAtJDDCWUQHrvvW+y2Wyfen5/3DJ37tw7M7uZ\nmS33vJ5nn525c8vnTnnfc8/nFGJmCIIgCM7B1dEDEARBEHKLCL8gCILDEOEXBEFwGCL8giAIDkOE\nXxAEwWF4OnoA6VBZWcnV1dUZ2dfOuhbUt4RwVJ+SjOxPEAShs7Jw4cL9zFxlXt4lhL+6uhoLFizI\nyL5ueW053l6+BwtuPTUj+xMEQeisENFWq+WOc/V4XC6EItGOHoYgCEKH4Tjh97oJ4YgkrQmC4Fwc\nJ/wetwvhqFj8giA4F8cJv9dFCEUYUqpCEASn4jjh97iVU45ERfgFQXAmDhR+AgCERfgFQXAojhN+\nr0s5ZYnsEQTBqThO+HWLXyJ7BEFwKM4Tfpe4egRBcDbOE351cldCOgVBcCrOE36XuHoEQXA2jhN+\nr1smdwVBcDaOE34J5xQEwek4T/glnFMQBIfjOOH3SjinIAgOJ2vCT0QDiGguEa0iopVEdI3p9RuI\niImoMltjsEKiegRBcDrZbMQSBnADMy8iomIAC4loNjOvIqIBAE4DsC2Lx7fEq0b1hMTiFwTBoWTN\n4mfm3cy8SH3cAGA1gH7qyw8CuBFAztXXLeGcgiA4nJz4+ImoGsB4APOJ6BwAO5l5aYptZhHRAiJa\nUFNTk7GxiKtHEASnk3XhJ6IiAK8AuBaK++eXAH6dajtm/jszT2TmiVVVCb2C241M7gqC4HSyKvxE\n5IUi+s8x86sAhgIYDGApEW0B0B/AIiLqnc1xGNHCOcXiFwTBqWRtcpeICMDjAFYz8wMAwMzLAfQ0\nrLMFwERm3p+tcZjRLH6Z3BUEwalk0+KfCuAiACcR0RL178wsHi8txMcvCILTyZrFz8zzAFCKdaqz\ndXw7PBLOKQiCw3Fg5q5q8YvwC4LgUBwn/FqRtoi4egRBcCjOE35x9QiC4HCcJ/wyuSsIgsNxnvBL\nz11BEByOc4VfXD2CIDgUxwl/rEibuHoEQXAmjhN+IoLXTeLqEQTBsThO+AHF6hfhFwTBqThS+L0u\nl/TcFQTBsThS+D1uQkQsfkEQHIojhd/tckkClyAIjsWRwu91k0T1CILgWBwp/OLqEQTByThT+F0u\nhET4BUFwKA4VfpLqnIIgOBZnCr9bJncFQXAuzhR+l0zuCoLgXJwp/FKyQRAEB+NI4fe6XFKdUxAE\nx+JI4VcsfnH1CILgTBwp/FKkTRAEJ+NI4fe6xdUjCIJzcaTwe1wk1TkFQXAszhR+KdkgCIKDcabw\nu1zi4xcEwbE4U/jd4uoRBMG5OFP4XeLqEQTBuThT+KVWjyAIDsaRwu91SQKXIAjOxZHC73a5EBGL\nXxAEh5I14SeiAUQ0l4hWEdFKIrpGXX4fEa0homVE9B8iKsvWGOzwugkhsfgFQXAo2bT4wwBuYOaR\nACYDuIqIRgKYDWA0Mx8NYB2Am7M4Bks8bkJrKIoPVu3N9aEFQRA6nKwJPzPvZuZF6uMGAKsB9GPm\n95k5rK72BYD+2RqDHTsOtgAArnh2Ya4PLQiC0OHkxMdPRNUAxgOYb3rpBwDesdlmFhEtIKIFNTU1\nGR1P75I8AMDY/qUZ3a8gCEJXIOvCT0RFAF4BcC0z1xuW/wqKO+g5q+2Y+e/MPJGZJ1ZVVWV0TNed\nOhzFeR6UF/ozul9BEISugCebOyciLxTRf46ZXzUsvxTA2QBOZuach9fked0YUlWEoGTvCoLgQLIm\n/EREAB4HsJqZHzAsnwngRgAnMnNzto6fCr/bhWA40lGHFwRB6DCyafFPBXARgOVEtERd9ksADwPw\nA5itXBvwBTP/KIvjsMTrUSJ7BEEQnEbWhJ+Z5wEgi5feztYx24LP7UJ9Szj1ioIgCN0MR2buAoDP\n40IwLBa/IAjOw7HC73W7pDSzIAiOxLHC7/O4EBCLXxAEB+Jc4ReLXxAEh+Jc4fe4JI5fEARH4lzh\nd8vkriAIzsSxwu/1iKtHEARn4ljh96ntF6PSe1cQBIdhK/xEdKPh8XdMr92dzUHlAp9HOXVpyCII\ngtNIZvF/z/DY3CxlZhbGklN8buXUxc8vCILTSCb8ZPPY6nmXQ7P4RfgFQXAayYSfbR5bPe9yeFWL\nPyRN1wVBcBjJirSNJaJ6KNZ9vvoY6vO8rI8sy4jFLwiCU7EVfmZ253IgucbrVrxVksQlCILTaFM4\nJxEVEtGFRPRWtgaUK/xi8QuC4FBSCj8R+Yjom0T0EoDdAE4G8LesjyzL6K4esfgFQXAYtq4eIjoN\nwAUATgMwF8A/ARzLzJflaGxZxedWPFli8QuC4DSSWfzvAhgCYBozX8jM/wXQbVQy36cIf3NQunAJ\nguAskkX1HAMliesDItoE4AUA3WbCt9CvCb80XBcEwVnYWvzMvISZb2LmoQBuAzAOgJeI3iGiWTkb\nYZYo9CnXvKaAWPyCIDiLtKJ6mPkzZr4aQH8ADwKYnNVR5YBCvwi/IAjOJNnk7jE2L+0H8OfsDCd3\nFKg+/iZx9QiC4DCS+fgXAFgBReiB+Po8DOCkbA0qF/g9LnhcJJO7giA4jmTCfz2A8wC0QJnY/Q8z\nN+ZkVDmAiFDgc6MpIBa/IAjOItnk7h+ZeRqAqwEMADCHiF4konE5G12WKfR7xMcvCILjSDm5y8yb\nALwO4H0AkwAMz/agckWBzy3hnIIgOI5kk7tDoMTxnwNgOxR3z93M3JKjsWWdIr8HTeLjFwTBYSTz\n8W8AsAyKtV8PYCCAHxMpc7zM/EDWR5dlCnzi6hEEwXkkE/47EGu4UpSDseScQr8bu+pCHT0MQRCE\nnJKsHv/tORxHh1Ca78PynYc6ehiCIAg5pU31+NsCEQ0gorlEtIqIVhLRNeryciKaTUTr1f89sjWG\nVAypKsTe+gAaWsXqFwTBOWRN+AGEAdzAzCOhlHi4iohGArgJwBxmHgZgjvq8Qziip+LB2ljT1FFD\nEARByDlZE35m3s3Mi9THDQBWA+gHJUroaXW1pwGcm60xpEIT/g37uk1emiAIQkqSTe4CAIjID+Db\nAKqN6zPzHekehIiqAYwHMB9AL2berb60B0Avm21mAZgFAAMHDkz3UG2iT6nSM35fQ2tW9i8IgtAZ\nScfifx2KlR4G0GT4SwsiKgLwCoBrmbne+BozM2KRQzC99ndmnsjME6uqqtI9XJvwuZXTD0cshyAI\ngtAtSWnxA+jPzDPbs3Mi8kIR/eeY+VV18V4i6sPMu4moD4B97dl3JnC7CERASPruCoLgINKx+D8j\nojFt3TEpmV6PA1htSvZ6A8Al6uNLoNxRdAhEBK/bhZBY/IIgOIh0LP5pAC4los0AAlDKMzMzH51i\nu6kALgKwnIiWqMt+CeB3AF4kossBbAVwfrtGniG8LhKLXxAER5GO8J/Rnh0z8zzE1/A3cnJ79pkN\nvB6XCL8gCI4ineqcWwGUAfi6+lemLusWKK4eEX5BEJxDSuFXM26fA9BT/XuWiK7O9sByhU98/IIg\nOIx0XD2XAziOmZsAgIh+D+BzAH/K5sByhdctPn5BEJxFOlE9BMDYrSQCe999l8Mjrh5BEBxGOhb/\nkwDmE9F/1OfnQgnT7BZ43S4Ew+LqEQTBOaQUfmZ+gIg+hBLWCQCXMfPirI4qh/jE1SMIgsNI1nqx\nhJnriagcwBb1T3utnJlrsz+87ON1uxCOivALguAckln8/wJwNoCFiK+nQ+rzIVkcV87wuAkhcfUI\nguAgknXgOlv9Pzh3w8k9XrcLDSHpuysIgnNIJ45/TjrLuio+ieoRBMFhJPPx5wEoAFCptkfUQjhL\noDRU6RZ43S4pyywIgqNI5uO/AsC1APpC8fNrwl8P4M9ZHlfOkFo9giA4jWQ+/ocAPEREVzNzt8jS\ntcLrIgRF+AVBcBDpxPH/iYhGAxgJIM+w/J/ZHFiukCJtgiA4jXR67t4GYDoU4X8bSpnmeQC6h/B7\nSHz8giA4inRq9ZwHpX7+Hma+DMBYAKVZHVUO8bpd4uoRBMFRpCP8LcwcBRAmohIoPXIHZHdYuUPC\nOQVB6Aj+8uEGPP3Zlg45djrCv4CIygD8A0p0zyIoZZm7BR43oTUURWsoknplQRAEFWZG9U1v4eE5\n69u87fbaZtz77lrc9sbKLIwsNel04LqSmeuY+W8ATgVwiery6RZ43cpbcP6j3eZaJghCDgiEFU/B\nA7PXtXnbLzYdAADke90ZHVO6JEvgOibZa8y8KDtDyi1NAaVcw7Idhzp4JIIgdCUOZ26wVb1o+L3p\nOF0yT7Konj+o//MATASwFEoS19EAFgCYkt2h5YblO2OC/++vtuG4wRWorizswBEJgnA4HGwKwudx\nodCfTruR9hMItV/4A6prmTsooND2csPMM5h5BoDdAI5h5onMPAHAeAA7czXAbPPz00cAAEryPPjF\nK8txxTMLO3hEgiC0BWaOC9AYf+dsnPyHj7J+3MOx+Ds6kjCd+4wRzLxce8LMKwAclb0h5ZYJg8px\n3oT+qG9VXD5uV7fpKikIHUYkyliwJdayIxpl3Pzqcqzd05DxY93x5ioM+9U7YIP5vKe+NaPHMF9c\nACAYbrt4MzNeXrgDjareRC1M/m0HmrFiZ3Zdz+ncCy0joscAPKs+/z6AZdkbUu4pMtwSHtGzqANH\nIgjdgwdmr8UjczdiUEUBnrj0WLiJ8PyX2/Dphv34+MYZCetHowxXO42uJz/dAgBoCkbifsvpsudQ\nK3qX5iVd5443V+HJT7dg091n6uMMhNseCfjW8t342UtL9ecBi4vHCffNBQBs+d1Zbd5/uqRj8V8G\nYCWAa9S/VeqybkOh3235WBCE9vHVloMAgK0HmnHzq8vhIkUsrSzcVxftwJBfvo1ddS1tPs5eg2Xf\n0Bpq8/ZzVu/F5HvmYO7afUnX0y4uRhdNeyz+/Q2BuOfBcBTRaO4d/emEc7Yy84PM/E3170Fmzux9\nVAdjnASyugILgtA2GlpjzY1cBKi6byly/1msTBmu29t2N9Bxd8dagxiPmS5Lt9cBAJZsq0tr/ZZg\nzMpvj1aELc7/tSW5nzK1FX4ielH9v5yIlpn/cjfE7GO8PWzPVVwQhHiM1rfbRbqlb2XcknpVONwI\nl/qWECJttJ5jx05vu1aDe6etWtESjODTDfsTll//4lLLC+KjH23E61m6KCRziF2j/j87K0fuRIjw\nC06jrjkIZqBHoS8r+zda342tYdz6upKhauXq0Vz7jMNT/obWcJt/vzEXlP06/1uzV39stPiNx2Jm\n/SKiEYpEceVzi3D1SUfg6P5luOnVZZi7tsZ27KUF3rhl97yzBgBwzrjM971KVo9/t/p/a8aP2snw\nuGM3PuLqEboaW/Y3YVBFQYLwJGPcHbOVbVNMIO6sa0FFoQ95aWSYNrSGUOjzwOWiOIt/qSE50kpg\nNfE9nLh4AKhvDVlOuG7Y14DqisK433ns2Mr/ZBedHzy1QH/cqo4xHImi2eT2Mb9Hu+taMXvVXhwz\nsAc+33gAi5O4k2qbgwnCn02SuXoaiKje4q+BiOpzNsIc0ByIWSdi8QtdiTV76jH9/g/x2Cebs7L/\nqb/7X1q5LU99uhljbn8fz85X7EQ7C9rK4tcuV03Bw6uXdc0LS7C9NjZBHI0yFm49iFMe+BiXPfWV\n5TZahE66HqIWNfFq+C3v4Kp/xYoXNAYS5xcONCkTuX/7aCPueWcNttU22+63tilo+1o2Jn+TJXAV\nM3OJxV8xM5dkfCQdyOh+sSrT2qx9NMrYsK+xo4YkCGmxt14Rl4/WWbsQDoew+ltIZ9+aVb8xxW/G\nUvhVi7/JQjyTYeXP/9+aWHROaziCN5ftAgB8sj7Rt55qXC8u2I7vmmp4aRm35kMv2FKLs//0SVy1\nTU3MD7WkjjY6mET49zcFbF9rL2kXiiCinkQ0UPtLY/0niGgfEa0wLBtHRF8Q0RIiWkBEk9o78Ewy\nul8p1tw5EzNGVOkW/7Pzt+KUBz7Cwq0HO3h0gmCP5qpoCrY9oiUVrW24+9V837XNIf2CYYWV9Zrs\nHBZsqbWc4AyEI9hyoAkAcMtZ1vmkzcEI5qxWLgR2iZna791qXDe+vAzzN9fGLWuxqeL7zoo9WLGz\nPq7a5oEkYm4eU22z/bq76jIfRJlS+InoG0S0HsBmAB8B2ALgnTT2/RSAmaZl9wL4DTOPA/Br9Xmn\nIM/rhs/j0r8I6/cqlsuyHemFeQlCR6BZyc2BzJcVDxhEbsXOQ7qFvXjbQdz55qq4SBhNEGsaWvHH\nD+zLFEdZKUlsjL/XrG2rc/j+Y/NxzQtLsM+UifunORv0sgzFebGpygc/iFXKfGLeZmyrbUZlkQ+R\naGLmbSTKulVuJ+hmLn96QUL28fQRVfjQNGl7/qOf47731ibdV6EvNieQzNXT3MY7oXRIx+K/E8Bk\nAOuYeTCUblxfpNqImT8GUGteDEBzE5UC2JX+ULOPz+PWXT2VRX4AwP7GzN9mCR3H5v1NOTtWMBzN\nepMfLXqmORQvDgu3HsTsVXutNrHkmS+2osaUXGS0+M/+0zw8pNadv+ypr/D4vM1YsTM21acJ5xeb\navHnuRtsj9MYCOP4e+fGlUHXtrXyk1eoUUf/XbY7bvkag/gW+b2477yjE7b9y4cbAQAnHdkz7jga\n97y9Gs98ocxJNLdhfkErqaxx/LCqOHfOwaYgvtxcm/B+mjFOxmvCb/6+/Pz0EfjaEZVpjy1d0hH+\nEDMfAOAiIhczz4VSrbM9XAvgPiLaDuB+ADfbrUhEs1R30IKamsz7L63wuV26lVOmzrDvb0h+uyZ0\nHT5eV4MZ93+I1xbnJmFm9G3vYdrv/5fVY9hZ/N/+62f4v38usNokge21zbj1tRW48rn4SdyASSg/\nWV+DQDiCumZF5N5avhvf/utnmLt2X8K6qdh6IDbRqYlus4Wrp6xAEf49h+Kzeo3ljIvyPDhvQn/L\n45x0ZE8c3b8MANBqEvf/GL4HbWnElGcqpTypujzu+ard6cW+9DGUidCioMzj8HuyU7Y5nb3WEVER\ngI8BPEdEDwFor9n0YwDXMfMAANcBeNxuRWb+u1oRdGJVVVU7D9c2/N5Y/13ttlaz+PccasWmGpns\n7cpomaG56r0QjET1yVeNg01B/OylpZYil4whN79lKeSaldwWi9WMJjZmX3KrKbyyviWEfYbzWbL9\nIBZuPYgfP7swbVeJFdr8QJOFq6choAii0XrecbA5bhI5z+OyDWX9xti+epil+XwKDOVZ3l6+B9U3\nvZVW2QdC/LHKi+JzIRZvS29ecNyAMrz102morihAQ2sYX2w6gKv+tThunY4U/nMAtEAR6ncBbATw\n9XYe7xIAr6qPXwLQKSZ3NXxulx7Hr10ANOGffM8cnJSDUq9C9qAk9WJyxR8/WIeXF+7ASwt2tGm7\nKCPBdXP+o5/j0Y82AVC+r+lmnxp5f+UenPrgxwASc1jMMfEba5pw/L1z9ed7DikXitZQ9PCEX93W\nanK3vkVZVmNwuU77/dw4V492V2BFSb5H73JlHmOhLzGNaeWu1Nb6UtO8n7kw3Aerk9f90fB5XBjV\ntxQl+V40BsL44dML8LEpgsrvyU7tsGRx/I8Q0VRmbmLmCDOHmflpZn5Ydf20h10ATlQfnwSg7c0q\ns4jfMLkbUv+bvyzSm7fr0hkqbmu1Wg53LJEo48vNtWhQLf5IlNOu8W6MujEWJ9vfGMA8Q9hjqmTG\n3YdidwgtwShG9inB8F7pV7fVLlSaxb98x6G431c0yroFbuVyPXVkL3xw/YkY0bvY9hjFeV7dNbOn\nvhV1huiZfF+iqK6xcdOsuTMWp/Lc/G1xrxWa9rNke3oBIZo1X+T3oLE1bBl5lK0OXcn2ug7A/US0\nhYjuJaLxbdkxET0PpSn7CCLaQUSXA/g/AH8goqUA7gYwq70DzwY+j0u3nLRJFvPt4ZG3vtvm23Sh\nc5CsQqRGayiCMbe9h/dW7mnTvq9+fjGm3DMn5Xpa1GBbsmytsJo01gTUKjSxNRTRBcl4gXj+y+1x\n6134+Py4bezoW5oXd2FoDUUwZWgF3r/uRNttzGjbtwQjGN6rCAeagnHRMU3BsP5+1TQGsHLXobjY\n/WK/J2UZ9eK8mMV/yRNfYso9sTmXZBa/OZEzWeayVUawHSN6xS5SmjVf5PegMRC27L+bc1cPMz/E\nzFOgWOgHADxBRGuI6DYiGp5qx8x8ATP3YWYvM/dn5seZeR4zT2Dmscx8HDN3qnZXPrcLzIpVFlB/\nHE2BcEKiSHuqAAodj14hMonw76xrQUMgjN+rdVLS5b9Ld8VZwHZoVq6rDcJvJfKWwq8Kdb2Fn/re\nd9fi3Ec+xYZ9jWllp2890IRLn7TOdgWAPmX5cc8bA+GESc9UBEKKkdUcimBMP2UC1hjWqDVH6lOa\nh9qmIM56eB7+/vEm/XVza8XTR/VKOEZxnhd+g6C2hCL6hHiBhcW/UPXPp+Pr/8bYvgk5BL1LlAnb\naTaROBdOGaRHDGp3HEV5HjS0hi0TvXwd5eNn5q3M/HtmHg/gAgDnAlidldF0MNqb/NgnmxEKKz/Q\npmBYT73WaGsFQKFzQGkU5NLI1icc5ba7eqws71AkcYTPfbENzIyDzTEB0dw6m/c3qv+b0qpHlSoi\nqI9F4xKzxfp/xw9Ouo+xd7yPPfWtiEQZfcuU/RnvputVIRxaFbPqPzeEUpqvnY9eNBGXfq06blmJ\nweLXGHXbe3hg9jp4LUR1U00TdtW1YI6aATxzVG/MuUG5izFHDp06shd+ePyQuGWPXTIRy28/zdb9\nVFHoQziqvP/aXUSx34OddS1oCUXgdcefVM59/BpE5CGirxPRc1ASt9YC+FZWRtNJ+P27a+JcPeZo\nh7DFj07o/OgFuZJ8fIc7DZAsaxUwunrS36fVxKn2/Rw3oAxnHd0HAPDnuRvwyfr9OGjwYwcjUazf\n26C7MGoaAimLoYUiUazbmzyCzUr4ze6Qi6dUY2hVYdL9aHXwtQ5YxsgeTfiHGPZhtMSt7lxuPXsk\nFt96qv680Oex9OU/PGd9QnKm5jY695FPcePLSuX5700aoF947v/O2Lj1rdwwlUV+FOd5bb8H5YU+\nRFT90C5IRWoCWoHPjU9vOgkvXjFFv2PIlqvHtjonEZ0KxcI/E8CXAF4AMIuZc5cBk2N2HIzFChu/\nVMtNX5BgRCZ4uyJaGF460S/tiZABFJdHsigTzeJvDUWxs64F/UwuEyteXxyf5/jKwh26Zfz94wai\nd2ke3lITnBZuPYgPDZEhwXBUj9oBgN2HWlJ+f636vXrdFHeX0ac0cdxmgc3zunV3jR0/fk4pdFZe\n4IPf44pLRKtThd/ox29sTV5Q0e2iuFLTLhcluKCK/R40BMJ6Qbc7zx2NYwaWYUB5ASbfPQf7DKGj\nFYV+27H7LXzympj3tflcKwp9CKkWf75Pm9xVcoaOG1yOnsV56Fmcp3cC7AiL/2YAnwE4ipm/wcz/\n6s6iDwBXnKjctpUVeON8qG8sjf/hSenmrokrDR+/ZpG3955Om/+xcwdqE6/3vrsGU3/3v4RSBIAi\naJp7Z3ttM+56O96zesNLS/X69j6PK85X/dCc9XpXKSAxOGFbbXPCMiNeN1nmOQwsL4h7buxRq72W\nZxKpfJ/b0lfusfBzleZ7Uej3YOXOepz/t89xsCmoR+AYhd84BxBIcnc1pDJ2l2B29fTrkY8B5Yow\nTxzUAxdNHqSEVeZ5MWFQj7h1zTH6RnwWk7p5qphfPm0w/nHxRIzsE1/PsrzQp3sMtHFpb8cAw3us\nXQxy7uNn5pOY+TFmdkyVsv49CvCjE4eiORCJ+1Jp/UM1pHRz1yaZj1/zv6ZDJMq48eWluN1QmEub\nWLX7jmjH1koQG6NYDjYF8cwXWzH8lncw8bcfALCexDXidbuSRpys2h0v4vvqA3hlkX0OQZ7XjZ11\nLQkuhkEV8S6bquKYJTyqryJuEdMFNc/jwrCeib5ubX0jJfle5HvdmLdhP77cUov3V+3R5yqGGyJh\ntMJnxwwsw3Wn2MeY/PfqaVhwyyn6ORnxul04qneJ/tjIcYPjs3ArkjSqsQq11C4GHrcLp47slZCb\nUFbg00N6tTsGLSigf4/YXUKRbvF3XAKXoyjN9yIYier+RSvE4u88BMNR3PfemrSiMLQf3MsLd+Dz\njdapKJo1ZndTUN8a0o+1+1ALXlywA08ZSvFqCUd2MfXmuw1tsvJAYwAz/vAhbn1NKWbbGAiDOTE2\n3+yC8rpdKLAIS9Qwui8BxWLWGoebuXjKILQEI9h9qDXBh2+2+KsNFwKtJIK5rLLH7cJTlx2LF6+Y\nYjpOdcKxFYs/JtC76lpxsDkIn9uVIL69Svx49cqpSUM5C/0ePXrGLJ4eN+lJV2aLetLgirjn5ovG\nt4+JTfAaLX5tP+YwXfOdnzFWXw8z/Vo1RvUtwTfHx/atRSxlK44/WetFR6LV6ElWYEks/s7DW8t3\n4ZG5G9HQGsYd54xOuq7xR3jh4/Ox8e4zE9axaoZt5Ojb3wegdK6yLDGQwuI3W/CagbFub6NeA0dj\n96HWuFZ/APDSwnhr3esmy/hvjYNN8fvckKTsSA/VGt12oAm9S/OwxVBPp4dp3qLS4AK5fNpgMBgX\nTFKqtfcq8eulKiqK/Kgoit0dfPzzGZZhi6UF3rgL2Krd9Sgv8KGswJsgpuaxpIKIUOBzY3S/Uny5\nuRZel0sv12C2+McOKEWPAm9cZJSRP5w/Fm8u26V23IptO+f6E7HJogDgY5dMxOtLdqFvaV6CC037\n3AZXFuKtnx4f95ou/B0V1eM0yvLV4myNgThLwfghi/B3HrR4eLsfqhGjqNsF1URUV086/V+tqklq\nPn47i9/sX9dcAVZledfsqU8Qfi3aRMPndllGrWjUtcTvN1kocon63d9Y04S+psnb0nxPXLSMUYx9\nHheunH6Ebh2/efXxeP2qqRFdjfcAACAASURBVJbHGFhRYBnvX+TzxFnQs1ftxb8XbNdF/ukfxKq7\nlLWjReGqO2bi2lOGKcfK8+jJW+a7Ab/HjQ9/PiPpvrSLhVGUB5QX4MThiTXFjuxdgl/MPBIXTanG\nfaaooGSf2/HDKnHOuL4ozc9OO0YRfhOluvAH4yyLkrzYB5BuaryQfTSxaUkjmzpi8N/bJVCFUrh6\njFh1jHrkww14d8VuveSHsq/EuvX6c72BiSLQRkt6V11ryho4XtPkrhnzXUQytNIDjYEw+pTl4dnL\nj0OxanmWFnjRo9CHW88eiWtOVgT0+GGV6FmcGPVSVezH2AFltscxCqbmUnK5SL8TOHdcX/11rUbP\nicOr9LDOtlr8GpOqy/HDaYNxz7fG6HcXVmUSSvO9+MmMI3D9qdZzCB411v5wJ16T3akd3b8MD31v\nvG0DmcNFXD0mjA2Pywt92KNGXZTke/UwL6uGzkLHoAl/OtUpjRa/y+Y325bkPCvh31TThB89uwjv\nX3eCvuy2N1bi9FG9MfWIyoTyxdokb22jIvz9yvKxX31c1xxMad163a4Ed4URuwYfU4ZUxCVDAUCB\nIRP2uMEVmDasEtecMgy/fWu1bhBdPi2WlPXM5cclHZsdRov/vetOwCH14qRd/L4zcQAGVhTi4Tnr\n48avCWV7LH5AmXO45eyRAKDPJ9hN5v/s9BH2+1G/PIcryek0sM8WYvGbME5iGSMXjLdc4urpPGjh\nkWkJvyEO3c7i1y4O6Vj8Vq4eDeN35J+fb8Wdb64CkGjxa92VapsCKMnzxJUhONgcSnD1mDFnepqp\ns2npd52FNZtnsGCPHxafQGS84z1ctGgWj4tQkufVwxi1sQ7oUYDrTx2O00f1wl3fjM3b5OnC3z6L\n34hm8VtlQKfiyUuPxfkT++uTx+2lrSUuMokIv4niPC+qK5QvovEWusTQ3k2Ev/MQy7Buo8VvJ/xt\ncONpFn8PCwvU7A7U3BNmH39zKNartqLIH5fRe7A5mPK8NGFedcfpCeWBlX1Yu3qsEtQqVUPnznNH\n6z58zZ2RSV+zNuZR/Urjlper0TtajsCjF03E948bpL+ulUjXfp+Hg2bxt6dD2pj+pbj3vLFwHaYb\nRiz+Tob2hTTeQpcYvvgSztl5CLfB4jf6+O1KJqSK6jGiuWkqLCw/s3Gwt0FxGZrHqXXOqm0KoEeB\nN67JR11zSL9DmHVCfE0YDe07WuDzWFqQ22qbE5adOLwKE6vLE+raHDOwB+b9YgYumhwT22nDqnDh\n5IGorkxeeqEt5HndeObySXjq0mPjlr94xRT86YLxtr7z7eq5GOP624tm8Xdk+ZVkLrpsI8JvgZZg\nYrwtN97qivB3HjSLrc0+fluLP30haAyE4TXEhBvZY6rUubuuFe+u2B3Xw7myyIdgJIpgOIpNNU3o\nU5qfYPG3BJXzG2+YLH3g/Fh0iFE8Us1P+DwuXHHiEDxw/li4XYTbvzFKf+3ebys9a/v3iLem+5Xl\n47fnjsm4SB0/rCqutAKgJIl9fWxfmy1iyW/DMiL87bf4uwMyuWvBqL6Kxb/B0N6tJF9cPZ0RTeyS\nRfVc9uSXWLrjUFwEinaXvmZPPfqU5OuT+skyd43uEWZGUyCMQr/HMrvy2n8viXveEorgR88uils2\nqm8pPlpXg0l3f4C65hCmj6jSy4P4PS7d4vd54kM2tdK/QLzwJwtpvWrGUJwxug9Gm9wrGucfO8B2\n287ChZMH4tkvtlleaNuK9pm15Q4vU9x5zii8b+qmlmvE4rdAs/g3GxIyig0W/0Nz1uv9W4WORbPQ\nm5P4wueurUFtUzCuXZ/bRQhFopj5x0/ww3/G6s7HMncTBcF4p7f1QDMaA2EU+jztDusbP1Cx4rWQ\ny1NHxurJ9y7N0338+V43ig1zTL0MWbVW9WKs+PnpR1qK/qVfq8Zfvn9Mu8afa3577hhs+d1ZGdmX\nFibZljmdTHHRlOp2R0RlChF+CyqL/JgxoiruB2GOavhgdcdesZ3Ki19tx8hfv6v/YLVKh3ZROHbd\n0ogIG9Us1hU7lZLFY3/zPm56VUmQ2nWoVfcpaxiFf/r9H6KxNYwiv0ePS+9V0rYoj8EGv/l7154Q\nF63So8CHQy0hNAbCKPC5UVUUE/uKQp9e6MzrsZ9gfOXHX0s5htu/MQpnjunTpnF3B7Q7paBDS6yL\n8Nvw5GWTcOaYPnpd7KK8+NvLZMWbhOzxm/+uRHMwgqZgBI2BMOauSd7Y2mjlG3ERsEqtUT9IjRI5\n1BKKC+8zNhYHEvM3Nu9vQkl+zNVz0xlH4srpQ1Oew8xRvfGDqYPjWv+Z69aXFXjBrNQVOtAURGVx\n7PuW73PrwQZGV88bP5mKpy6LTZiaK012JFdOH5rUf59rtIvuT086ooNH0jGIjz8Ff794AnYfasV6\nk2tH/PwdgybLLcEIbntjBT5YHRP+Rz/aiFknDIkrJ7Cv3rrmUpSBtepnWmWRfWpm8/4mfLphf9yy\n9fsacfqoXjF/cYQt67BXFvn0pCxAKf89fmAPfX+j+5XofVv/36SB+GT9fkweUqFX7gyGo3F1bHxu\nF0rzvahtCsaVOD66f1nCZOVlU6szEgVzuNw488iOHkIchX5PxtxGXRER/hQU+DwYWlWEzTXxBZga\nLQp0CbmjKRjG+n3xBcfueWcNxg0ow3FDlAqLLy/cYVtlNRiOYn+DIsatoUjKC/m3/vKp5eRpuaFR\nRyActSyfUFnkjxN+LSZ+YnUP3DhzBC40hE+eMaYPtvzuLMxdG7ug/dNQpwZQ3FQleUptG3MBM3Ot\n+9u+PgqCYEaEP03cpgzJxkD6NVCE9tMYCOO8v36G608djme+2KqHbTbbXHi1omfr9jbgZy8ttd1v\nIBzRM0UbA5GUNXHsImYqCn16Bq9SsTFR+M0XA81N4/e4ceV0a1eDsR7NCRbFv0ryvZZZu+YLgSBY\nIT7+NNEsqT6leSjN98a1gBOyx1dbarFmTwNmPbMQn6yPuVqag2HLWilaP9lUfWVbQ1G9N21TIJyy\nNILdnE55oU+PzhlSVWhZx8kc9ZNO+QOrbGBjglZJnteyWbggpINY/GmihX8RgCK1Z6eQfexcMM3B\niKV1q5VKSBaP//PTR+C+99bqlR+bAuGUFr9dlcSKIh/OGdcPo/uVYmhVEd5ZvjthHeME7I0zR6QV\n/mlVj+azm07Wy0T065F/2LViBOciwp8mHkM5x+I8j2VlRiHz2GVWNtlY/FpzFKsmKRraZKzWLKQx\nELYN+6xvDaHA69Zb/pnRrPehVUo3qJmje+P6U4fjgdnr9HV8bhfu/fbRaA1HLLtPWe838adZXujT\n69lce8ow2zIOL/1oSlySlyCYEeFPE93iJ0Kh35O0MqOQOexcMHY+fq3nrbHX6eDKQkwfUaW3HNSq\nQ2p3E4FwVG+ZaObo29/H2Uf3sS2HYI7iISKcO65fnPD3Kctrc2asdjdzlk2MfYHPY9ty8djqcsvl\ngqAhwp8mmo+fmVHk99iWu02XtXsasGT7QXz32IGZGF6XZV9DK1qDUQy0qbho1aYPUITdqt6OFsVj\nvCMrzvPgmpOH6cI/3NCrtXdJHvbUt8bV0DHz5rJE9820IyrxwHfHomdxomVdYOgde8+3xuDccf1s\n952M9XedAbdM1gpZQGaH0sTo4y3KU3z8oUgUf/7fegz/1Tt49KONbdrf6X/8GL94Zbnt68yMu95a\nhaXb69o95s5KKBLFDS8uxaaaRky6aw5OuG+u7bp24ZiKj99ifd3ij90RFPk9cdE2Ew0Wcd8yRbgX\nbj2YdMyDKgrwkxlKBE6hz42bzjjSUvSV12P21AWTBiZtsZcMr9t12KV/BcEKEf400SbolBhqL+pb\nwnj+y224//11CEaiuOedNe3ar50POxRh/OOTzfjmXz5t95g7K8t3HsIri3bghiThlhp2Fr+tT151\n2ZgtfmMhNbeLcM+3xgCAXq7gqc+2JB3HL888CjecNhyf3DgDK++YaVvsDIhF32gXCkHobIirJ02M\nFn+PAi/qmoNYvO3wrfHmYASl+YpQHGwK6qVqteiUDigemDOMFz1mtozSsRP+f83fZhnVUqeu32wQ\n/pI8b8K+L5g0EN+Z0B8etwtlBT7LmP8Xr5iCd1bsxpOfbsHIPiUgIr1bVDKIyNFZoULnRyz+NPHE\nCb8P4Shjwdbaw96vNnk5e9VejL9zNr7crOzTGMb45rJduPW1FYd9LCOhSBSXPPElrn5+cbv3Udcc\nxKhfv4v5pt6tqdCscWPte7vJcjvhP9gcSsjcBYDlO+ow/JZ38PD/NujLJquZvGa0MglnH209gVpd\nUYCbzzgK71xzfFqCLwhdhawJPxE9QUT7iGiFafnVRLSGiFYS0b3ZOn6mibP4Vat8e23LYe9Xiz75\naosi+JpP3yj8P/nXYjzzxdbDPpaRZTvq8NG6GvxXrf/eHpbuOISmYAR/nrsh9coGNHeMMVKmziYz\ndq+h1k7f0jycMbo3BpQn1sPRONgcSoj9P+nInknHY9cCryhPKbl8VJ+SpNsLQlcjmxb/UwBmGhcQ\n0QwA5wAYy8yjANyfxeNnDausyvaiWfzaZSWq1he2SlyKZtDvEwwf/r48ek3ztu1Lm4A1WuxWlv1X\nW2qxane9/rwoz4O/XjgBb1w1La3j9CnNw9yfTU/o9GSF2eqf94sZtuGSgtDVyZrwM/PHAMy+kB8D\n+B0zB9R1ktfU7YQQwVZIXl20A9/522eWr63b26DXdzf6tjW3h+aDvuedNfhw7T4EI4lx6q0W5QA0\nAuEIlu84lN5JIDMt57S7oFQt/8xYRepo5RPW7mnANS8sRk1DAN/52+dx62iWeWm+1zaT1khFkS+u\n5v1jF0/Eq1da16h/6Hvjsez20/Tn5haEgtCdyLWPfziA44loPhF9RETH2q1IRLOIaAERLaipqcnh\nEFPTwyKdHgCuf3EpvtpyULfM31+5Byt2KmJ82oMf6/Xdjdat1jnKqGM/e2kZWi1qzSTrK3vGHz/B\n1/88T7emU5GspIFGbVMwrndsIBzB4m2xsEdtyFb7ag3ZX4isxqi5et5fuQevL9mF61+MtS7U7rDy\n1IYnLheldddV7I9f55SRvXDMQOsa9W4XpVVDRxC6A7kWfg+AcgCTAfwcwItkU06Qmf/OzBOZeWJV\nVWJ1wlyjRZD8ZMYRKUXnQFMQ1Te9hVnPLMTZf5qHWlO6vzH5S8tANSYjBcIRParHiF0Wa2sogk1q\nm8hUxcY00nH1HHPnbEy+Z47+/OZXl+Obf/kMe+uVi4HWtMTK4v/FK8vw9T/Ps0yMssqS1d6TWvX/\ngi2xC8yRvRUfu99QpKw8DfeNVv64Lfz89BG4ZMqg1CsKQhcm107MHQBeZaWh6ZdEFAVQCaBzmfQW\n5PvceoheKl/7ttr42v3LdsTCPpkZ+xpiYqjFoxst/kA4aunjtyskZhT7dJtHG630SJTjXCdPf7YF\n9723NmGb91cq7SYPtYTQqyRPdxdZHVOLTrK6ENW3huB1U1y3q3q12qnWOEU714smD0K+z43PNx2I\nm4StKPQDSIzqMdIe4b9KYu8FB5Br4X8NwAwAc4loOAAfgP3JN+l8pMqmbDCVbNYmKP0eFwbf/DaG\nGUoGrNpdj3V7G+L2GbQRfjtXj7EXbLrNo40+/lAkCrcrJqq3vbEybt0fPr0Ao/qW6CGXmltGO66V\nxa+JejjKCEWiWLWrHmMHKOWLG1rDGNG7GL2K8zBHbZ1433tr0bskD3vrW+H3uPR9j+lfql88agwX\nzIqiNCz+DE7CC0J3IpvhnM8D+BzACCLaQUSXA3gCwBA1xPMFAJeo1n+XI5n2m107G/cpdwCamBmj\nWZ78dAtOe/BjkKnWpLXwW8e6t4babvEbrW3zRK+5i9MHq/fioTnr9efaHIW23a66Flz13KK4bFnt\njqI1FMFdb63GOY98is2qO6oxoDQp95gaidzw0lLsbWjFqL6x8Mk8rxtfG6rE4S8xlK8wT772KvHj\nznPiu021x+IXBCeQNYufmS+weenCbB0zlxT6PQmWvYZZ+BdtS6wD4/O4EIpEoV32zDMdbfHxGy3+\ndCNs4i3++G28bhfCUfu5grrmIL52zxxUqaV/61vDeGv5bpwwvFIvOhdR9xkIR/H5RiXB6z+LdmBD\nTSOaAmH0LsnTE6iM7K0P4LjBFVikZkXne904Qr1DunByrKBdtaGoW3mhD/N/eQoA4NbXY3crxRal\njQVBkJIN7aYoifAfNFXu1CxdIz2L/dhxMJYAZrbU2+LjN1r86YZpGmPvze4hr5tgkzALANi0vwm7\nDrVilyHiBwDmrN6HOav34dGLJiBksPg1F5GWTesipX691V1TMBzFIEOWbL7XDSLChrvOiLtQaNU8\nh/UswmtXTbUcp9fiwiIIgpRsaDdVxfbdj2qbElXTnP3Zs9iP44dV6s/NLfuMwn9k72IA6fn4rSz+\n/Y0BPRJHw3iBCEaiWLHzEJ5Vs4NTdYhaYlOj6P1Ve/H+qr1oCkb0cbSGIgkuqigDhX63pcUPAL0M\nTUS0gmfmdfuXKcIfiTIK/db2Szqx/oLgRET428kj/+8Y/OjEoZav1TYpk5Dj1MlMQPFBGxndrxR/\nvXAChvdS3Bh1pouFFuv+wqzJ+MfFEwEkuno21TTikbkbTBZ/ovBP/O0HOO7uOXHLQnEWP+PsP83D\nLWo9oFSW8pIUpaKbAmF9/62hqGU3rAKfJ2EuQcM4KWtXTqF/j3xc+rVqPHzBeNtx2O1fEJyOCH87\nGVBegJvOONLytYOqiP/yzKP0ZdUVSgbp5CFKLfiLpwxCkd+DH05T2ueZ4901d9FRfUr0HIL6lhB+\n8fIyPQP4sqe+wn3vrY1LskrXxx82RfUYH6ey+FP1pzUWXLPLSSj0uW0t8jLDpKxdLXuXi3D7N0Yl\nlEe+/ztjMbSqEOeO64szRlsXXxMEpyM+/gwxqbocV84Yikuf/AoHVIu/p8EddPJRPXHehP4Y1bcE\nwUgUfjULtUQVuf2mCeGDasik3+PSa8l/sHovlu44hH0NrXjyskkIqNm9xn6w7QvnjF0smgLhpBb/\njBFVWLrjEA61hGwvMsbonmteWGK5ToHfg5F9S/Hc/G0JrxkbjdtZ/HacN6E/zpvQv03bCILTEIs/\nQ7z4oyl6+V9NtMsNseYleV6M7lcKItJFH4iFHB4wWfxaJqvP7VL6/Prc2KP66bXtNf+30X+fdjhn\n1DqcszGF8D9x6bFYdOupuP0bo2zXSdboXKPQ58bM0b2x+NZT0bc0vpNVmcHVk99G4RcEITUi/IfJ\nbV8fiYmDlPovfo8LRLFwTqNoldjElJfkKzddCa6eJiW7VUvs6lHo00sUa5OZmjW8ryEm/PWtIeyr\nb8V7K/dg9qq9lsd88avt+OuHsVaR763coz9uDIThc9v7xrUKG5VJSiYcakndj1irfNmj0IeIKZXD\nGH8vwi8ImUdcPYfJZVMH47KpgwEooljgdaMpGIHHRXGWc4lNTLlW8M1clO1gcxA+w/YVRbHwz221\nTdjX0Kq7gIw163/yr/jGKhvvPlN/rHW5MiZjAcBfDBeBxtaw5QQxABzdP+ZPT5YVu/VAs+1rGoWG\nhuSDygvjzsHo3vGnmG8QBKHtyK8qwxSo1rjZUi22qfzYpzQPvUsSm3YfagnFTbIaLeyvthzEqQ98\nDL96DHOoppGhv3xbf6xdXJKFOTYGwgmhpQAwoDwfd39zjP68LN/e4k+n/7Dx3P564TH4y/ePsVxP\nmo0LQuYR4c8wmpvCbxJ+u0gZIsL0EYnVR2ubgnHbmKtRHmoJGVw9iRUwrfjrhxuwYEttSuE3R+FU\nFPrwyY0nxUXQlLWzDo5Wp8iYQFZR5MeZY/rgo59Px8s/mgIAOOWoXu3avyAIqRFXT4bRhD/fl/41\n9QhD0TaNQDgal8hUYdFYXJNvqyxfKx7+3wbM35xc+JsC4YT9Wc1PtFf4TzqyJ9bva0SRhetrUEUh\nBqlhr49eNCEjzWIEQUhEhD/DaMKvNQ255ayjsHp3Q9JtNGu+OM+Dt396vN6wRcvYBax93ekKvhHm\n5IlNDa3pCX9bJ13/duEETBjUA6X5XowfWIYpNg3QNdwuiqsYKghC5hDhzzCa8Gti+cPjh6TcRhP+\naJTRtyzWSFxrQGLcr5FkrRjt2NfQmrSXbKOVxW9hndv0z7GlurJAL3MxUxKrBKFDEeHPMJpAGwU8\nFUpTEaApGIlzwxzZJ2bxXzRlEMb0L8WlT3yJJrV0Q7MhXr7I74nLmLWjpiGAIVX2bpr6lkQfv10o\nqpm5P5uOF77ahkKfB6eP6o33Vu7BA7PXAQAKvPJVE4TOgvwaM4wmkr2SFHEzU27TVOTo/rFaP163\nC8dWl8claO0xRPMMKC/AarXhSzKagpG42j5m3lq+KyGcM91etIMrC3HzGbEyFSN6F+vCb1d6QRCE\n3CNRPRlGK5lgNXlpR7lN8/Yii6qTxjIJxqbtgyvjG5NcceIQvPET63LF5n4BRrR4+qlHxHzwWpKZ\nmTk3nIg5N5wIAJYhqUYKRPgFodMgFn+G0erUWIm2HWZr+JIpg/TCbGbMWa4axjmAN34yFWPU8hBn\njemDt5bvjlv3QBLh1/ja0Eo8+N1xmHTXHFuLf2iVEo305tXT4iKQrJAMXEHoPIjFn2GOH6bE5E9O\nEbViptDnxgWTBgAAfnPOaFx98jDL9ewaVRonbAt8bn3y9eIpg/TlkwaXpz2eHQebUV7gw4wRVSnP\nZXS/0qT9CQBJxBKEzoRY/BnmlJG9sOqO05NGzlix8o6Z7TreN8f3w+rd9bhsajUen7cZQHzJg+OG\nVGDL787C9tpmNAXDmPnHT9Lab3GeFx63C09eNqld4xIEofMiwp8F2ir6beFb4/vh1cU79eeDKgrw\n4HfHxa1j5VYZUF6AnXUtCctPHF6FPK8L763ci7OO7oNvjusHr8eFSdXp3x0k48lLj8WyHYcysi9B\nEDKDCH8X497zjsZPTx6G6fd/CMC6Xr1dBE2R4YI0aXA5HvzuOPQry8dri3fivZV74SbCKSMzWyph\nxpE9MePInhndpyAIh4f4+LsYHrcrbiLVKqM3z2Mt/AWGiphj+pWin5proF0o7HrXCoLQvZBfehdE\na8ACxMo6G7GbSPW6XfB5XAiGo3H7OPWoXvjFzCNx4eSBmR+sIAidDhH+LoixXIK5amcq/JrwG+4K\nXC7Cj6dbN44XBKH7Ia6eLk6FTdavHVE1AUzcOoLgXET4uzh2iV52aHV+BlUUpFhTEITuigh/F8fK\nx58O1ZWFGR6JIAhdBRH+Lo5dZ69UDOghFr8gOBURfoehTQa394IhCELXR2b4uigvzJocV50TUMoi\nDyhPbsnPvu4ENAfb3sBFEITugwh/F8WqcNrcn01PuV1FkR9tKx8nCEJ3I2v3+0T0BBHtI6IVFq/d\nQERMRJXZOr4gCIJgTTYdvU8BSCg5SUQDAJwGYFsWjy0IgiDYkDXhZ+aPAdRavPQggBsB2FSWFwRB\nELJJTn38RHQOgJ3MvNRYdsBm3VkAZqlPG4lobTsPWwlgfzu37Q7I+cv5y/k7l0FWC3Mm/ERUAOCX\nUNw8KWHmvwP4ewaOu4CZJx7ufroqcv5y/nL+zj1/O3IZzD0UwGAAS4loC4D+ABYRUe8cjkEQBMHx\n5MziZ+blAPSOHKr4T2RmJ9+GCYIg5JxshnM+D+BzACOIaAcRXZ6tY6XgsN1FXRw5f2cj5y8kQMwS\nXCMIguAkpGCLIAiCwxDhFwRBcBjdVviJaCYRrSWiDUR0U0ePJ1MQ0QAimktEq4hoJRFdoy4vJ6LZ\nRLRe/d9DXU5E9LD6PiwjomMM+7pEXX89EV3SUefUHojITUSLiehN9flgIpqvnue/icinLverzzeo\nr1cb9nGzunwtEZ3eMWfSdoiojIheJqI1RLSaiKY48PO/Tv3+ryCi54koz0nfgcOGmbvdHwA3gI0A\nhgDwAVgKYGRHjytD59YHwDHq42IA6wCMBHAvgJvU5TcB+L36+EwA7wAgAJMBzFeXlwPYpP7voT7u\n0dHn14b34XoA/wLwpvr8RQDfUx//DcCP1cdXAvib+vh7AP6tPh6pfi/8UMKMNwJwd/R5pXnuTwP4\nofrYB6DMSZ8/gH4ANgPIN3z2lzrpO3C4f93V4p8EYAMzb2LmIIAXAJzTwWPKCMy8m5kXqY8bAKyG\n8kM4B4ogQP1/rvr4HAD/ZIUvAJQRUR8ApwOYzcy1zHwQwGxY1FbqjBBRfwBnAXhMfU4ATgLwsrqK\n+fy19+VlACer658D4AVmDjDzZgAboHxvOjVEVArgBACPAwAzB5m5Dg76/FU8APKJyAOgAMBuOOQ7\nkAm6q/D3A7Dd8HyHuqxbod6yjgcwH0AvZt6tvrQHQC/1sd170ZXfoz9CqfcUVZ9XAKhj5rD63Hgu\n+nmqrx9S1++q5z8YQA2AJ1VX12NEVAgHff7MvBPA/VAKPe6G8pkuhHO+A4dNdxX+bg8RFQF4BcC1\nzFxvfI2V+9huGadLRGcD2MfMCzt6LB2EB8AxAP7KzOMBNEFx7eh0588fANT5i3OgXAT7AihE17pb\n6XC6q/DvBDDA8Ly/uqxbQEReKKL/HDO/qi7eq97CQ/2/T11u91501fdoKoBvqJnfL0C5vX8IigtD\ny0Q3not+nurrpQAOoOue/w4AO5h5vvr8ZSgXAqd8/gBwCoDNzFzDzCEAr0L5XjjlO3DYdFfh/wrA\nMHWW3wdlQueNDh5TRlB9k48DWM3MDxheegOAFplxCYDXDcsvVqM7JgM4pLoE3gNwGhH1UC2o09Rl\nnRpmvpmZ+zNzNZTP9X/M/H0AcwGcp65mPn/tfTlPXZ/V5d9TIz4GAxgG4MscnUa7YeY9ALYT0Qh1\n0ckAVsEhn7/KNgCTiahA/T1o74EjvgMZoaNnl7P1ByWaYR2UmfpfdfR4Mnhe06Dcxi8DsET9OxOK\nz3IOgPUAPgBQrq5PmlMgHgAAAnFJREFUAB5R34flUOojafv6AZQJrQ0ALuvoc2vHezEdsaieIVB+\ntBsAvATAry7PU59vUF8fYtj+V+r7shbAGR19Pm0473EAFqjfgdegROU46vMH8BsAawCsAPAMlMgc\nx3wHDvdPSjYIgiA4jO7q6hEEQRBsEOEXBEFwGCL8giAIDkOEXxAEwWGI8AuCIDgMEX5BMEBEv1Kr\nPi4joiVEdBwRXUtEBR09NkHIFBLOKQgqRDQFwAMApjNzgIgqoVS//AzSH1roRojFLwgx+gDYz8wB\nAFCF/jwo9WDmEtFcACCi04jocyJaREQvqXWTQERbiOheIlpORF8S0RHq8u+odeOXEtHHHXNqghBD\nLH5BUFEFfB6UMr8fQKnb/pFaF2giM+9X7wJehZLl2UREv4CSIXqHut4/mPkuIroYwPnMfDYRLQcw\nk5l3ElEZK2WUBaHDEItfEFSYuRHABACzoJQ+/jcRXWpabTKUBh6fEtESKDVgBhlef97wf4r6+FMA\nTxHR/0FpEiQIHYon9SqC4ByYOQLgQwAfqpa6uSUhQWlgcoHdLsyPmflHRHQclOYxC4loAjMfyOzI\nBSF9xOIXBBUiGkFEwwyLxgHYCqABSptLAPgCwFSD/76QiIYbtvmu4f/n6jpDmXk+M/8ayp2EsRSw\nIOQcsfgFIUYRgD8RURmAMJRqjrMAXADgXSLaxcwzVPfP80TkV7e7BUolWADoQUTLAATU7QDgPvWC\nQlAqaC7NydkIgg0yuSsIGcI4CdzRYxGEZIirRxAEwWGIxS8IguAwxOIXBEFwGCL8giAIDkOEXxAE\nwWGI8AuCIDgMEX5BEASH8f8BCnP8MqEQzx8AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p_OtUA4QWobb",
        "colab_type": "text"
      },
      "source": [
        "Según esta gráfica, parece que el MAE deja de mejorar significativamente después de 150 epochs. Pasado ese punto, comenzamos a sobreajustar.\n",
        "\n",
        "Una vez que hayamos terminado de ajustar otros parámetros de nuestro modelo (además del número de epochs, también podríamos ajustar el tamaño de las capas ocultas), podemos entrenar un modelo final con todos los datos de entrenamiento."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "v4Xk7C-yFHAb",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 134
        },
        "outputId": "7e029dd2-2b20-44cb-d702-b762e1296196"
      },
      "source": [
        "# Get a fresh, compiled model.\n",
        "model = build_model()\n",
        "# Train it on the entirety of the data.\n",
        "model.fit(train_data, train_targets,\n",
        "          nb_epoch=150, batch_size=16)\n",
        "test_mse_score, test_mae_score = model.evaluate(test_data, test_targets)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "creating: createZooKerasSequential\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createZooKerasDense\n",
            "creating: createRMSprop\n",
            "creating: createZooKerasMeanSquaredError\n",
            "creating: createZooKerasMAE\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sREgEXx-FK6I",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "0eb6c92e-6b05-40c0-9584-a0ee0e50e30d"
      },
      "source": [
        "test_mae_score"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.6815447807312012"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aUFuEfy4Wobj",
        "colab_type": "text"
      },
      "source": [
        "Nos desviamos unos $1.700."
      ]
    }
  ]
}