mutliclass_classification.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "mutliclass_classification.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyN2+KQicmRDt9236iUPMNDa",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/jpcompartir/dl_notebooks/blob/main/mutliclass_classification.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Looking at a multi_class classification - following on from binary classification. Using the Keras API again, we'll build a keras.Sequential model using dense layers with relu activations into softmax classification head, we'll compile with a categorical_crossentropy loss function, rmsprop optimizer, accuracy as a metric - could add f1_precision too probably?\n",
        "\n",
        "---\n",
        "\n",
        "We'll be using the Reuters dataset to classify posts into 46 topics (which guides our architecture choices, we're going to need > 46 units in each of the hidden layers)"
      ],
      "metadata": {
        "id": "DBQaLTNATAtU"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "fQVcf3lAS8lG"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "from tensorflow import keras\n",
        "from tensorflow.keras import layers\n",
        "from tensorflow.keras.datasets import reuters"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Read the data in, using the familiar 2 tuples - load_data() and num_words = arguments. Then define the vectorise_sequences() function we defined in the binary classification problem - from memory and if you've forgotten, try to figure it out."
      ],
      "metadata": {
        "id": "bGMPXBstWH2M"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "(train_data, train_labels), (test_data, test_labels) = reuters.load_data(num_words = 10000)"
      ],
      "metadata": {
        "id": "_hTow-4FVucC"
      },
      "execution_count": 23,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def vectorise_sequences(sequences, dimension = 10000):\n",
        "  results = np.zeros((len(sequences), dimension))\n",
        "\n",
        "  for i, sequence in enumerate(sequences):\n",
        "    for j in sequence:\n",
        "      results[i, j] =  1\n",
        "  return results"
      ],
      "metadata": {
        "id": "1zL3sLx0UyaZ"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x_train = vectorise_sequences(train_data)\n",
        "y_train = np.asarray(train_labels).astype(\"float32\")\n",
        "\n",
        "x_test = vectorise_sequences(test_data)\n",
        "y_test = np.asarray(test_labels).astype(\"float32\")"
      ],
      "metadata": {
        "id": "APotnLVSX9r1"
      },
      "execution_count": 24,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "word_index = reuters.get_word_index()\n",
        "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
        "decoded_review = \" \".join([reverse_word_index.get(i - 3, \"?\") for i in train_data[0]])\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "APix7sslWU7O",
        "outputId": "18d4b21b-d41f-4242-804b-7ecf774e2458"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/reuters_word_index.json\n",
            "557056/550378 [==============================] - 0s 0us/step\n",
            "565248/550378 [==============================] - 0s 0us/step\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Write this up into a function again:"
      ],
      "metadata": {
        "id": "CrD8qmpPXYxi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def decode_review(index, data = train_data):\n",
        "  word_index = reuters.get_word_index()\n",
        "  reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
        "  \n",
        "  return \" \".join([reverse_word_index.get(i - 3, \"?\") for i in data[index]])"
      ],
      "metadata": {
        "id": "A359gCMnWYQ9"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "decode_review(1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 53
        },
        "id": "SjTKGMBZXp0R",
        "outputId": "dc770040-f82f-410f-bec0-62cc92bb746c"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'? generale de banque sa lt ? br and lt heller overseas corp of chicago have each taken 50 pct stakes in ? company sa ? factors generale de banque said in a statement it gave no financial details of the transaction sa ? ? turnover in 1986 was 17 5 billion belgian francs reuter 3'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Our loss function - categorical cross entropy will expect our labels to be formatted in one-hot form. We can use the Keras API or define our own function, following similar logic to our vectorise_sequences function. We'll be outputting the scores for each topic, so we'll need to set dimensions as 46."
      ],
      "metadata": {
        "id": "i7-5UQb1bytw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def to_one_hot_labels(labels, dimension = 46):\n",
        "  results = np.zeros((len(labels), dimension))\n",
        "\n",
        "  for index, label in enumerate(labels):\n",
        "    results[index, label] = 1 \n",
        "  \n",
        "  return results"
      ],
      "metadata": {
        "id": "rYSzBNMSbtbw"
      },
      "execution_count": 20,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "y_train = to_one_hot_labels(train_labels)\n",
        "y_test = to_one_hot_labels(test_labels)"
      ],
      "metadata": {
        "id": "oAplegWbckBP"
      },
      "execution_count": 25,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Or using the Keras API"
      ],
      "metadata": {
        "id": "MZZKzlPec55A"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.utils import to_categorical\n",
        "y_train = to_categorical(train_labels)\n",
        "y_test = to_categorical(test_labels)"
      ],
      "metadata": {
        "id": "L8boFq40c5HK"
      },
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Oki, now we can look at building the model."
      ],
      "metadata": {
        "id": "1OV63FEUXucv"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = keras.Sequential([\n",
        "  layers.Dense(64, activation = \"relu\"),\n",
        "  layers.Dense(64, activation = \"relu\"),\n",
        "  layers.Dense(46, activation = \"softmax\")\n",
        "])"
      ],
      "metadata": {
        "id": "l_gq2SThXtG-"
      },
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.compile(\n",
        "  optimizer = \"rmsprop\",\n",
        "  loss = \"categorical_crossentropy\",\n",
        "  metrics = [\"accuracy\"]\n",
        ")"
      ],
      "metadata": {
        "id": "VkjeuRpcdH9V"
      },
      "execution_count": 48,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Set up the validation splits, taking 1k labels this time:"
      ],
      "metadata": {
        "id": "dwhfVZnTdgfr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x_val = x_train[:1000]\n",
        "y_val = y_train[:1000]\n",
        "\n",
        "partial_x_train = x_train[1000:]\n",
        "partial_y_train = y_train[1000:]"
      ],
      "metadata": {
        "id": "60lrN8DfdVw3"
      },
      "execution_count": 49,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "It's now model training time:"
      ],
      "metadata": {
        "id": "dtYzrWWZd0LN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "history = model.fit(partial_x_train, partial_y_train, epochs = 20, batch_size = 512, validation_data  = (x_val, y_val))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9HMA79yhdzag",
        "outputId": "d917d6c8-3b93-4a8d-ae65-e9643ac526ff"
      },
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "16/16 [==============================] - 1s 55ms/step - loss: 2.6332 - accuracy: 0.5099 - val_loss: 1.7391 - val_accuracy: 0.6490\n",
            "Epoch 2/20\n",
            "16/16 [==============================] - 1s 45ms/step - loss: 1.3970 - accuracy: 0.7127 - val_loss: 1.2829 - val_accuracy: 0.7120\n",
            "Epoch 3/20\n",
            "16/16 [==============================] - 1s 45ms/step - loss: 1.0200 - accuracy: 0.7831 - val_loss: 1.1212 - val_accuracy: 0.7550\n",
            "Epoch 4/20\n",
            "16/16 [==============================] - 1s 46ms/step - loss: 0.8050 - accuracy: 0.8261 - val_loss: 1.0275 - val_accuracy: 0.7890\n",
            "Epoch 5/20\n",
            "16/16 [==============================] - 1s 46ms/step - loss: 0.6424 - accuracy: 0.8648 - val_loss: 0.9650 - val_accuracy: 0.8020\n",
            "Epoch 6/20\n",
            "16/16 [==============================] - 1s 45ms/step - loss: 0.5141 - accuracy: 0.8938 - val_loss: 0.9262 - val_accuracy: 0.8090\n",
            "Epoch 7/20\n",
            "16/16 [==============================] - 1s 87ms/step - loss: 0.4150 - accuracy: 0.9141 - val_loss: 0.9367 - val_accuracy: 0.8060\n",
            "Epoch 8/20\n",
            "16/16 [==============================] - 1s 48ms/step - loss: 0.3404 - accuracy: 0.9268 - val_loss: 0.8952 - val_accuracy: 0.8110\n",
            "Epoch 9/20\n",
            "16/16 [==============================] - 1s 45ms/step - loss: 0.2796 - accuracy: 0.9399 - val_loss: 0.9067 - val_accuracy: 0.8110\n",
            "Epoch 10/20\n",
            "16/16 [==============================] - 1s 46ms/step - loss: 0.2422 - accuracy: 0.9448 - val_loss: 0.9044 - val_accuracy: 0.8230\n",
            "Epoch 11/20\n",
            "16/16 [==============================] - 1s 45ms/step - loss: 0.2057 - accuracy: 0.9489 - val_loss: 0.9344 - val_accuracy: 0.8190\n",
            "Epoch 12/20\n",
            "16/16 [==============================] - 1s 45ms/step - loss: 0.1784 - accuracy: 0.9534 - val_loss: 0.9443 - val_accuracy: 0.8150\n",
            "Epoch 13/20\n",
            "16/16 [==============================] - 1s 46ms/step - loss: 0.1642 - accuracy: 0.9541 - val_loss: 1.0005 - val_accuracy: 0.8030\n",
            "Epoch 14/20\n",
            "16/16 [==============================] - 1s 95ms/step - loss: 0.1493 - accuracy: 0.9554 - val_loss: 0.9728 - val_accuracy: 0.8160\n",
            "Epoch 15/20\n",
            "16/16 [==============================] - 1s 52ms/step - loss: 0.1389 - accuracy: 0.9569 - val_loss: 0.9991 - val_accuracy: 0.8100\n",
            "Epoch 16/20\n",
            "16/16 [==============================] - 2s 98ms/step - loss: 0.1321 - accuracy: 0.9554 - val_loss: 1.0665 - val_accuracy: 0.8060\n",
            "Epoch 17/20\n",
            "16/16 [==============================] - 2s 99ms/step - loss: 0.1258 - accuracy: 0.9546 - val_loss: 1.0624 - val_accuracy: 0.8030\n",
            "Epoch 18/20\n",
            "16/16 [==============================] - 1s 82ms/step - loss: 0.1185 - accuracy: 0.9573 - val_loss: 1.0559 - val_accuracy: 0.8080\n",
            "Epoch 19/20\n",
            "16/16 [==============================] - 1s 77ms/step - loss: 0.1158 - accuracy: 0.9569 - val_loss: 1.0695 - val_accuracy: 0.8130\n",
            "Epoch 20/20\n",
            "16/16 [==============================] - 1s 79ms/step - loss: 0.1176 - accuracy: 0.9548 - val_loss: 1.1249 - val_accuracy: 0.8040\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Plot our outputs:"
      ],
      "metadata": {
        "id": "NiEkcJffedK4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "history_dict = history.history\n",
        "accuracy = history_dict[\"accuracy\"]\n",
        "val_accuracy = history_dict[\"val_accuracy\"]\n",
        "\n",
        "loss = history_dict[\"loss\"]\n",
        "val_loss = history_dict[\"val_loss\"]\n",
        "\n",
        "epochs = range(1, len(loss) + 1)"
      ],
      "metadata": {
        "id": "oB-bModMd-x7"
      },
      "execution_count": 51,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "epochs"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KRzeQqQre-Gy",
        "outputId": "54ef0ef6-c142-4b5e-d048-ae0cb5607aa0"
      },
      "execution_count": 44,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "range(1, 21)"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "plt.plot(epochs, accuracy, \"bo\", label = \"Training Accuracy\")\n",
        "plt.plot(epochs, val_accuracy, \"b\", label = \"Validation Accuracy\")\n",
        "plt.plot(title = \"Multiclass Classification Accuracy\")\n",
        "plt.plot(ylabel = \"Accuracy\")\n",
        "plt.plot(xlabel = \"Epoch\")\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "cE7R-SrLem0W",
        "outputId": "37c81d85-3c37-41f8-c50f-e9287abf4bb1"
      },
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3hU1b3/8feXgCAXUQgK5ZKgRW5iuETwggJqW1AKimJFbFVaUR6torUeKlU5Vk49R+tRW+058VJr4QcKFoWCUhFB6uVIQEBBVC5R4gUxCELRQsj6/bFmwiRMkkkyM3tm8nk9zzyZ2bNn7+9shs+sWXvvtc05h4iIpL9GQRcgIiLxoUAXEckQCnQRkQyhQBcRyRAKdBGRDNE4qBVnZ2e73NzcoFYvIpKWVq1a9aVzrl205wIL9NzcXAoLC4NavYhIWjKzj6p6Tl0uIiIZQoEuIpIhFOgiIhlCgS4ikiEU6CIiGUKBLiIxmzkTcnOhUSP/d+bMoCuqnfrWn+rvX4Eu0oDUJ5BmzoSJE+Gjj8A5/3fixNovI6hArW/9qfD+a+ScC+Q2YMAAJ9LQzJjhXE6Oc2b+74wZyXv9jBnONW/unI8jf2vePPZl5ORUfG34lpOTnPUHXX/Q7z8MKHRV5KoCXSRJ0j3QzKK/3iw56w+6/qDff1h1ga4uF5EY1ffn8tSpsG9fxWn79vnpyXj9xx/XbnplXbrUbnq81x90/UG//1go0EViEI/+03QPtOnToXnzitOaN/fTk7H+oOsP+v3HpKqme6Jv6nKRZKtP/3M8fi4H3eUQjz7cIPvwg66/vq9XH7pInNT3P1N9+0/jUUMqBFp9BRmoqSAe9SvQJSME2cKO1w6thh5oUn/VBbr555MvPz/fafhciVW4Dztyp2Dz5lBQAOPH1/z6Ro18BFdmBmVliV+/SLyY2SrnXH6057RTVNJCfY/wqO8OqfHjfXjn5PgvgZwchbmkHgW6pIX6HuFR3yMUwId3UZFv0RcVKcwl9SjQJS2ohS1SMwW6pAW1sEVqpkCXpKnPmZZqYYvULLCLREvDUvkokfCZlhB7KI8frwAXqY5a6JIU9T1KRURqpkCXpEjGwEQiDZ0CXZIiKQMTiTRwCnSJWX12asbjKBURqZ4CXWJS3+FjdZSKSOJpLBeJSW6uD/HKcnL8Md0ikhway0XqTTs1RVKfAl1iop2aIqlPgS4x0U5NkdQXU6Cb2XAze9/MNpnZlCjP55jZy2a2zsyWmVmn+JcqQdJOTZHUV+NOUTPLAj4AvgcUAyuBcc65DRHzzAH+5pz7s5mdDVzlnPtxdcvVTlERkdqr707RgcAm59wW59x+YDYwutI8vYClofuvRHleREQSLJZA7whsi3hcHJoWaS0wJnT/QqCVmbWtf3kST/U5MUhEUl+8doreAgwxs7eBIcAnwMHKM5nZRDMrNLPCHTt2xGnVEov6nhgkIqkvlkD/BOgc8bhTaFo559ynzrkxzrl+wNTQtF2VF+ScK3DO5Tvn8tu1a1ePsqW2NNqhSOaLJdBXAt3MrKuZHQFcCsyPnMHMss0svKxfAU/Et0ypL50YJJL5agx051wpcD2wGHgPeMY5t97M7jKzUaHZhgLvm9kHwHGAjk5OMToxSCTzxXTFIufcImBRpWl3RNyfC8yNb2kST9OnV7xiEOjEIJFMozNFGwidGCSS+XRN0QZE1+QUyWxqoYuIZAgFuohIhlCgi4hkCAW6iEiGUKCLiGQIBXoa0eBaIlIdHbaYJsKDa4VPDAoPrgU6FFFEPLXQ04QG1xKRmijQ04QG1xKRmijQ04QG1xKRmijQ08T06X4wrUgaXEtEIinQ04QG1xKRmugolzSiwbVEpDpqoYuIZAgFuohIhlCgi4hkCAW6iEiGUKCLiGQIBbqISIZQoIuIZAgFuohIhlCgi4hkCAV6EukCFSKSSDr1P0l0gQoRSTS10JNEF6gQkURToCeJLlCRGsrKYNMm2LkTnAu6GpH4UpdLknTp4rtZok2XxCorg9deg7lz4dln4ZNP/PTmzaFTp0O3zp0Pv9+mjR+uWCQdKNCTZPr0in3ooAtUJNLBg/CPfxwK8c8+g2bNYMQIGD4c/vlPKC4+dHvlFfj0U/+6SM2aHR72lR9nZyv0JTUo0JMkvONz6lTfzdKliw9z7RCNn4MH4dVXfYj/9a/w+ec+kM87D8aOhfPPh1atqn/99u0+4Ldtqxj427b5ZX/yCZSWVnxd06YVwz5a6Ldr549uSgX//KdvTOhLKPOYC6gjMT8/3xUWFgaybskcpaWwfPmhEP/iCzjySB/eY8f6MG/ZMn7rKys7FPqVAz/y8YEDFV93xBHQsaMP95wc/0th5Eg46qj41VadnTv9NpoxA1asgNatoVcv6N3b/w3f79hRQZ/qzGyVcy4/6nMKdEk3paW+i2TuXJg3D3bs8C3OkSPh4ot9iLdoEVx9ZWXw5ZfRW/nFxbBxo/9SaNoUfvAD/8Xzwx/6kI2nb7+FhQt9iC9c6L9kevaECy+EXbtg/XrYsMFvv7BWrSoGffhv584NM+h37YLNmyvetm2DHj1gyBA46yzf5ZZMCnQJxFdf+f8AmzYd+s+waZMPtbKyui931y7YvduH9siRPhBHjDj8ItqpqqwM3nwT5szxX0rFxb4F//3v+/cyahQcfXTdl/3qqz7E587126l9e7jsMrj8cujb9/Bg3rHDB/uGDYdCfv16/2snrGXLQy35nj39iXHhbqUOHaBxmnbeOue/XCt/RsP3S0oqzn/ccf5XzMaNh/aHnXQSDB3qA37IEN+9lkgK9Absq6/g7bcP3Vavhi1bfGAce6z/8EXeKk879lg45pjo/b/O+Z2Nlf8ThB9/9VXF+Tt0gO9+1+8/qE8ANGvmw2/ECN+9ks7KyuCttw6F+8cfQ5Mm/v1dfDGMHu23f03eecefvDZzpv+CaNkSxozxIX722ZCVVfvavvzyUNBHhv3nn1ecr1Ej/6VR3c7j73zHv68glJb6VnW00N6yxe9TiHwvXbrACSf4z+oJJxy6f/zxh7rv9u+HwkLf3bdsmd8BHw74Xr0qBvxxx8X3/SjQG4jPPvOBHRneRUWHnu/UCfr1gxNP9C23HTsq3nbtir7crCxo2/ZQyLdo4Q/B3LwZvvmm4nw5OVX/Z0iXFnRQnIOVK324z5njt3GTJnDuuT7cL7jAH0YZVlwMs2b51vi6dX77Dx/uQ3zUqMRt7127DnUfVe5WCk/bu7fia8wOhX6nThUbDpUbEdnZtQ//b76BrVuj/yIsKqq4I7tpU/95jPx8hu/n5vpfS7V14ACsWuXDfflyH/DhbdCjhw/4cMi3b1/75Ueqd6Cb2XDgQSALeMw5d0+l57sAfwaODs0zxTm3qLplKtDrzjn/4Q2Hdvjv9u2H5unWzYd3//7+b79+Nf8U3L/ft8oqB/2OHf7nd/j+3r3RWzE5OcG1wjKNc74FOHeuD/etW/2vmnPO8aGwZInfj+AcDBrkQ/ySS3w4Bs05+PrrqncYFxf7z1NJSdVdb0cfHT3s27Xzv1g+/bRicBcXV3z9UUcd3qgI3+/YMfFHHB044P9PRrbg9+zxz3XvDnff7b+k66JegW5mWcAHwPeAYmAlMM45tyFingLgbefcH82sF7DIOZdb3XIV6LVTWgpLl/rW2IIFh1rTWVl+x1U4tPv3h7y85B09IYnnnA+HcLhv3uwD6vLLfd94t25BV1g3Bw/6brnIxkK0BkTkLfILoH37QyFdObTbtk2tnbilpb7hFQ74G2+E732vbsuqLtBj6ckcCGxyzm0JLWw2MBrYEDGPA8IR0hr4tG6lSqTwf+QZM/xP6+3bfcvlggvgtNN8eJ90ku9TlsxlBgMG+Nt//Ifvw27fPrUCqy6ysnz3SqxHiZSV+S+Ar77y7z+eh6MmWuPGcMop/nbLLQlcTwzzdAS2RTwuBgZVmmca8Hcz+znQAjg32oLMbCIwEaCLznmv0tath3Zwbdzo+/RGjvQtsvPO832A0jCZ+Z3LDVGjRr7l3bZt0JWkrngdbDQOeNI59zszOw34i5md5Jyr0EPmnCsACsB3ucRp3RmhpMT/nJ4xw487Av4Y15tv9n1tsRzpICINWyyB/gnQOeJxp9C0SD8FhgM4594ws2ZANvAFUqVvvoG//c2H+Asv+B0pvXrBb38L48b5nYwiIrGKJdBXAt3MrCs+yC8FLqs0z8fAOcCTZtYTaAbsQA7jnN8pEj7x4+uv/TG6N97ox3XJy0v/vlERCUaNB+8450qB64HFwHvAM8659WZ2l5mNCs32C+BqM1sLzAKudEEd4J5A9b2EXGkpXHGFP9Fjzhx/4seSJf5kknvvjX4Wn4hIrHRiUYwqX0IO/IkbBQWxjZj47bdw6aXw/PNwxx0wZUr6n+UoIslX3WGLKTKgZ+qrzyXk9u71o/89/zz8/vfw7/+uMBeR+EvTIXWSr66XkNu50x9qWFgIf/4z/OQn8a9NRATUQo9ZVYfNV3c4/eef+/Eb3n7b7wBVmItIIinQYzR9+uGDHVV3CbmiIhg82I/mtnChP7tTRCSRFOgxGj/e7wDNyfFHouTkVL1DdONGH+YlJfDSS360PBGRRFMfei2MH1/zES2rV/ur0GRl+YF4Tj45ObWJiKiFHkcrVsCwYb4rZsUKhbmIJJcCPU5efNG3zDt08GMfp+uQpiKSvhTocTBnjr9CTI8evmXeuXPNrxERiTcFej09/rg/A3TQIH8FmURfIFZEpCoK9Hq4/3742c/8BX0XL4bWrYOuSEQaMgV6HTjnx2P5xS9g7Fh/Sr8ugCwiQdNhi7VUVgaTJ/sxWX76U/jf//WHKIqIBE0t9Fr65S99mN98Mzz6qMJcRFKHAr0WCgp8v/nPfw733aexy0UktSjQY/Tyy3DddTBihA91hbmIpBoFegzef99fqLl7d5g9Gxprz4OIpCAFeg1KSmDkSGjSBBYsgKOOCroiEZHo1Nasxv79vmX+8cf+pKGuXYOuSESkagr0KjgHkybBsmUwYwacfnrQFYmIVE9dLlX43e/giSfg9ttjuwi0iEjQFOhRPP883HorXHIJTJsWdDUiIrFRoFeyZg1cdhnk58OTT0IjbSERSROKqwiffQY//CG0aeNb6UceGXRFIiKx007RkH37YPRo+Oorf4GKDh2CrkhEpHYU6PgBt664AgoL4bnnoG/foCsSEak9BTpw550wd64fn2XUqKCrERGpmwbfhz5jBtx9tx8K9+abg65GRKTuGnSgv/aaD/KhQ+GRRzTgloiktwYV6DNnQm6uPxSxY0c47zzIyYFnn4Ujjgi6OhGR+mkwfegzZ8LEif5oFoBPP/V/b7/dH6YoIpLuGkwLferUQ2Ee6Q9/SH4tIiKJ0GAC/eOPazddRCTdNJhA79KldtNFRNJNgwn06dMPv6Bz8+Z+uohIJogp0M1suJm9b2abzGxKlOf/28zWhG4fmNmu+JdaPyecAAcP+isOmfmjWwoKNDSuiGSOGo9yMbMs4GHge0AxsNLM5jvnNoTncc7dFDH/z4F+Cai1zsrK4MYb/fgsH3wALVsGXZGISPzF0kIfCGxyzm1xzu0HZgOjq5l/HDArHsXFy8yZ8NZbcM89CnMRyVyxBHpHYFvE4+LQtMOYWQ7QFVhaxfMTzazQzAp37NhR21rrZO9emDIFBg6Eyy9PyipFRAIR752ilwJznXMHoz3pnCtwzuU75/LbtWsX51VHd889/iSiBx7QxSpEJLPFEnGfAJ0jHncKTYvmUlKou6WoyI+gOH48nHZa0NWIiCRWLIG+EuhmZl3N7Ah8aM+vPJOZ9QCOAd6Ib4l1d+ut/lDFe+4JuhIRkcSrMdCdc6XA9cBi4D3gGefcejO7y8wiRw+/FJjtnHOJKbV2li+HOXPg3/4NOnUKuhoRkcSzoPI3Pz/fFRYWJmTZBw/6izyXlMDGjf4EIhGRTGBmq5xz+dGey8jRFv/0J1izBmbPVpiLSMORccd97N4Nt90GgwfDJZcEXY2ISPJkXAv97rvhyy/hhRd0BSIRaVgyqoX+4Yfw4INw1VUwYEDQ1YiIJFdGBfovfgHNmmkERRFpmDKmy+Wll2DBAn/Mefv2QVcjIpJ8GdFCLy2FyZP9ELmTJwddjYhIMDKihf4//wMbNsC8edC0adDViIgEI+1b6Dt3wp13wtlnw+jqBvUVEclwaR/o06bBrl1+NEUdpigiDVlaB/r69fDII3DNNdCnT9DViIgEK20D3Tm4+WZo1QruuivoakREgpe2O0UXLoS//x3++78hOzvoakREgpeWLfT9+33rvEcPuO66oKsREUkNadlC//3v/Wn+ixZBkyZBVyMikhrSroX+xRe+z3zECH8TEREv7QL9oYdg3z64//6gKxERSS1p1+Vyxx1w7rm+/1xERA5Juxb6EUfA0KFBVyEiknrSLtBFRCQ6BbqISIZQoIuIZAgFuohIhlCgi4hkCAW6iEiGUKCLiGQIBbqISIZQoIuIZAgFuohIhlCgi4hkCAW6iEiGUKCLiGQIBbqISIZQoIuIZAgFuohIhogp0M1suJm9b2abzGxKFfNcYmYbzGy9mf2/+JYpIiI1qfESdGaWBTwMfA8oBlaa2Xzn3IaIeboBvwLOcM59ZWbHJqpgERGJLpYW+kBgk3Nui3NuPzAbGF1pnquBh51zXwE4576Ib5kiIlKTWAK9I7At4nFxaFqkE4ETzew1M3vTzIZHW5CZTTSzQjMr3LFjR90qFhGRqOK1U7Qx0A0YCowDHjWzoyvP5JwrcM7lO+fy27VrF6dVi4gIxBbonwCdIx53Ck2LVAzMd84dcM5tBT7AB7yIiCRJLIG+EuhmZl3N7AjgUmB+pXmew7fOMbNsfBfMljjWKSIiNagx0J1zpcD1wGLgPeAZ59x6M7vLzEaFZlsMlJjZBuAV4JfOuZJEFS0iIocz51wgK87Pz3eFhYWBrFtEJF2Z2SrnXH6053SmqIhIhlCgi4hkCAW6iEiGUKCLiGQIBbqISIZQoIuIZIgaR1sUkcQ6cOAAxcXFfPvtt0GXIimkWbNmdOrUiSZNmsT8GgW6SMCKi4tp1aoVubm5mFnQ5UgKcM5RUlJCcXExXbt2jfl16nIRCdi3335L27ZtFeZSzsxo27ZtrX+1KdBFUoDCXCqry2dCgS4ikiEU6CJpZuZMyM2FRo3835kz67e8kpIS+vbtS9++fWnfvj0dO3Ysf7x///5qX1tYWMgNN9xQ4zpOP/30+hVZyeTJk+nYsSNlZWVxXW66005RkTQycyZMnAj79vnHH33kHwOMH1+3ZbZt25Y1a9YAMG3aNFq2bMktt9xS/nxpaSmNG0ePivz8fPLzo44TVcHrr79et+KiKCsrY968eXTu3Jnly5czbNiwuC07UnXvO1WphS6SRqZOPRTmYfv2+enxdOWVV3LttdcyaNAgbr31Vt566y1OO+00+vXrx+mnn877778PwLJlyxg5ciTgvwwmTJjA0KFDOf7443nooYfKl9eyZcvy+YcOHcrFF19Mjx49GD9+POERXxctWkSPHj0YMGAAN9xwQ/lyK1u2bBm9e/dm0qRJzJo1q3z69u3bufDCC8nLyyMvL6/8S+Spp57i5JNPJi8vjx//+Mfl72/u3LlR6zvzzDMZNWoUvXr1AuCCCy5gwIAB9O7dm4KCgvLXvPjii/Tv35+8vDzOOeccysrK6NatG+HLa5aVlfHd736XZF5uM72+fkQauI8/rt30+iguLub1118nKyuLr7/+mhUrVtC4cWOWLFnCbbfdxrPPPnvYazZu3Mgrr7zCnj176N69O5MmTTrsOOq3336b9evX853vfIczzjiD1157jfz8fK655hpeffVVunbtyrhx46qsa9asWYwbN47Ro0dz2223ceDAAZo0acINN9zAkCFDmDdvHgcPHmTv3r2sX7+eu+++m9dff53s7Gx27txZ4/tevXo17777bvnhgk888QRt2rThm2++4ZRTTuGiiy6irKyMq6++urzenTt30qhRIy6//HJmzpzJ5MmTWbJkCXl5eSTzcptqoYukkS5daje9PsaOHUtWVhYAu3fvZuzYsZx00kncdNNNrF+/Puprzj//fJo2bUp2djbHHnss27dvP2yegQMH0qlTJxo1akTfvn0pKipi48aNHH/88eUhWlWg79+/n0WLFnHBBRdw1FFHMWjQIBYvXgzA0qVLmTRpEgBZWVm0bt2apUuXMnbsWLKzswFo06ZNje974MCBFY79fuihh8jLy+PUU09l27ZtfPjhh7z55pucddZZ5fOFlzthwgSeeuopwH8RXHXVVTWuL54U6CJpZPp0aN684rTmzf30eGvRokX5/dtvv51hw4bx7rvvsmDBgiqPj27atGn5/aysLEpLS+s0T1UWL17Mrl276NOnD7m5ufzjH/+o0O0Sq8aNG5fvUC0rK6uw8zfyfS9btowlS5bwxhtvsHbtWvr161ftseGdO3fmuOOOY+nSpbz11luMGDGi1rXVhwJdJI2MHw8FBZCTA2b+b0FB3XeIxmr37t107NgRgCeffDLuy+/evTtbtmyhqKgIgKeffjrqfLNmzeKxxx6jqKiIoqIitm7dyksvvcS+ffs455xz+OMf/wjAwYMH2b17N2effTZz5syhpMRfETPc5ZKbm8uqVasAmD9/PgcOHIi6vt27d3PMMcfQvHlzNm7cyJtvvgnAqaeeyquvvsrWrVsrLBfgZz/7GZdffnmFXzjJokAXSTPjx0NREZSV+b+JDnOAW2+9lV/96lf069evVi3qWB155JE88sgjDB8+nAEDBtCqVStat25dYZ59+/bx4osvcv7555dPa9GiBYMHD2bBggU8+OCDvPLKK/Tp04cBAwawYcMGevfuzdSpUxkyZAh5eXncfPPNAFx99dUsX76cvLw83njjjQqt8kjDhw+ntLSUnj17MmXKFE499VQA2rVrR0FBAWPGjCEvL48f/ehH5a8ZNWoUe/fuTXp3C+iaoiKBe++99+jZs2fQZQRu7969tGzZEucc1113Hd26deOmm24KuqxaKyws5KabbmLFihX1Xla0z4auKSoiKe/RRx+lb9++9O7dm927d3PNNdcEXVKt3XPPPVx00UX89re/DWT9aqGLBEwtdKmKWugiIg2UAl1EJEMo0EVEMoQCXUQkQyjQRRq4YcOGlZ8+H/bAAw+Un0YfzdChQwkf1HDeeeexa9euw+aZNm0a9913X7Xrfu6559iwYUP54zvuuIMlS5bUpvxqNbRhdhXoIg3cuHHjmD17doVps2fPrnaArEiLFi3i6KOPrtO6Kwf6XXfdxbnnnlunZVVWeZjdREnEiVZ1pUAXSSGTJ8PQofG9TZ5c/TovvvhiFi5cWD6eSVFREZ9++ilnnnkmkyZNIj8/n969e3PnnXdGfX1ubi5ffvklANOnT+fEE09k8ODB5UPsgj/G/JRTTiEvL4+LLrqIffv28frrrzN//nx++ctf0rdvXzZv3lxhWNuXX36Zfv360adPHyZMmMC//vWv8vXdeeed9O/fnz59+rBx48aodTXEYXYV6CINXJs2bRg4cCAvvPAC4Fvnl1xyCWbG9OnTKSwsZN26dSxfvpx169ZVuZxVq1Yxe/Zs1qxZw6JFi1i5cmX5c2PGjGHlypWsXbuWnj178vjjj3P66aczatQo7r33XtasWcMJJ5xQPv+3337LlVdeydNPP80777xDaWlp+TgtANnZ2axevZpJkyZV2a0THmb3wgsvZOHCheXjtYSH2V27di2rV6+md+/e5cPsLl26lLVr1/Lggw/WuN1Wr17Ngw8+yAcffAD40RVXrVpFYWEhDz30ECUlJezYsYOrr76aZ599lrVr1zJnzpwKw+wCcR1mV+Ohi6SQBx4IZr3hbpfRo0cze/ZsHn/8cQCeeeYZCgoKKC0t5bPPPmPDhg2cfPLJUZexYsUKLrzwQpqHhoMcNWpU+XPvvvsuv/71r9m1axd79+7lBz/4QbX1vP/++3Tt2pUTTzwRgCuuuIKHH36YyaGfG2PGjAFgwIAB/PWvfz3s9eFhdu+//35atWpVPszuyJEjWbp0afkQt+Fhdp966qm4DLM7b948gPJhdnfs2FHlMLujR49m8uTJcR1mN61a6PG+lqKIeKNHj+bll19m9erV7Nu3jwEDBrB161buu+8+Xn75ZdatW8f5559f7dCx1bnyyiv5wx/+wDvvvMOdd95Z5+WEhYfgrWr43YY6zG7aBHr4WooffQTOHbqWokJdpP5atmzJsGHDmDBhQvnO0K+//poWLVrQunVrtm/fXt4lU5WzzjqL5557jm+++YY9e/awYMGC8uf27NlDhw4dOHDgQHlXA0CrVq3Ys2fPYcvq3r07RUVFbNq0CYC//OUvDBkyJOb301CH2U2bQE/WtRRFGqpx48axdu3a8kDPy8ujX79+9OjRg8suu4wzzjij2tf379+fH/3oR+Tl5TFixAhOOeWU8ud+85vfMGjQIM444wx69OhRPv3SSy/l3nvvpV+/fmzevLl8erNmzfjTn/7E2LFj6dOnD40aNeLaa6+N6X005GF202ZwrkaNfMu8MjM/LrRIutLgXA1TLMPsJmRwLjMbbmbvm9kmM5sS5fkrzWyHma0J3X4Wy3JrI5nXUhQRSaREDbNbY6CbWRbwMDAC6AWMM7NeUWZ92jnXN3R7LK5VktxrKYqIJNKUKVP46KOPGDx4cFyXG0sLfSCwyTm3xTm3H5gNjI5rFTEI6lqKIskQVNenpK66fCZiCfSOwLaIx8WhaZVdZGbrzGyumXWOtiAzm2hmhWZWWJezooK4lqJIojVr1oySkhKFupRzzlFSUkKzZs1q9bp4nVi0AJjlnPuXmV0D/Bk4u/JMzrkCoAD8TtE4rVskrXXq1Ini4uK4nPotmaNZs2Z06tSpVq+JJdA/ASJb3J1C08o550oiHj4G/FetqhBpwJo0aVLhjEORuoqly2Ul0M3MuprZEcClwPzIGcysQ8TDUcB78StRRERiUWML3TlXambXA4uBLOAJ59x6M7sLKHTOzQduMLNRQCmwE7gygTWLiEgUaXNikYiIVH9iUWCBbmY7gI8CWQ8hp9kAAASmSURBVHnNsoEvgy6iGqqvflK9Pkj9GlVf/dSnvhznXNSxdgML9FRmZoVVfQOmAtVXP6leH6R+jaqvfhJVX9oMziUiItVToIuIZAgFenQFNc8SKNVXP6leH6R+jaqvfhJSn/rQRUQyhFroIiIZQoEuIpIhGmygm1lnM3vFzDaY2XozuzHKPEPNbHfEhTvuSHKNRWb2Tmjdh52FZd5DoQuPrDOz/kmsrXvEdlljZl+b2eRK8yR9+5nZE2b2hZm9GzGtjZm9ZGYfhv4eU8VrrwjN86GZXZGk2u41s42hf795ZnZ0Fa+t9rOQ4BqnmdknEf+O51Xx2movhJPA+p6OqK3IzNZU8dqEbsOqMiWpnz/nXIO8AR2A/qH7rYAPgF6V5hkK/C3AGouA7GqePw94ATDgVOD/AqozC/gcf8JDoNsPOAvoD7wbMe2/gCmh+1OA/4zyujbAltDfY0L3j0lCbd8HGofu/2e02mL5LCS4xmnALTF8BjYDxwNHAGsr/39KVH2Vnv8dcEcQ27CqTEnm56/BttCdc58551aH7u/BDygWbZz3VDYaeMp5bwJHVxooLVnOATY75wI/89c59yp+PKFIo/FDOhP6e0GUl/4AeMk5t9M59xXwEjA80bU55/7unCsNPXwTP5ppYKrYfrFIyoVwqqvPzAy4BJgV7/XGoppMSdrnr8EGeiQzywX6Af8X5enTzGytmb1gZr2TWhg44O9mtsrMJkZ5PtaLjyTapVT9nyjI7Rd2nHPus9D9z4HjosyTCttyAv4XVzQ1fRYS7fpQt9ATVXQZpML2OxPY7pz7sIrnk7YNK2VK0j5/DT7Qzawl8Cww2Tn3daWnV+O7EfKA3wPPJbm8wc65/vjruV5nZmclef01Mj+k8ihgTpSng95+h3H+923KHatrZlPxo5XOrGKWID8LfwROAPoCn+G7NVLROKpvnSdlG1aXKYn+/DXoQDezJvgNP9M599fKzzvnvnbO7Q3dXwQ0MbPsZNXnnPsk9PcLYB7+Z22kGi8+kgQjgNXOue2Vnwh6+0XYHu6KCv39Iso8gW1LM7sSGAmMD/2HP0wMn4WEcc5td84ddM6VAY9Wse5AP4tm1hgYAzxd1TzJ2IZVZErSPn8NNtBD/W2PA+855+6vYp72ofkws4H47VUSbd4E1NfCzFqF7+N3nr1babb5wE9CR7ucCuyO+GmXLFW2ioLcfpXMB8JHDVwBPB9lnsXA983smFCXwvdD0xLKzIYDtwKjnHP7qpgnls9CImuM3C9zYRXrrvFCOAl2LrDROVcc7clkbMNqMiV5n79E7fFN9RswGP/TZx2wJnQ7D7gWuDY0z/XAevwe+zeB05NY3/Gh9a4N1TA1ND2yPgMexh9d8A6Qn+Rt2AIf0K0jpgW6/fBfLp8BB/D9kD8F2gIvAx8CS4A2oXnzgcciXjsB2BS6XZWk2jbh+07Dn8H/Cc37HWBRdZ+FJG6/v4Q+X+vw4dShco2hx+fhj+zYnKgao9UXmv5k+HMXMW9St2E1mZK0z59O/RcRyRANtstFRCTTKNBFRDKEAl1EJEMo0EVEMoQCXUQkQyjQRUQyhAJdRCRD/H+IgWNaQNWgBwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "plt.clf()\n",
        "plt.plot(epochs, loss, \"bo\", label = \"Training Loss\")\n",
        "plt.plot(epochs, val_loss, \"b\", label = \"Validation Loss\")\n",
        "plt.plot(title = \"Multiclass Classifaction Loss\")\n",
        "plt.plot(xlabel = \"Epoch\")\n",
        "plt.plot(ylabel = \"Loss\")\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "BCc3Lre1f-uQ",
        "outputId": "251da6c3-f956-49ba-e4e6-178899c5464c"
      },
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZhU5Zn38e/NboMomwtrgwoEZW/AXXiZoFECESHKkFFCIuKlEkmiZmJGnSTMqMObOCRRXxKNJvaIW0IgYNyiwcRXY0PAQMAICIoLQhtZREDgnj+eanqhqruaWk7X6d/nus5VVeecqnN3dfWvTj/nOc8xd0dERApfk6gLEBGR7FCgi4jEhAJdRCQmFOgiIjGhQBcRiYlmUW24Y8eOXlxcHNXmRUQK0rJly7a5e6dkyyIL9OLiYsrKyqLavIhIQTKzTamWqclFRCQmFOgiIjGhQBcRiYnI2tBFJD8+/fRTNm/ezJ49e6IuReqhVatWdO3alebNm6f9HAW6SMxt3ryZo48+muLiYsws6nIkDe5OeXk5mzdvpmfPnmk/r6CaXEpLobgYmjQJt6WlUVck0vDt2bOHDh06KMwLiJnRoUOHev9XVTB76KWlMH067N4dHm/aFB4DTJkSXV0ihUBhXniO5HdWMHvoN99cGeYVdu8O80VEpIAC/a236jdfRBqG8vJyBg0axKBBgzjhhBPo0qXLocf79u2r9bllZWXMnDmzzm2ceeaZWan1hRdeYOzYsVl5rSgUTKB3716/+SJyZLJ9rKpDhw6sWLGCFStWMGPGDGbNmnXocYsWLdi/f3/K55aUlDB37tw6t/HSSy9lVmRMFEygz54NRUXV5xUVhfkikh0Vx6o2bQL3ymNV2e6AMHXqVGbMmMGIESO48cYb+fOf/8wZZ5zB4MGDOfPMM3n99deB6nvMt912G9OmTWPkyJH06tWrWtC3adPm0PojR45k4sSJ9O3blylTplBxVbYlS5bQt29fhg4dysyZM+u1J/7www/Tv39/TjvtNG666SYADhw4wNSpUznttNPo378/P/zhDwGYO3cu/fr1Y8CAAVx22WWZv1n1UDAHRSsOfN58c2hm6d49hLkOiIpkT23HqrL9t7Z582ZeeuklmjZtyo4dO3jxxRdp1qwZzz77LN/+9rd54oknDnvO2rVref7559m5cyd9+vTh6quvPqyf9l/+8hdWr15N586dOeuss/jTn/5ESUkJV111FUuXLqVnz55Mnjw57TrfffddbrrpJpYtW0a7du0YM2YMCxYsoFu3brzzzjusWrUKgI8++giA22+/nTfffJOWLVsempcvBbOHDuEDtXEjHDwYbhXmItmVz2NVkyZNomnTpgBs376dSZMmcdpppzFr1ixWr16d9DkXXXQRLVu2pGPHjhx33HFs2bLlsHWGDx9O165dadKkCYMGDWLjxo2sXbuWXr16HerTXZ9Af/XVVxk5ciSdOnWiWbNmTJkyhaVLl9KrVy82bNjAddddx+9+9zvatm0LwIABA5gyZQoPPfQQzZrld5+5oAJdRHIrn8eqWrdufej+v/3bvzFq1ChWrVrFokWLUva/btmy5aH7TZs2Tdr+ns462dCuXTtWrlzJyJEjuffee/nqV78KwOLFi7nmmmtYvnw5w4YNy9n2k1Ggi8ghUR2r2r59O126dAHggQceyPrr9+nThw0bNrBx40YAHnnkkbSfO3z4cP7whz+wbds2Dhw4wMMPP8x5553Htm3bOHjwIJdccgnf//73Wb58OQcPHuTtt99m1KhR3HHHHWzfvp1du3Zl/edJpWDa0EUk96I6VnXjjTdyxRVX8P3vf5+LLroo669/1FFHcffdd3PBBRfQunVrhg0blnLd5557jq5dux56/Nhjj3H77bczatQo3J2LLrqI8ePHs3LlSr785S9z8OBBAP7zP/+TAwcO8KUvfYnt27fj7sycOZNjjz026z9PKlZxBDjlCmbdgF8AxwMOzHP3/66xzkjgN8CbiVm/cvfv1va6JSUlrgtciOTemjVr+MxnPhN1GZHbtWsXbdq0wd255pprOOWUU5g1a1bUZdUq2e/OzJa5e0my9dPZQ98PfMPdl5vZ0cAyM3vG3f9WY70X3b1we+SLSKz99Kc/5cEHH2Tfvn0MHjyYq666KuqSsq7OQHf394D3Evd3mtkaoAtQM9BFRBqsWbNmNfg98kzV66ComRUDg4FXkiw+w8xWmtmTZnZqiudPN7MyMyvbunVrvYsVEZHU0g50M2sDPAFc7+47aixeDvRw94HAj4AFyV7D3ee5e4m7l3TqlPSi1SIicoTSCnQza04I81J3/1XN5e6+w913Je4vAZqbWcesVioiIrWqM9AtDMp7H7DG3X+QYp0TEuthZsMTr1uezUJFRKR26eyhnwX8C/B/zGxFYrrQzGaY2YzEOhOBVWa2EpgLXOZ19YcUkUZh1KhRPPXUU9Xm3XXXXVx99dUpnzNy5EgqujVfeOGFScdEue2225gzZ06t216wYAF/+1tl/41bbrmFZ599tj7lJ9VQh9lNp5fLH4FaL53h7j8GfpytokQkPiZPnsz8+fM5//zzD82bP38+d955Z1rPX7JkyRFve8GCBYwdO5Z+/foB8N3v1np6TMHTqf8iklMTJ05k8eLFhy5msXHjRt59913OOeccrr76akpKSjj11FO59dZbkz6/uLiYbdu2ATB79mx69+7N2WeffWiIXQh9zIcNG8bAgQO55JJL2L17Ny+99BILFy7khhtuYNCgQaxfv56pU6fy+OOPA+GM0MGDB9O/f3+mTZvG3r17D23v1ltvZciQIfTv35+1a9em/bNGPcyuTv0XaUSuvx5WrMjuaw4aBHfdlXp5+/btGT58OE8++STjx49n/vz5fPGLX8TMmD17Nu3bt+fAgQOMHj2a1157jQEDBiR9nWXLljF//nxWrFjB/v37GTJkCEOHDgVgwoQJXHnllQB85zvf4b777uO6665j3LhxjB07lokTJ1Z7rT179jB16lSee+45evfuzeWXX84999zD9ddfD0DHjh1Zvnw5d999N3PmzOFnP/tZne9DQxhmV3voIpJzFc0uEJpbKoavffTRRxkyZAiDBw9m9erV1dq7a3rxxRe5+OKLKSoqom3btowbN+7QslWrVnHOOefQv39/SktLUw6/W+H111+nZ8+e9O7dG4ArrriCpUuXHlo+YcIEAIYOHXpoQK+6NIRhdrWHLtKI1LYnnUvjx49n1qxZLF++nN27dzN06FDefPNN5syZw6uvvkq7du2YOnVqymFz6zJ16lQWLFjAwIEDeeCBB3jhhRcyqrdiCN5sDL9bMczuU089xb333sujjz7K/fffz+LFi1m6dCmLFi1i9uzZ/PWvf8042LWHLiI516ZNG0aNGsW0adMO7Z3v2LGD1q1bc8wxx7BlyxaefPLJWl/j3HPPZcGCBXzyySfs3LmTRYsWHVq2c+dOTjzxRD799FNKq1wv7+ijj2bnzp2HvVafPn3YuHEj69atA+CXv/wl5513XkY/Y0MYZld76CKSF5MnT+biiy8+1PQycOBABg8eTN++fenWrRtnnXVWrc8fMmQIl156KQMHDuS4446rNgTu9773PUaMGEGnTp0YMWLEoRC/7LLLuPLKK5k7d+6hg6EArVq14uc//zmTJk1i//79DBs2jBkzZhy2zdo0xGF26xw+N1c0fK5Ifmj43MJV3+Fz1eQiIhITCnQRkZhQoIs0AhqJo/Acye9MgS4Sc61ataK8vFyhXkDcnfLyclq1alWv56mXi0jMde3alc2bN6OLyhSWVq1aVetFkw4FukjMNW/enJ49e0ZdhuSBmlxERGJCgS4iEhMKdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISEwo0EVEYqLOQDezbmb2vJn9zcxWm9nXkqxjZjbXzNaZ2WtmNiQ35YqISCrpXIJuP/ANd19uZkcDy8zsGXf/W5V1PgeckphGAPckbkVEJE/q3EN39/fcfXni/k5gDdClxmrjgV948DJwrJmdmPVqRUQkpXq1oZtZMTAYeKXGoi7A21Ueb+bw0MfMpptZmZmV6QrkIiLZlXagm1kb4AngenffcSQbc/d57l7i7iWdOnU6kpcQEZEU0gp0M2tOCPNSd/9VklXeAbpVedw1MU9ERPIknV4uBtwHrHH3H6RYbSFweaK3y+nAdnd/L4t1iohIHdLp5XIW8C/AX81sRWLet4HuAO5+L7AEuBBYB+wGvpz9UkVEpDZ1Brq7/xGwOtZx4JpsFSUiIvWnM0VFRGJCgS4iEhMKdBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXUQkJgou0Hfvhv/5H3CPuhIRkYal4AL9kUdgyhT4/e+jrkREpGEpuECfPBmOOw5+kOraSSIijVTBBXqrVnDNNbBkCaxZE3U1IiINR8EFOsDVV4dg/+EPo65ERKThKMhA79QJLr8cfvEL+OCDqKsREWkYCjLQAWbNgr174Z57oq5ERKRhKNhA79sXLroIfvIT2LMn6mpERKJXsIEO8PWvw9at8NBDUVciIhK9gg70UaNg0KDQhVEnGolIY1fQgW4W9tLXrIHf/S7qakREolXQgQ5w6aXQubNONBIRKfhAb9ECrrsOnn0WVq6MuhoRkegUfKADXHUVFBXpRCMRadxiEejt2sG0aWEUxnffjboaEZFoxCLQAa6/HvbvD/3SRUQao9gE+kknwRe+EM4c/fjjqKsREcm/OgPdzO43sw/MbFWK5SPNbLuZrUhMt2S/zPR84xvwj3/Agw9GVYGISHTS2UN/ALigjnVedPdBiem7mZd1ZM48E4YPDwdHDxyIqgoRkWjUGejuvhT4MA+1ZMws7KWvWwe//W3U1YiI5Fe22tDPMLOVZvakmZ2aaiUzm25mZWZWtnXr1ixturoJE6BHD51oJCKNTzYCfTnQw90HAj8CFqRa0d3nuXuJu5d06tQpC5s+XLNm8LWvwdKlUFaWk02IiDRIGQe6u+9w912J+0uA5mbWMePKMvCVr0DbttpLF5HGJeNAN7MTzMwS94cnXrM809fNRNu2cOWV8Oij8NZbUVYiIpI/6XRbfBj4/0AfM9tsZl8xsxlmNiOxykRglZmtBOYCl7lHP5jtzJnh9kc/irYOEZF8SaeXy2R3P9Hdm7t7V3e/z93vdfd7E8t/7O6nuvtAdz/d3V/Kfdl1694dJk2CefNgx44wr7QUiouhSZNwW1oaZYUiItnVLOoCcunrX4f58+H++8OFpadPh927w7JNm8JjgClToqtRRCRbLKrWkZKSEi/LQzeUc88N7ejuydvTe/SAjRtzXoaISFaY2TJ3L0m2LDZjuaTy9a+HvfFUB0d10FRE4iL2gf75z8PJJ4cLYSTTvXt+6xERyZXYB3rTpmFo3X37oGXL6suKimD27GjqEhHJttgHOsDUqeEiGAMGhDZzs3A7b54OiIpIfMS6l0uF1q1hxgy4/XZ4440wdrqISNw0ij10gGuvDeO8zJ0bdSUiIrnRaAK9c2eYPBnuuy9cBENEJG4aTaBD6ML48cfw059GXYmISPY1qkAfOBBGjw7NLhVnjIqIxEWjCnSAf/1XeOcdOPtsnSEqIvHS6AJ99GhYtAg2bICSEnj22agrEhHJjkYX6ABjx8Krr8IJJ8D558Odd4axXkREClmjDHSAU06Bl1+GiRPhppvg0kth166oqxIROXKNNtAB2rQJw+v+13/BE0/A6aeHE49ERApRow50CMMAfPOb8PTT8P77MGwY/Pa3UVclIlJ/jT7QK4weDcuWhWEBPv95+Pd/h4MHo65KRCR9CvQqevSAP/4RrrgCbrsNvvAF2L496qpERNKjQK/hqKPg5z+HH/8YnnwyNMGsXh11VSIidVOgJ2EG11wDzz8PO3fCiBHw+ONRVyUiUjsFei3OPju0qw8YAJMmwbe+BQcORF2ViEhyCvQ6dO4ML7wQxlO/4w644AIoL4+6KhEpVB9/DB9+mJvXVqCnoUULuOeeMPTuiy/CkCHw4IPhsnYiIrU5cCCcmf4f/wGjRoWrp911V262pUCvh2nTQqAfe2y4rF2vXjBnDuzYEXVlItKQbNoUhun+4hfhuONg+HC4+Wb46COYNSsMP5IL5hENYlJSUuJlZWWRbDtT7uFEpDvvhN//Htq2DU0yX/taaKIRkcZlx47QieKZZ0I2VJxx3rkzjBkDn/0s/NM/hXDPlJktc/eSpMsU6JlZtiwMHfDYY9C0KXzpS+HM0379oq5MRHJl//7QjPL00yHEX345NK0UFcHIkSHAx4yBz3wm9JrLJgV6Hrz5JvzgB6Gd/ZNPwr9UN9wA55yT/V+oSKF6801YvDgMhNe8eeXUokX1x8mmquu0anX41LTpkdflHi56849/1D59+CFs2wavvBJOOjSDoUMr98LPOANatsze+5WMAj2Pysvh7rvDVZG2bQt92G+4IZx1mskHTqRQrV8fzuN47LHwH22uNGsWgv2oo5IHftXp448PD+vaOjmYhWNn7dqFafDgEOCjR0OHDrn7mZLXokDPitLScGDjrbege3eYPRumTEm+7iefhJ4wc+aED/TJJ8M3vhGGFTjqqPzWLZJvb7wRAvzxx+Evfwnzhg0L53NMmBDalvftg08/rXtKtt7evbBnT/Xpk08On5dqeVFRZTi3awft21d/XHM65hho0kC6kGQU6GZ2PzAW+MDdT0uy3ID/Bi4EdgNT3X15XUUVWqCXlsL06dWvRVpUBPPmpQ51CO1qv/51OID66qvQqVM4C/Wf/zmMyS4SF2vXVu6Jv/ZamHf66eGaAxMnhrGSJHOZBvq5wC7gFykC/ULgOkKgjwD+291H1FVUoQV6cXHoilRTjx7pXZvUHZYuDQdQFy8O8/r2hfHjYdy40DSjJhkpNKtXV4Z4xZhHZ50VAvySS6Bbt2jri6OMm1zMrBj4bYpA/3/AC+7+cOLx68BId3+vttcstEBv0iT5ZerM6j/M7saNsHBhmP7wh3DE/LjjwrC948aF7k1FRVkpW2Lm44/hz38OTQAnnRRu82nnTnj99fDZffxxWLMm/A2cfXZlc0qXLvmtqbGpLdCbZeH1uwBvV3m8OTHvsEA3s+nAdIDu3btnYdP507178j30I/kxioth5swwffRRGNVx4cKwl3PffaGN/bOfDeE+diwcf3zG5UuBOngwNF889VSY/vSn6gfvOnQIwX7SSeFEt4r7J50EJ55Y/3bfvXvDMaING0KPlJpTxbAXTZrAueeG5sMJE8K2JHrZCPS0ufs8YB6EPfR8bjtTs2cnb0OfPTuz1z32WJg8OUz79oVmmd/8pnIP3iy0Q44fH6Y+fdQNMu7ef7/yBJWnn4YPPgjz+/cPOwGjRoUDfOvXh+Bdvz70g3700eqDx7VqBT17Vg/5Xr1CM+GHHyYP7Hfeqf6faPPmYf1evUL3vJ49w3TeedrRaIjU5FIP9enlkil3WLkyhPpvfgPLE4eZTzklNM2MGBG6Tp10UsM5+i5HZu/ecGGVp58Oe+ErV4b5HTuG/s0VfZzrOgv500/Df5EVIV912rAhNNfUZBaaSCqCuubUubOO7TQ0uW5Dvwi4lsqDonPdfXhdr1mIgR6lt9+GRYtCwD//fOW/3W3awMCBIdwHDQq3p56a+5Mb5Mi5hx4hFQH+wgthj7t583BAccwYOP/88PvM1pe1e9jTX78+hH779mGvu3t3fVYKTaa9XB4GRgIdgS3ArUBzAHe/N9Ft8cfABYRui1929zqTWoF+5PbuDT0KVqwIfXxXrAjTrl1hebNmYeiBqiE/cGBo3qkP99Bnd+fO6tPevaGHTpcuja/5xx22bAnHPnbuDO/5rl2V92veJptXXl7ZjNK7d2WAn3ceHH10tD+fNHw6sagROHgw7H1VhHxF0L//fuU6PXuGgD/11NDWWjOod+4MgwxVfVzbBT2OPx5KSqpPJ5yQ+581H8rLw8kxf/975fTGG2FK1nRRU1FR+O+pTZsQ0lXvt20bmszGjAkHyEXqQ4HeiL3/fmW4V9y+8UZoF23bNgRMfadmzWDVKigrC9OaNZUH0rp2rR7wQ4eGtuCGaNcuWLfu8ND++9+rX4CgadPwZdi7d5hOPjk0WVQEdc3Abt1a7c6SOwp0qWb//hA42Wou2bUrfFlUBHxZWQjFCsXF1UN+4MCw579jR5i2b09+m2pZRdNS06aHT02aJJ9fdXIP5wK8+271n6Nr18rQPuWUyvs9e4b2bZGGQIEuebd9e+iZU1YWhjwoKwvd4tLVqlX4D+KYYw6/bdMmrHPgQJgOHqy8n84E4WBg1dA++WSdzCWFIdcnFokc5phjQn/pUaMq55WXh9H2Vq0KPSuShXXbtmFq0SK62kUKlQJd8qZDh8p+1SKSfTolRUQkJhToIiIxoUDPo9LS0OOjSZNwW1oadUUiEidqQ8+TmhfI2LQpPIbcjQcjIo2L9tDz5Oabq4/UCOHxzTdHU4+IxI8CPU/eeqt+80VE6kuBniepLoRRYNf5EJEGTIGeJ7NnH34mYjYukCEiUkGBnidTpsC8eeHqL2bhdt48HRAVkexRL5c8mjJFAS4iuaM9dBGRmFCgi4jEhAJdRCQmFOgiIjGhQBcRiQkFegHR4F4iUht1WywQGtxLROqiPfQCocG9RKQuCvQCocG9RKQuCvQCocG9RKQuCvQCocG9RKQuCvQCocG9RKQu6uVSQDS4l4jURnvoIiIxoUAXEYkJBXojo7NNReIrrUA3swvM7HUzW2dm30qyfKqZbTWzFYnpq9kvVTJVcbbppk3gXnm2qUJdJB7qDHQzawr8BPgc0A+YbGb9kqz6iLsPSkw/y3KdkgU621Qk3tLZQx8OrHP3De6+D5gPjM9tWZILOttUJN7SCfQuwNtVHm9OzKvpEjN7zcweN7NuyV7IzKabWZmZlW3duvUIypVM6GxTkXjL1kHRRUCxuw8AngEeTLaSu89z9xJ3L+nUqVOWNi3p0tmmIvGWTqC/A1Td4+6amHeIu5e7+97Ew58BQ7NTnmSTzjYVibd0Av1V4BQz62lmLYDLgIVVVzCzE6s8HAesyV6Jkk1TpsDGjXDwYLitb5ir26NIw1Xnqf/uvt/MrgWeApoC97v7ajP7LlDm7guBmWY2DtgPfAhMzWHNEhFdZEOkYTN3j2TDJSUlXlZWFsm25cgUF4cQr6lHj7C3LyK5Z2bL3L0k2TKdKSppU7dHkYZNgS5pU7dHkYZNgS5pU7dHkYZNgS5py0a3R/WSEckdXeBC6iWTi2yol4xIbmkPXfJGg4OJ5JYCXfJGvWREckuBLnmjXjIiuaVAl7zJRi8ZHVQVSU2BLnmTaS8ZXXFJpHY69V8KhoYeENGp/xIT2TioqiYbiTMFuhSMTA+qqslG4k6BLgUj04Oq6gcvcadAl4KR6UFVNdlI3OnUfykomQw90L178oOq9W2y0dAF0lBpD10ajYbQZKM9fMklBbo0GlE32WTjoKy+EKQ2CnRpVDK5SHamvWwy3cPXF4LURYEukqZMm2wy3cPXF4LURYEukqZMm2wy3cPXF4K+UOrk7pFMQ4cOdZHG5KGH3IuK3EMchqmoKMxPR48e1Z9bMfXokd7zzZI/3yw/28/058/0+RWv0aNH+Jl79KjfcxvC893dgTJPkasKdJE8yuQPurF/IRT6F0o2vpDcFegisdGYvxAK/Qsl0+dXqC3Q1YYuUkAy6aWT6TGATA8KZ3oMIepjEFE/Px0KdJFGpJC/EAr9CyUvV+xKteue60lNLiKNT5QHFaNuA1cbuohIFkXdSyXXvVx0xSIRkQKiKxaJiDQCaQW6mV1gZq+b2Toz+1aS5S3N7JHE8lfMrDjbhYqISO3qDHQzawr8BPgc0A+YbGb9aqz2FeAf7n4y8EPgjmwXKiIitUtnD304sM7dN7j7PmA+ML7GOuOBBxP3HwdGm5llr0wREalLOoHeBXi7yuPNiXlJ13H3/cB2oEPNFzKz6WZWZmZlW7duPbKKRUQkqbxegs7d5wHzAMxsq5kluSBYg9AR2BZ1EbVo6PVBw69R9WVG9WUmk/p6pFqQTqC/A3Sr8rhrYl6ydTabWTPgGKC8thd1905pbDsSZlaWqltQQ9DQ64OGX6Pqy4zqy0yu6kunyeVV4BQz62lmLYDLgIU11lkIXJG4PxH4vUfVwV1EpJGqcw/d3feb2bXAU0BT4H53X21m3yWcsbQQuA/4pZmtAz4khL6IiORRWm3o7r4EWFJj3i1V7u8BJmW3tEjNi7qAOjT0+qDh16j6MqP6MpOT+iI79V9ERLJLp/6LiMSEAl1EJCYabaCbWTcze97M/mZmq83sa0nWGWlm281sRWK6Jdlr5bDGjWb218S2Dxua0oK5iTF0XjOzIXmsrU+V92WFme0ws+trrJP398/M7jezD8xsVZV57c3sGTN7I3HbLsVzr0is84aZXZFsnRzV919mtjbxO/y1mR2b4rm1fh5yWN9tZvZOld/jhSmeW+uYTzms75EqtW00sxUpnpvT9y9VpuT185dqXN24T8CJwJDE/aOBvwP9aqwzEvhthDVuBDrWsvxC4EnAgNOBVyKqsynwPtAj6vcPOBcYAqyqMu9O4FuJ+98C7kjyvPbAhsRtu8T9dnmqbwzQLHH/jmT1pfN5yGF9twHfTOMzsB7oBbQAVtb8e8pVfTWW/1/glijev1SZks/PX6PdQ3f399x9eeL+TmANhw9p0NCNB37hwcvAsWZ2YgR1jAbWu3vkZ/66+1JC19mqqo419CDwhSRPPR94xt0/dPd/AM8AF+SjPnd/2sOQGQAvE07ei0SK9y8d6Yz5lLHa6kuMH/VF4OFsbzcdtWRK3j5/jTbQq0oM9zsYeCXJ4jPMbKWZPWlmp+a1MHDgaTNbZmbTkyxPZ5ydfLiM1H9EUb5/FY539/cS998Hjk+yTkN5L6cR/utKpq7PQy5dm2gSuj9Fk0FDeP/OAba4+xspluft/auRKXn7/On2fisAAAI3SURBVDX6QDezNsATwPXuvqPG4uWEZoSBwI+ABXku72x3H0IYuvgaMzs3z9uvU+Ls4XHAY0kWR/3+HcbD/7cNsq+umd0M7AdKU6wS1efhHuAkYBDwHqFZoyGaTO1753l5/2rLlFx//hp1oJtZc8IbX+ruv6q53N13uPuuxP0lQHMz65iv+tz9ncTtB8CvCf/WVpXOODu59jlgubtvqbkg6vevii0VTVGJ2w+SrBPpe2lmU4GxwJTEH/1h0vg85IS7b3H3A+5+EPhpiu1G/f41AyYAj6RaJx/vX4pMydvnr9EGeqK97T5gjbv/IMU6JyTWw8yGE96vWgcdy2J9rc3s6Ir7hANnq2qsthC4PNHb5XRge5V/7fIl5V5RlO9fDVXHGroC+E2SdZ4CxphZu0STwpjEvJwzswuAG4Fx7r47xTrpfB5yVV/V4zIXp9huOmM+5dI/AWvdfXOyhfl4/2rJlPx9/nJ1xLehT8DZhH99XgNWJKYLgRnAjMQ61wKrCUfsXwbOzGN9vRLbXZmo4ebE/Kr1GeFqUuuBvwIleX4PWxMC+pgq8yJ9/whfLu8BnxLaIb9CGJv/OeAN4FmgfWLdEuBnVZ47DViXmL6cx/rWEdpPKz6H9ybW7Qwsqe3zkKf6fpn4fL1GCKcTa9aXeHwhoWfH+nzWl5j/QMXnrsq6eX3/asmUvH3+dOq/iEhMNNomFxGRuFGgi4jEhAJdRCQmFOgiIjGhQBcRiQkFuohITCjQRURi4n8B530TPYIFVG0AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "p44ZbUPQfgAJ"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}