12_Training_A_Neural_Network_Model_III.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "12. Training A Neural Network Model - III.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMMkGWxHRmvlQf9nAJcaUgb",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/tanaymukherjee/Natural-Language-Processing/blob/master/12_Training_A_Neural_Network_Model_III.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-xcO7tAglYbC",
        "colab_type": "text"
      },
      "source": [
        "# Training A Neural Network Model - III"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bn6Uy7z3lZ79",
        "colab_type": "text"
      },
      "source": [
        "## Training best practices"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L4ubwIBP-VN7",
        "colab_type": "text"
      },
      "source": [
        "### Good data v/s bad data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0jGR3w6rleZC",
        "colab_type": "text"
      },
      "source": [
        "1. Rewrite the TRAINING_DATA to only use the label GPE (cities, states, countries) instead of TOURIST_DESTINATION.\n",
        "2. Don't forget to add tuples for the GPE entities that weren't labeled in the old data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZSbTjKKZ-hAz",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        },
        "outputId": "7717a498-c97b-4218-bbaf-345ec6acb5dc"
      },
      "source": [
        "# Bad data:\n",
        "\n",
        "TRAINING_DATA = [\n",
        "    (\"i went to amsterdem last year and the canals were beautiful\", {'entities': [(10, 19, 'TOURIST_DESTINATION')]}),\n",
        "    (\"You should visit Paris once in your life, but the Eiffel Tower is kinda boring\", {'entities': [(17, 22, 'TOURIST_DESTINATION')]}),\n",
        "    (\"There's also a Paris in Arkansas, lol\", {'entities': []}),\n",
        "    (\"Berlin is perfect for summer holiday: lots of parks, great nightlife, cheap beer!\", {'entities': [(0, 6, 'TOURIST_DESTINATION')]})\n",
        "]\n",
        "     \n",
        "print(*TRAINING_DATA, sep='\\n')"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('i went to amsterdem last year and the canals were beautiful', {'entities': [(10, 19, 'TOURIST_DESTINATION')]})\n",
            "('You should visit Paris once in your life, but the Eiffel Tower is kinda boring', {'entities': [(17, 22, 'TOURIST_DESTINATION')]})\n",
            "(\"There's also a Paris in Arkansas, lol\", {'entities': []})\n",
            "('Berlin is perfect for summer holiday: lots of parks, great nightlife, cheap beer!', {'entities': [(0, 6, 'TOURIST_DESTINATION')]})\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5_onIpHx_CuP",
        "colab_type": "text"
      },
      "source": [
        "**NOTE:** Whether a place is a tourist destination is a subjective judgement and not a definitive category. It will be very difficult for the entity recognizer to learn."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M9pQXZk7-rcq",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 87
        },
        "outputId": "7f8a12f7-5eaa-479d-9473-e15498f8de56"
      },
      "source": [
        "# Good data\n",
        "\n",
        "TRAINING_DATA = [\n",
        "    (\"i went to amsterdem last year and the canals were beautiful\", {'entities': [(10, 19, 'GPE')]}),\n",
        "    (\"You should visit Paris once in your life, but the Eiffel Tower is kinda boring\", {'entities': [(17, 22, 'GPE')]}),\n",
        "    (\"There's also a Paris in Arkansas, lol\", {'entities': [(15, 20, 'GPE'), (24, 32, 'GPE')]}),\n",
        "    (\"Berlin is perfect for summer holiday: lots of parks, great nightlife, cheap beer!\", {'entities': [(0, 6, 'GPE')]})\n",
        "]\n",
        "\n",
        "print(*TRAINING_DATA, sep='\\n')"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('i went to amsterdem last year and the canals were beautiful', {'entities': [(10, 19, 'GPE')]})\n",
            "('You should visit Paris once in your life, but the Eiffel Tower is kinda boring', {'entities': [(17, 22, 'GPE')]})\n",
            "(\"There's also a Paris in Arkansas, lol\", {'entities': [(15, 20, 'GPE'), (24, 32, 'GPE')]})\n",
            "('Berlin is perfect for summer holiday: lots of parks, great nightlife, cheap beer!', {'entities': [(0, 6, 'GPE')]})\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lxqlPMlO_j6a",
        "colab_type": "text"
      },
      "source": [
        "### Training multiple labels"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XFLqcgG2_kX_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        },
        "outputId": "41e25c34-9c32-4971-ae41-154a90ecf466"
      },
      "source": [
        "# Bad implementation\n",
        "\n",
        "TRAINING_DATA = [\n",
        "    (\"Reddit partners with Patreon to help creators build communities\", \n",
        "     {'entities': [(0, 6, 'WEBSITE'), (21, 28, 'WEBSITE')]}),\n",
        "  \n",
        "    (\"PewDiePie smashes YouTube record\", \n",
        "     {'entities': [(18, 25, 'WEBSITE')]}),\n",
        "  \n",
        "    (\"Reddit founder Alexis Ohanian gave away two Metallica tickets to fans\", \n",
        "     {'entities': [(0, 6, 'WEBSITE')]}),\n",
        "    # And so on...\n",
        "]\n",
        "\n",
        "print(*TRAINING_DATA, sep='\\n')"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('Reddit partners with Patreon to help creators build communities', {'entities': [(0, 6, 'WEBSITE'), (21, 28, 'WEBSITE')]})\n",
            "('PewDiePie smashes YouTube record', {'entities': [(18, 25, 'WEBSITE')]})\n",
            "('Reddit founder Alexis Ohanian gave away two Metallica tickets to fans', {'entities': [(0, 6, 'WEBSITE')]})\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cNg1pTyz_wfR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        },
        "outputId": "f2c47588-c4dc-407f-c356-77d2b50fe7db"
      },
      "source": [
        "# Good implementation\n",
        "\n",
        "TRAINING_DATA = [\n",
        "    (\"Reddit partners with Patreon to help creators build communities\", \n",
        "     {'entities': [(0, 6, 'WEBSITE'), (21, 28, 'WEBSITE')]}),\n",
        "  \n",
        "    (\"PewDiePie smashes YouTube record\", \n",
        "     {'entities': [(0, 9, 'PERSON'), (18, 25, 'WEBSITE')]}),\n",
        "  \n",
        "    (\"Reddit founder Alexis Ohanian gave away two Metallica tickets to fans\", \n",
        "     {'entities': [(0, 6, 'WEBSITE'), (15, 29, 'PERSON')]}),\n",
        "    # And so on...\n",
        "]\n",
        "\n",
        "print(*TRAINING_DATA, sep='\\n')"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "('Reddit partners with Patreon to help creators build communities', {'entities': [(0, 6, 'WEBSITE'), (21, 28, 'WEBSITE')]})\n",
            "('PewDiePie smashes YouTube record', {'entities': [(0, 9, 'PERSON'), (18, 25, 'WEBSITE')]})\n",
            "('Reddit founder Alexis Ohanian gave away two Metallica tickets to fans', {'entities': [(0, 6, 'WEBSITE'), (15, 29, 'PERSON')]})\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}