00_Pre_Neural_NLP.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<a href=\"https://colab.research.google.com/github/ZohebAbai/Deep-Learning-Projects/blob/master/00_Pre_Neural_NLP.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UsuMpzlz0k-B"
      },
      "source": [
        "# Heuristics-based & Statistical Methods in NLP (Pre-2010s)\n",
        "\n",
        "![SA](https://miro.medium.com/max/1200/1*TUqC3XRqvzErhnFC4h1Rcw@2x.png)\n",
        "\n",
        "This notebooks contains Sentiment Analysis based on\n",
        "1. VADER (Heuristics)\n",
        "2. SVM (Advanced Statistics)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jb7w_dsF1aYX"
      },
      "source": [
        "## VADER\n",
        "\n",
        "Valence Aware Dictionary and Sentiment Reasoner (VADER) is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media. VADER does not requires any training data but is constructed from a generalizable, valence-based, human-curated gold standard sentiment lexicon. (A sentiment lexicon is a list of lexical features e.g., words, which are generally labelled according to their semantic orientation as either positive or negative.). VADER has been found to be quite successful when dealing with social media texts, editorials, movie reviews, and product reviews. This is because VADER not only tells about the Positivity and Negativity score but also tells us about how positive or negative a sentiment is."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vOe59IoT1vXa"
      },
      "source": [
        "### Import Libraries\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RKbme1um0SsV",
        "outputId": "4cb0058f-f02f-4a79-9c57-7d3759c4e754"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package vader_lexicon to /root/nltk_data...\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/nltk/twitter/__init__.py:20: UserWarning: The twython library has not been installed. Some functionality from the twitter package will not be available.\n",
            "  warnings.warn(\"The twython library has not been installed. \"\n"
          ]
        }
      ],
      "source": [
        "import nltk\n",
        "nltk.download('vader_lexicon')\n",
        "\n",
        "from nltk.sentiment.vader import SentimentIntensityAnalyzer\n",
        "analyser = SentimentIntensityAnalyzer()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NEXJ2rW32Aml"
      },
      "source": [
        "### Scoring\n",
        "We will use the polarity_scores() method to obtain the polarity indices for the given sentence."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "2MDvvsh013bl"
      },
      "outputs": [],
      "source": [
        "def sentiment_analyzer_scores(sentence):\n",
        "    score = analyser.polarity_scores(sentence)\n",
        "    print(\"{} {}\".format(sentence, str(score)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "96r1XrOH2IkS"
      },
      "source": [
        "Let's see how it performs on a custom sentence"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TMK42T0L2FbH",
        "outputId": "f18dbda0-2ee5-40fc-8b73-a19bdc53d002"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "VADER is smart, handsome, and funny. {'neg': 0.0, 'neu': 0.254, 'pos': 0.746, 'compound': 0.8316}\n"
          ]
        }
      ],
      "source": [
        "sentiment_analyzer_scores(\"VADER is smart, handsome, and funny.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1q_fgHMu2QIl"
      },
      "source": [
        "1. The Positive, Negative and Neutral scores represent the proportion of text that falls in these categories. This means our sentence was rated as 75% Positive, 25% Neutral and 0% Negative. Hence all these should add up to 1.\n",
        "\n",
        "2. The compound score is computed by summing the valence scores of each word in the lexicon, adjusted according to the rules, and then normalized to be between -1 (most extreme negative) and +1 (most extreme positive). This is the most useful metric if you want a single unidimensional measure of sentiment for a given sentence. Calling it a 'normalized, weighted composite score' is accurate. positive sentiment: compound score >= 0.05 neutral sentiment: (compound score > -0.05) and (compound score < 0.05) negative sentiment: compound score <= -0.05"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vqx05uKl2fz0"
      },
      "source": [
        "### Where does VADER Fail?\n",
        "You saw how better VADER model is for analyzing social media texts. However, on more nuanced examples, it performs poorly.\n",
        "\n",
        "Consider the review “everything tastes like garbage to me but we keep coming back because my wife loves the pasta”. Clearly the reviewer does NOT like this restaurant, despite the fact that his or her wife “loves” the pasta. So for humans, this review is a clear negative.\n",
        "\n",
        "Let's see what VADER does."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a3h0HCGq2LlG",
        "outputId": "fad8f2c1-e7dd-4ade-862d-907a9a26c5cd"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "everything tastes like garbage to me but we keep coming back because my wife loves the pasta {'neg': 0.0, 'neu': 0.688, 'pos': 0.312, 'compound': 0.7783}\n"
          ]
        }
      ],
      "source": [
        "sentiment_analyzer_scores(\"everything tastes like garbage to me but we keep coming back because my wife loves the pasta\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IE-hR7vB2qN6"
      },
      "source": [
        "\n",
        "VADER instead returns a compound score of 0.78, which means highly positive. It relies on the polarity of certain words to determine the overall sentiment. It doesn't have a broader syntactic understanding of the sentence."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "axAITZne3B-G"
      },
      "source": [
        "## SVM\n",
        "\n",
        "![svm](https://www.reneshbedre.com/assets/posts/svm/svm_linear.webp)\n",
        "\n",
        "Let's perform a SVM based Sentiment Analysis based on Support a Vector Machine Model on Twitter Sentiments of US Airline passengers."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kbaaIqBi3be2"
      },
      "source": [
        "### Import Libraries\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iV1rl1lb2l0G",
        "outputId": "ed097c33-3f88-4d46-aaa0-ccf673e8df02"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ]
        }
      ],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from bs4 import BeautifulSoup\n",
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "import seaborn as sns\n",
        "\n",
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.stem import SnowballStemmer\n",
        "from nltk.tokenize import TweetTokenizer\n",
        "\n",
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.model_selection import train_test_split, StratifiedKFold, cross_val_score\n",
        "from sklearn.pipeline import make_pipeline, Pipeline\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.metrics import make_scorer, accuracy_score, f1_score\n",
        "from sklearn.metrics import roc_curve, auc\n",
        "from sklearn.metrics import confusion_matrix, roc_auc_score, recall_score, precision_score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ndeqhqQe3vnH"
      },
      "source": [
        "### Import Data\n",
        "\n",
        "Download it from [here](https://www.kaggle.com/datasets/crowdflower/twitter-airline-sentiment)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 496
        },
        "id": "1_Q76dJa3pHk",
        "outputId": "7a681f9a-9c35-4ad6-f6a8-78cb4a85fe92"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-e23095fd-6725-4805-b5cc-1d780cb20e9f\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tweet_id</th>\n",
              "      <th>airline_sentiment</th>\n",
              "      <th>airline_sentiment_confidence</th>\n",
              "      <th>negativereason</th>\n",
              "      <th>negativereason_confidence</th>\n",
              "      <th>airline</th>\n",
              "      <th>airline_sentiment_gold</th>\n",
              "      <th>name</th>\n",
              "      <th>negativereason_gold</th>\n",
              "      <th>retweet_count</th>\n",
              "      <th>text</th>\n",
              "      <th>tweet_coord</th>\n",
              "      <th>tweet_created</th>\n",
              "      <th>tweet_location</th>\n",
              "      <th>user_timezone</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>570306133677760513</td>\n",
              "      <td>neutral</td>\n",
              "      <td>1.0000</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Virgin America</td>\n",
              "      <td>NaN</td>\n",
              "      <td>cairdin</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>@VirginAmerica What @dhepburn said.</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2015-02-24 11:35:52 -0800</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Eastern Time (US &amp; Canada)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>570301130888122368</td>\n",
              "      <td>positive</td>\n",
              "      <td>0.3486</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0.0000</td>\n",
              "      <td>Virgin America</td>\n",
              "      <td>NaN</td>\n",
              "      <td>jnardino</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>@VirginAmerica plus you've added commercials t...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2015-02-24 11:15:59 -0800</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Pacific Time (US &amp; Canada)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>570301083672813571</td>\n",
              "      <td>neutral</td>\n",
              "      <td>0.6837</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Virgin America</td>\n",
              "      <td>NaN</td>\n",
              "      <td>yvonnalynn</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>@VirginAmerica I didn't today... Must mean I n...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2015-02-24 11:15:48 -0800</td>\n",
              "      <td>Lets Play</td>\n",
              "      <td>Central Time (US &amp; Canada)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>570301031407624196</td>\n",
              "      <td>negative</td>\n",
              "      <td>1.0000</td>\n",
              "      <td>Bad Flight</td>\n",
              "      <td>0.7033</td>\n",
              "      <td>Virgin America</td>\n",
              "      <td>NaN</td>\n",
              "      <td>jnardino</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>@VirginAmerica it's really aggressive to blast...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2015-02-24 11:15:36 -0800</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Pacific Time (US &amp; Canada)</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>570300817074462722</td>\n",
              "      <td>negative</td>\n",
              "      <td>1.0000</td>\n",
              "      <td>Can't Tell</td>\n",
              "      <td>1.0000</td>\n",
              "      <td>Virgin America</td>\n",
              "      <td>NaN</td>\n",
              "      <td>jnardino</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>@VirginAmerica and it's a really big bad thing...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2015-02-24 11:14:45 -0800</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Pacific Time (US &amp; Canada)</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e23095fd-6725-4805-b5cc-1d780cb20e9f')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-e23095fd-6725-4805-b5cc-1d780cb20e9f button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-e23095fd-6725-4805-b5cc-1d780cb20e9f');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "             tweet_id airline_sentiment  airline_sentiment_confidence  \\\n",
              "0  570306133677760513           neutral                        1.0000   \n",
              "1  570301130888122368          positive                        0.3486   \n",
              "2  570301083672813571           neutral                        0.6837   \n",
              "3  570301031407624196          negative                        1.0000   \n",
              "4  570300817074462722          negative                        1.0000   \n",
              "\n",
              "  negativereason  negativereason_confidence         airline  \\\n",
              "0            NaN                        NaN  Virgin America   \n",
              "1            NaN                     0.0000  Virgin America   \n",
              "2            NaN                        NaN  Virgin America   \n",
              "3     Bad Flight                     0.7033  Virgin America   \n",
              "4     Can't Tell                     1.0000  Virgin America   \n",
              "\n",
              "  airline_sentiment_gold        name negativereason_gold  retweet_count  \\\n",
              "0                    NaN     cairdin                 NaN              0   \n",
              "1                    NaN    jnardino                 NaN              0   \n",
              "2                    NaN  yvonnalynn                 NaN              0   \n",
              "3                    NaN    jnardino                 NaN              0   \n",
              "4                    NaN    jnardino                 NaN              0   \n",
              "\n",
              "                                                text tweet_coord  \\\n",
              "0                @VirginAmerica What @dhepburn said.         NaN   \n",
              "1  @VirginAmerica plus you've added commercials t...         NaN   \n",
              "2  @VirginAmerica I didn't today... Must mean I n...         NaN   \n",
              "3  @VirginAmerica it's really aggressive to blast...         NaN   \n",
              "4  @VirginAmerica and it's a really big bad thing...         NaN   \n",
              "\n",
              "               tweet_created tweet_location               user_timezone  \n",
              "0  2015-02-24 11:35:52 -0800            NaN  Eastern Time (US & Canada)  \n",
              "1  2015-02-24 11:15:59 -0800            NaN  Pacific Time (US & Canada)  \n",
              "2  2015-02-24 11:15:48 -0800      Lets Play  Central Time (US & Canada)  \n",
              "3  2015-02-24 11:15:36 -0800            NaN  Pacific Time (US & Canada)  \n",
              "4  2015-02-24 11:14:45 -0800            NaN  Pacific Time (US & Canada)  "
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "data = pd.read_csv(\"/content/Tweets_Airline.csv\")\n",
        "data.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PlHy5o9y5Brv"
      },
      "source": [
        "We take only the tweets we are very confident with. We use the BeautifulSoup library to process html encoding present in some tweets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "1UPTIVpa47Bd"
      },
      "outputs": [],
      "source": [
        "data_clean = data.copy()\n",
        "data_clean = data_clean[data_clean['airline_sentiment_confidence'] > 0.65]\n",
        "data_clean['text_clean'] = data_clean['text'].apply(lambda x: BeautifulSoup(x, \"lxml\").text)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QcXvEQzl5J6F"
      },
      "source": [
        "For simplicity we are going to distinguish two cases: tweets with negative sentiment and tweets with non-negative sentiment"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MM6uLlEb5UZF",
        "outputId": "27ebff51-e46c-46ae-9a8c-e84217ad88de"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0         neutral\n",
              "2         neutral\n",
              "3        negative\n",
              "4        negative\n",
              "5        negative\n",
              "           ...   \n",
              "14634    negative\n",
              "14636    negative\n",
              "14637     neutral\n",
              "14638    negative\n",
              "14639     neutral\n",
              "Name: airline_sentiment, Length: 13651, dtype: object"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "data_clean['airline_sentiment']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "QRa1iMUm5Ffn",
        "outputId": "547f9e0a-488b-42ca-fb7e-9e160b82dcf1"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-c7a0d7c6-a7d3-49be-b5ce-8aa07e43423e\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text_clean</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>@VirginAmerica What @dhepburn said.</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>@VirginAmerica I didn't today... Must mean I n...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>@VirginAmerica it's really aggressive to blast...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>@VirginAmerica and it's a really big bad thing...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>@VirginAmerica seriously would pay $30 a fligh...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c7a0d7c6-a7d3-49be-b5ce-8aa07e43423e')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-c7a0d7c6-a7d3-49be-b5ce-8aa07e43423e button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-c7a0d7c6-a7d3-49be-b5ce-8aa07e43423e');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "                                          text_clean  sentiment\n",
              "0                @VirginAmerica What @dhepburn said.          0\n",
              "2  @VirginAmerica I didn't today... Must mean I n...          0\n",
              "3  @VirginAmerica it's really aggressive to blast...          1\n",
              "4  @VirginAmerica and it's a really big bad thing...          1\n",
              "5  @VirginAmerica seriously would pay $30 a fligh...          1"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "data_clean['sentiment'] = data_clean['airline_sentiment'].apply(lambda x: 1 if x == 'negative' else 0)\n",
        "data_clean = data_clean.loc[:, ['text_clean', 'sentiment']]\n",
        "data_clean.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zo6DKeWk5w-R"
      },
      "source": [
        "### We split the data into training and testing set:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "U2pK5OKZ5OrT"
      },
      "outputs": [],
      "source": [
        "train, test = train_test_split(data_clean, test_size=0.2, random_state=1)\n",
        "X_train = train['text_clean'].values\n",
        "X_test = test['text_clean'].values\n",
        "y_train = train['sentiment']\n",
        "y_test = test['sentiment']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "chKlaUSP55wZ"
      },
      "source": [
        "### Preprocessing the Data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "32AIecSU5z0I"
      },
      "outputs": [],
      "source": [
        "def tokenize(text): \n",
        "    tknzr = TweetTokenizer()\n",
        "    return tknzr.tokenize(text)\n",
        "\n",
        "def stem(doc):\n",
        "    return (stemmer.stem(w) for w in analyzer(doc))\n",
        "\n",
        "en_stopwords = set(stopwords.words(\"english\")) \n",
        "\n",
        "vectorizer = CountVectorizer(\n",
        "    analyzer = 'word',\n",
        "    tokenizer = tokenize,\n",
        "    lowercase = True,\n",
        "    ngram_range=(1, 1),\n",
        "    stop_words = en_stopwords)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D1jQsovM6R2S"
      },
      "source": [
        "### Grid Search\n",
        "We are going to use cross validation and grid search to find good hyperparameters for our SVM model. We need to build a pipeline."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "8DhuAoSj6M52"
      },
      "outputs": [],
      "source": [
        "kfolds = StratifiedKFold(n_splits=5, shuffle=True, random_state=1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yH8Bp3bU6dSg",
        "outputId": "ed1926f4-a342-4752-eb46-8d461443e4ea"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 3 candidates, totalling 15 fits\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "0.9188979138877743"
            ]
          },
          "execution_count": 14,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "np.random.seed(1)\n",
        "\n",
        "pipeline_svm = make_pipeline(vectorizer, SVC(probability=True, \n",
        "                                             kernel= 'linear', \n",
        "                                             class_weight='balanced'))\n",
        "\n",
        "grid_svm = GridSearchCV(pipeline_svm,\n",
        "                    param_grid = {'svc__C': [0.01, 0.1, 1]}, \n",
        "                    cv = kfolds,\n",
        "                    scoring=\"roc_auc\",\n",
        "                    verbose=1,   \n",
        "                    n_jobs=-1) \n",
        "\n",
        "grid_svm.fit(X_train, y_train)\n",
        "grid_svm.score(X_test, y_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3anI6QfI7M8_",
        "outputId": "5273ac29-7154-474e-d50e-667caeeb9df5"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'svc__C': 0.1}\n",
            "0.9025857063886015\n"
          ]
        }
      ],
      "source": [
        "print(grid_svm.best_params_)\n",
        "print(grid_svm.best_score_)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SrgA9J9k7pVb"
      },
      "source": [
        "### Let's see how the model (with the best hyperparameters) works on the test data:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3c7P_-IM7mUm",
        "outputId": "61e56ab1-16ab-4c9f-ea46-c1ab1f2b59e5"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'acc': 0.8337605272793849,\n",
              " 'auc': 0.9189027336468586,\n",
              " 'f1': 0.8690888119953863,\n",
              " 'precision': 0.9133333333333333,\n",
              " 'recall': 0.828932893289329}"
            ]
          },
          "execution_count": 16,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "def report_results(model, X, y):\n",
        "    pred_proba = model.predict_proba(X)[:, 1]\n",
        "    pred = model.predict(X)        \n",
        "\n",
        "    auc = roc_auc_score(y, pred_proba)\n",
        "    acc = accuracy_score(y, pred)\n",
        "    f1 = f1_score(y, pred)\n",
        "    prec = precision_score(y, pred)\n",
        "    rec = recall_score(y, pred)\n",
        "    result = {'auc': auc, 'f1': f1, 'acc': acc, 'precision': prec, 'recall': rec}\n",
        "    return result\n",
        "\n",
        "report_results(grid_svm.best_estimator_, X_test, y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P9MKPPxF7yhR"
      },
      "source": [
        "### ROC Curve"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 513
        },
        "id": "6t1jRiRB70Yj",
        "outputId": "00c244e8-2f08-48d1-976c-8556eee4052f"
      },
      "outputs": [
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAAHwCAYAAACc4U/qAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeZyNZePH8c81ixn7vi9h7CTLRMjyJE+SiEqLJW1aLGlRxh5mpudRJKkopagoS+qXh6K0Z1+KiCF7BqmMMeu5fn/cx5hkOTRn7jMz3/frNa97O+fMl9erzHfu674uY61FREREREREIMjtACIiIiIiIoFCBUlERERERMRLBUlERERERMRLBUlERERERMRLBUlERERERMRLBUlERERERMRLBUlERERERMRLBUlERLKUMeYXY8xJY0yCMeZXY8xMY0wht3OJiIj4QgVJRET84UZrbSGgEdAYiHI5z1kZY0LcziAiIoFFBUlERPzGWvsrsBSnKAFgjOlijNlsjPndGLPCGFM307XKxpgFxpjDxpijxpgXz/a5xphgY8wwY0ycMea4MWat971VjTE2c/Hxfo/7vPt9jTHfGGMmGWOOAuO8ORpken1p7x2wMt7jzsaYDd7XfWuMaZjlf1EiIhIwVJBERMRvjDGVgOuBHd7jWsC7wGCgNLAY+MgYk88YEwz8H7AbqApUBOac46MfA+4AOgFFgHuARB9jNQd2AmWBscAC72ed0gP4wlobb4xpDLwOPACUBKYBHxpjwnz8XiIiksOoIImIiD98YIw5DuwF4oHR3vO3AR9baz+11qYCzwL5gZZAM6ACMMRae8Jam2St/focn38fMMJau806Nlprj/qY7YC1doq1Ns1aexJ4B7g90/U7vecA+gHTrLUrrbXp1to3gWTgKh+/l4iI5DAqSCIi4g83WWsLA+2AOkAp7/kKOHeIALDWenBKVEWgMrDbWpvmw+dXBuIuMdveM44/BwoYY5obY6riDAdc6L12GfC4d3jd78aY373fu8Ilfm8REQlwKkgiIuI31tovgJk4d4oADuCUDgCMMQancOzHKS5VfJw4YS8QcZbzJ7zbApnOlTsz1hkZ04H3cIbZ3QH8n7X2eKbvE22tLZbpq4C19l0fMoqISA6kgiQiIv72PNDBGHMFThG5wRjT3hgTCjyOM2TtW2AVcBB4xhhT0BgTboxpdY7PfA1ngoWaxtHQGFPSWnsYp2z18k7kcA9nL1Jnegdn+F9PTg+vA3gVeNB7d8l4c91gjCl88X8NIiKSE6ggiYiIX3lLy1vAKGvtNqAXMAU4AtyIMyV4ivdOzo1ADWAPsA+ntJzNRJyy9QnwJzAD51kmgPuBIcBRoD5O+bpQxpU4d58qAP/LdH6N9/NeBI7hTDbR17c/uYiI5ETGWnvhV4mIiIiIiOQBuoMkIiIiIiLipYIkIiIiIiLipYIkIiIiIiLipYIkIiIiIiLipYIkIiIiIiLi5ctifAGlVKlStmrVqm7HEBERERGRALV27doj1trSl/LeHFeQqlatypo1a9yOISIiIiIiAcoYs/tS36shdiIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl4qSCIiIiIiIl5+K0jGmNeNMfHGmB/Pcd0YY14wxuwwxmwyxjTxVxYRERERERFf+PMO0kyg43muXw/U9H71A172YxYREREREZELCvHXB1trvzTGVD3PS7oCb1lrLfC9MaaYMaa8tfagvzKJiIiIiMhZJCfDr7+6nSJLHPntt3/0fr8VJB9UBPZmOt7nPaeCJCIiIiLnlpoKGzeCx3Px7125EtLTsz5Tdli6FAoXvrT3fv89JCZCWNjZrx84cOm5AkQ8EA288g8/x82C5DNjTD+cYXhUqVLF5TQiIiIifpCeDqtXw+HDbic5bflyyJ/fne+9cyfs2AEFC4Ixf7325ZfuZAoUdete/HsKFoQ//oBu3c5+3VrIlw+uvPKfZXNBusfD04sWMfGTTziRnIwxxvnzXCI3C9J+oHKm40rec39jrZ0OTAeIjIy89D+tiIiI5H5HjvzzkrFpE8yZA8WLX9r7P/8cQkMh6CIe99627dK+V3bIly/7v2dKirO98krnh/vM2rVztkOGXPznWguNGv39M3MCY6BoUbdTBJxgYPW8eZxITqZz585ER0dzxRVXXPLnuVmQPgQGGGPmAM2BP/T8kYiISC50+DC88QakpZ37NZ98AoUKnf9zNmxwPissDIKDz/4ajwf+/PPSs55N5coXfs2Z0tOd5zm6dvX9PVdc4fzw3qcPlC9/8d/TH0JCoEGDc/99i7ggLS2NN998kyZNmtC4cWMAnn32WYYPH87VV1/9jz/fbwXJGPMu0A4oZYzZB4wGQgGsta8Ai4FOwA4gEbjbX1lEREQEOHYMTpz4+/nkZGdo16m7HZ9/fv47Bj//DLt3Q4ECfx/6dKb9++HgRfz+s2nTc18rW9YpHjfffP4f2FNS4LLLoFo137/vmTweuPxypxyISECw1rJgwQKGDx/Otm3b+Pe//83SpUsBqF+/fpZ9H3/OYnfHBa5boL+/vr+IiEiusXOnU24ysxZWrPjrA9fWwocfQunSf31tWhrMm3fx37dYsbOfT011ila7dk5JOp8yZZw7MLfeCoMGnf+1bgzjEpEcYfny5URFRbF69WoAqlevzl133YW11nnmKAvliEkaREREcp3ERKdogHM3Zr/3MdwDB2DrVqf4REdf+ufXqnV63+OBKlWcAtKv39mfq7EWWrVy9o1x3q9hVSLisu3bt/Pwww+zbNkyAMqVK8eoUaO49957yeenX6qoIImIiGT2++8QH+/sHzkCP/3kW1HYswe2b7/wjF8nTsA77/iep3x56N4dWrSAIkX+ei0oyHmAPXO+kBA9xC0iuUbBggX5+uuvKVq0KE899RSDBg2ioJ8n2FBBEhGR3OPnn+HQId9e+8038NJLsHevUyhCvP8kHj36zzKULHn+oWLp6c46JoULQ//+TqHyeKBCBahe3XlNuXLOMzQiInnMnj17eOWVVxg7diwhISFUqFCBBQsW0Lx5c0qUKJEtGVSQRETEPWlpf52OOSHBmalszZrTw898dfw4vPbapeXo0MGZAOBUprJloXZt57hIEd8f1C9W7NzP7YiIyDkdOXKEmJgYXnrpJZKTk6lZsyZ33+3M4Xb99ddnaxYVJBERyTqbN8PIked/cH/nTme9l0KFnGFp53MxK8anpTlDzYYOhX/9y7f3NG2qQiMi4qLjx48zadIknn32WY4fPw7A7bffniXTdV8qFSQREXGei1m61Jnq+aOPzr8Gy5dfOmUkPPzv1xITT+9HRJz9/R4PJCVBmzbQtq0zIUDLlqevh4Y6z9XUqHHh53lERCTHevPNNxkyZAiHvSMJOnbsSExMTMbaRm5RQRIRycnS0pwZ0DZtcoannenTT8++WnxSkjMddIUK8OOPf79+8CDUqXP279m0qfOcTpcuZ79+5ZXOlM5ZPO2qiIjkLomJiRw+fJgWLVoQGxtL27Zt3Y4EqCCJiASmxYudKZ5PPRdzpi1bnGmhz1aKzqZMmb8ep6U5i4Omp0O3bs7MbZ06wY03nn72RkREJItYa1m0aBHx8fH069cPgPvuu4/LLruM66+/PsvXMvonVJBERLJaYiJ89plTQk758suzTxW9aBGUKuU8k/Pbb2f/vMsv//u50FDnq29fZ9uyJdSr58yglllQEFStqrs5IiLimhUrVjB06FBWrlxJoUKF6NatG6VLlyY0NJROnTq5He9vVJBERHzl8ThFZvNmZ4jZRx+dLiQrVjhlxFpnyNu5nDl5QVKSM1FB69bOAqFduzqFx1ro0cP32dNEREQCzLp16xg2bBhLly4FoEyZMowYMYIiZ67pFmBUkEREzuXkSWeYW3S0MyFBUtLZX1e1qrONj3eGq115pbOOTa9eEBbmXDPGucMTGpot0UVERNySmJjIPffcw9y5cwEoUqQIQ4YMYfDgwRQqVMjldBemgiQieUdqqvPczqn9lSuduz6npKc7z/6ULAkzZzozu53SqhU0bOic69ABqlSBatWgdOls/SOIiIgEuvz583PgwAHCwsIYOHAgQ4cOpeSZQ8ADmAqSiOQue/bAggXOELXM5s2Db7/1/XOKFnVmfxs1Crp3d6acFhERkb85evQo//nPf7jrrruoX78+xhimTZtG4cKFqVSpktvxLpoKkojkLmPHwowZ575+333O3SBwnilq1eqvExjkzw+VK/s3o4iISC6QkJDA5MmT+e9//8uff/7J9u3bWbhwIQB169Z1Od2lU0ESkdzhhRfgkUec/aJFzz5RQsGCEKL/7YmIiPwTKSkpvPrqq4wbN45Dhw4B0KFDB4YPH+5ysqyhnxREJGc4eBB694bly52ik3myg99/P71fvTq8+qpTkkRERCRLffPNN/Tu3Ztdu3YB0KxZM2JjY7nmmmtcTpZ1VJBEJLBs2wbr1zuTJfz6qzOpQng4rFt3+jVt2kDNmn99n8cDAwdCrVrZm1dERCQPqVChAvv27aNu3bpER0dz0003BdQir1lBBUlEAkNamvN80Jtv/v3atddCp07OgqljxjiFSURERPzuq6++YtasWbzyyisEBQVRrVo1vvrqKyIjIwk+2wLouYAKkoi4b906aNr09PFLL8F11znTaOey30qJiIjkBBs3bmTYsGEsXrwYcJ4xuvXWWwFo3ry5m9H8TgVJRPzDWoiLg59+giNHTp//6Sc4fNiZLCElBd566/S1unWdoXWnFl4VERGRbBUXF8eoUaN45513AChUqBBPPPEEHTt2dDlZ9lFBEpF/Zv9+GDDAGSL3f//nTI4QEgJHj57/fRUqOAuzFisGYWHO9Nz9+mVPZhEREfmb4cOH89///pe0tDTy5ctH//79iYqKonQeWxRdBUlE/pkZM+CDD5z9yy+HfPngqquc48REaN/euSOUeaG4kiWhUKFsjyoiIiLnVqJECTweD3fffTdjxoyhSpUqbkdyhQqSiPhu1y5nCJy18OGHULo0bNzoXEtPh6Agd/OJiIiITxITE5kyZQpFihThoYceAuDhhx+mU6dOOXqR16yggiQivhs6FN5776/natSArl1VjkRERHKA1NRUZsyYwdixYzl48CDFixenV69eFC5cmPz58+f5cgQqSCJyNunpzoKsv/4Kycnw8svO2kTgFKJvv3WeMype3N2cIiIi4hOPx8PcuXMZOXIkcXFxADRt2pTY2FgKadj7X6ggieRVqanO9Nrp6bB6tXO8fz/88INTjs6mSROYNMkZWiciIiI5wqFDh7juuuvY6B0WX6tWLaKjo7n55ptz3SKvWUEFSSQvSUmBY8ec/TfegKios7/ussuciRQmTHAmWChQAMqVy7aYIiIiknXKlClDSEgIlSpVYsyYMdx1112EhKgGnIv+ZkRys7174fffne3bb4N3TYMMwcHw0UfO80MNG0Lhws4sdPnyuZNXRERE/rEffviBUaNG8eyzzxIREYExhvfff59y5cqRP39+t+MFPBUkkdzk+HFYtQo8Hhg0CLZu/ftrrrkGbrnF2a9WDfLQwm8iIiK52a5duxg9ejSzZ8/GWkuxYsV44403AKhWrZrL6XIOFSSRnMjjce4K/fSTsyDrrl3OkLmdO//+2mefdYbJlSkDLVs6d41EREQk1zh06BDjx49n2rRppKamEhoaykMPPcSwYcPcjpYjqSCJ5ASHD8Pdd8PHHztFJz7+7K9r0gQaNID773dmmWvaFEJDszeriIiIZJu5c+dy7733cuLECYwx9OnTh6effpqqVau6HS3HUkESCXQeD0RGwp49znGDBhARAUlJ0K6ds1+hAhQpAmXLuhpVREREslfDhg1JSkqia9eujB8/ngYNGrgdKcdTQRIJdP36OeWoalXYsgX0cKWIiEielJaWxsyZM/n000+ZM2cOxhjq1q3Lzz//TPXq1d2Ol2sEuR1ARM7jk09gxgxn/7PPVI5ERETyII/Hw/vvv0/9+vW5//77ee+991ieac1ClaOspTtIIoHAWmeRVmudIXWTJ8P//nd6FroJE5wZ50RERCTPsNby6aefEhUVxbp16wCoUaMG48aN45prrnE5Xe6lgiTitq+/htatz3190SLo0iX78oiIiIjrrLV069aNRYsWAVC+fHlGjx7NPffcQ6gmYPIrFSQRNx06dLocXXst3H67sx8eDt27a0idiIhIHmWMoUmTJnzxxRdERUUxYMAAChQo4HasPMFYa93OcFEiIyPtmjVr3I4h8s95PNC8OaxZA3Xrwo8/QpAeCxQREcmLdu/ezZgxY2jWrBkPPfQQACdOnCAlJYXixYu7nC7nMcastdZGXsp79dOYiBushbg4pxyBMwGDypGIiEieEx8fz+DBg6lVqxYzZ84kJiaGtLQ0AAoWLKhy5AL9RCaSnTweeO89pwzVquWce/ttKFfO3VwiIiKSrf7880/GjBlDREQEkydPJjU1lZ49e7JixQpCQvQUjJv0ty+SHY4dc6brfukl2LXLOVetGjz0ENxwg7vZREREJFtt3bqV1q1bc+TIEQBuuOEGoqOjueKKK1xOJqCCJOI/1sLGjfDyyzB7NiQmOhMyPPootG0LDRu6nVBERESyibUWYwwANWvWpGzZstSuXZvY2Fhan282W8l2KkgiWSk93ZmWOyEBRoyAvXud8/feCwMGQKNG7uYTERGRbGWtZeHChURHR7No0SIqVapEcHAwn3/+OaVKlcooTRI4VJBELsXs2XDw4F/PzZsHq1b9/bWbNsHll2dPLhEREQkYy5cvZ9iwYazy/nwwZcoU/vOf/wBQunRpN6PJeaggiZzJWli9GvbvP31uxQp44QXf3t+uHUyZAgUKQIUKzppGIiIikmesXr2aYcOGsWzZMgDKli3LqFGjuO+++1xOJr5QQRI5ZfNmp9hMm3bu17RoAf/+NwQHw113QcmSp68Z45QiERERybP+85//MHToUACKFi3Kk08+ySOPPELBggVdTia+UkGSvG3iRPjoI+cOUWY1a0JsLNSocfpchQqg2+EiIiJyhswTMPz73/9m7NixDBgwgKeeeooSJUq4nE4ulgqS5D3x8fDMM86QufR059wVV0DhwjBoEHTuDPnzu5tRREREAt6RI0eIjY1lx44dLFq0CIDGjRuzf/9+ihUr5nI6uVQqSJL3vPACTJoERYpAaCh89pmm3BYRERGfJSQkMGnSJCZMmMDx48cB+PHHH2nQoAGAylEOp4IkeUfbtvDll6ePf/rJGTYnIiIi4oPk5GSmTZvG+PHjOXz4MAAdO3YkJiYmoxxJzqeCJHnDjz865SgyEq6+Grp3VzkSERERn6Wnp9OkSRO2bNkCQIsWLYiNjaVt27YuJ5OspoIkuV9KCjz0kLPfrRsMG+ZuHhEREckRrLVYawkKCiI4OJgbb7wRYwwxMTEZ+5L7GGut2xkuSmRkpF2zZo3bMSTQWAuLFsGnnzqz0pUq5Uy7vW7d6dc0aQJr1jjnRURERM7jiy++YOjQofTr14+7774bgJMnT5IvXz6Cg4NdTicXYoxZa62NvJT36g6S5A6vvQb9+jn7BQvCH39AmzbOMLrDh6FrV+jYUeVIREREzmv9+vUMGzaMJUuWAJCamkrfvn0xxpBfs9zmCSpIkjvExTnb77+H5s3dzSIiIiI5zvbt2xk5ciRz584FoHDhwgwZMoTBgwdrKF0eo4IkOdv48TB3rjMJAziTMIiIiIhchC+++IL27duTnp5OWFgY/fv3JyoqilKlSrkdTVyggiQ5z88/w/z5zuQLY8Y4Q+pCQ2HmTNCYYBEREfFBWloaISHOj8ItW7YkIiKC1q1bM3r0aCpXruxyOnGTCpLkLN9+C61anT42BmbNcmanExEREbmAEydOMHnyZF5++WXWrVtH6dKlCQ0NZePGjYSHh7sdTwJAkNsBRC7KF1842+HDISnJ+VI5EhERkQtISUnhpZdeIiIiguHDh7Nv3z7ee++9jOsqR3KK7iBJzpGefrogjRwJYWHu5hEREZGA5/F4mDNnDiNHjmTnzp0AREZGEhsby7XXXutyOglEKkgS+AYMgBUrYPPm0+f0rJGIiIj44MEHH+TVV18FoE6dOkRHR9OtWzfNTCfnpCF2EtiOHYOpU+HECbj5ZujUCbZvhxB1exERETm75OTkjP2+fftSuXJlZsyYwQ8//ED37t1VjuS89FOmBKYTJ2D5crjtNuf4pptg0iR3M4mIiEhA27hxI8OHDyckJIQPPvgAcGaoi4uLIzQ01OV0klOoIElgev11GDTI2S9TBsaOdTePiIiIBKy4uDhGjRrFu+++i7WWQoUKcfDgQcqXLw+gciQXRUPsJPDs23e6HH3/PezaBYULu5tJREREAs7Bgwfp378/derU4Z133iE0NJRHHnmEuLi4jHIkcrF0B0kCh7Vw663OIrAATZtCs2bOWkciIiIimSQkJFCvXj1+//13goKC6Nu3L2PGjOGyyy5zO5rkcCpIEjiSk51yVK8eNG8OL72kciQiIiIZEhMTCQ8PJygoiEKFCtG7d2/27t3L+PHjqV+/vtvxJJfQEDsJDPv3Q8mSzn7fvs4zSFqwTURERIDU1FSmTZtGjRo1mDdvXsb5SZMmsXDhQpUjyVIqSBIYHngAEhMhXz64806304iIiEgAOLXIa7169XjwwQc5ePDgXwpSsNZFFD9QQRL3eDywaBFcdx18/DGUKAHHj0PFim4nExERERdZa1myZAlNmzbljjvuYMeOHdSsWZO5c+cyZ84ct+NJLqdnkCT7vfsufPstfP45bN58+vyoUc4dJBEREcnT5syZw53eESUVKlRgzJgx9O3bV9N1S7ZQQZLsk5wM778PvXs7RSg8HMLCYNkyaNVKEzKIiIjkYX/88QdFixYFoFu3blxxxRX06tWL/v37kz9/fpfTSV6iIXaSfR57zClHAEOHwh9/QFISXH21ypGIiEgetWvXLvr06UNERATHjh0DIDw8nPXr1/PEE0+oHEm20x0k8b9jx5z1jHbscI43b4a6dd3NJCIiIq46dOgQ0dHRvPLKK6SmphIaGspXX31Fly5dADD65am4xK93kIwxHY0x24wxO4wxQ89yvYox5nNjzHpjzCZjTCd/5hEXvP8+DB58uhx9+qmzzpH+pyciIpIn/fHHH4wcOZKIiAimTJlCWloavXv3Ztu2bRnlSMRNfruDZIwJBqYCHYB9wGpjzIfW2i2ZXjYCeM9a+7Ixph6wGKjqr0ySzd57D267zdkvWBC2boVKldzNJCIiIq7q0aMHn3zyCQA33ngj0dHRXH755S6nEjnNn0PsmgE7rLU7AYwxc4CuQOaCZIEi3v2iwAE/5hF/S0mB2bPhxAl47TXYtMk5P2cO3HwzhGhEp4iISF6TlpZGQkICxYoVA2DIkCGcPHmSZ555hpYtW7qcTuTv/PkTa0Vgb6bjfUDzM14zBvjEGDMQKAhce7YPMsb0A/oBVKlSJcuDShZYvx6aNPn7+S+/hNatsz+PiIiIuMpay/z58xkxYgSRkZHMnj0bgGuvvZb27dvrGSMJWG7/Sv8OYKa19jljTAtgljGmgbXWk/lF1trpwHSAyMhI60JOOZO1sGaNc6fI43G24EzbvWULFC0KxYqBVrgWERHJc5YtW0ZUVBRr1qwBwOPxcOLECQoWLAhoAgYJbP6cpGE/UDnTcSXvuczuBd4DsNZ+B4QDpfyYSbKCtRAV5cxMN326U46KF4fx4+HkSaheHUqWVDkSERHJY1atWkX79u3p0KEDa9asoVy5crz88sts3rw5oxyJBDp/3kFaDdQ0xlTDKUa3A3ee8Zo9QHtgpjGmLk5BOuzHTPJPvfYavP46fPedc/zuu9Ctm3PnSERERPKs/fv307JlS9LT0ylWrBhDhw5l4MCBFChQwO1oIhfFbwXJWptmjBkALAWCgdettZuNMWOBNdbaD4HHgVeNMY/iTNjQ11qrIXSB6vXX4f77Tx8vWQLXXedeHhEREXHVwYMHKVeuHMYYKlasyMMPP0zBggV58sknKV68uNvxRC6JyWl9JDIy0p4azyrZ5H//g4kTYdky5/jjj6GTlqwSERHJqw4fPkxsbCxTp05l/vz5dO7c2e1IIn9hjFlrrY28lPf6daFYyQUWL3bK0LJl0KKFM6RO5UhERCRPOn78OE8//TTVq1dn0qRJpKSksHLlSrdjiWQpt2exk0CWkgLDhzv7b7wBffu6GkdERETckZSUxCuvvEJ0dDRHjhwBoFOnTkRHR9OoUSOX04lkLRUkObuUFKhRA/Z6l7Lq0cPdPCIiIuKaqVOn8sQTTwDQqlUrYmNjaa11DiWX0hA7ObuDB0+Xo0OHQDPQiIiI5BnWWn755ZeM4wceeID27dvz0Ucf8dVXX6kcSa6mO0jyd5995kzEAM7MdWXKuJtHREREss3nn3/O0KFD2b17Nzt27KBQoUIUKlSIZacmaxLJ5VSQ5K/27oX27Z390FCoVs3dPCIiIpIt1q5dS1RUFJ9++ikAZcuWZevWrURGXtJEYCI5lobYyWmrVkGVKs7+Aw9AUhK0a+dqJBEREfGvbdu20aNHDyIjI/n0008pUqQI48ePZ8eOHSpHkifpDpI4Rei55+DDD53jO++EKVMgSP1ZREQkN7PWcvvtt7NhwwbCw8MZOHAgTz31FCVLlnQ7mohrVJAEVq6EESMgJARatYK333Y7kYiIiPjJ0aNH8Xg8lC5dGmMM48aN46OPPmLUqFFUrFjR7XgirtMtAgGPx9kuWwZff+1uFhEREfGLhIQExo8fT/Xq1Rk2bFjG+c6dOzNt2jSVIxEv3UESERERycWSk5OZPn0648ePJz4+HoADBw6Qnp5OcHCwy+lEAo/uIOV1Hg94F34TERGR3CM9PZ1Zs2ZRp04dBg0aRHx8PM2bN+ezzz7j448/VjkSOQfdQcrr7vjZ53wAACAASURBVLwT1q1z9mvVcjeLiIiIZJktW7bQp08fAOrWrUtMTAxdu3bFGONyMpHApoKU173/vrPdsgXKl3c3i4iIiPwjGzZsoFGjRgBcfvnlPP744zRo0IDevXvrjpGIjzTELi9JTYUhQ6BDByhQAIoWdYbYPfUU1K3rdjoRERG5RBs2bKBTp040btyY5cuXZ5x/9tln6du3r8qRyEXQHaS84uRJ+PRTePZZ57h2bUhOhrvucr5EREQkx9mxYwcjR45kzpw5ABQuXJi9e/e6nEokZ1NByisaNoQdO5z9FSugbVtX44iIiMilO3DgAGPHjmXGjBmkpaWRL18+BgwYQFRUFKVKlXI7nkiOpoKUF+zf75Sja6+Fu+92FoMVERGRHGvq1KlMmzaNoKAg7rnnHkaPHk2VKlXcjiWSK6gg5XZffQVt2jj7bdo4s9aJiIhIjpKYmMj27du54oorAHjiiSfYs2cPw4YNo66eIxbJUpqkIbebMsXZ3nKLMxmDiIiI5Bipqam8/PLLREREcOONN5KUlARA8eLFmTVrlsqRiB+oIOVmx4/DkiXODHXvvw/58rmdSERERHzg8Xh45513qFu3Lg8//DC//vorZcuW5ddff3U7mkiup4KUm02e7JSkEiXcTiIiIiI+sNby8ccf06RJE3r27ElcXBy1a9dm3rx5rFq1iqpVq7odUSTX0zNIuVliorP9+GN3c4iIiIhP0tLSGDx4MDt27KBSpUo8/fTT9OnTh5AQ/cgmkl30X1tuFxrqLAgrIiIiAWnTpk2UL1+e0qVLExoayoQJE9i5cycPP/ww4eHhbscTyXM0xC632r0bYmMhNdXtJCIiInIWO3fupFevXjRq1Ijo6OiM8zfddBOPPfaYypGIS3QHKTdKTobrrnP2T03xLSIiIgHh119/Zdy4cUyfPj1jkdd8mkhJJGCoIOVG48bBtm3Ovp4/EhERCQi///47EyZM4PnnnycxMRFjDHfddRdjxozR5AsiAUQFKbeZORNO3abfuBEKFXI1joiIiDh++uknYmJiAOjatSvjx4+nQYMGLqcSkTPpGaTc5sEHne3bb0PDhu5mERERycNSU1P53//+l3HcokULRo0axbfffssHH3ygciQSoFSQcpMhQ5znjwYNgjvvdDuNiIhInuTxeJg7dy7169enU6dOfPfddxnXnn76aVq0aOFiOhG5EA2xyy2shYULnf2+fV2NIiIikhdZa/nkk0+Iiopi/fr1ANSoUYOTJ0+6nExELobuIOUWQ4dCXBx06waNG7udRkREJE/5/vvvueaaa+jYsSPr16+nQoUKTJs2jS1btnDNNde4HU9ELoLuIOUWu3c72/Hj3c0hIiKSB73zzjusWLGC4sWLExUVxYABA8ifP7/bsUTkEqgg5QY7dsDcuVCzJtSr53YaERGRXO+XX37h4MGDGc8TDR8+nKJFi/L4449TrFgxl9OJyD+hIXY52d698MQTTjECuOoqd/OIiIjkcvHx8QwePJjatWvTp08fUlNTAShbtizjxo1TORLJBXQHKaeaNQv69Dl93KEDvPWWe3lERERysT///JPnnnuOiRMnkpCQgDGG5s2bk5CQQPHixd2OJyJZSHeQcqpT6x317w/Hj8Mnn7ibR0REJBdKTk5m0qRJVK9enbFjx5KQkEDnzp3ZsGEDs2fPVjkSyYVUkHKabdugUydITHTWO3rxRShUyO1UIiIiuZK1lokTJ3L06FGuvvpqvvrqKz766CMaajF2kVxLQ+xykoQEaNMG4uOd4/bt3c0jIiKSy1hrWbBgAddccw3FixcnPDycqVOnEhISwvXXX48xxu2IIuJnuoOUk0yd6pSjBg3A44EuXdxOJCIikmssX76cZs2accsttzBhwoSM8126dKFTp04qRyJ5hO4g5QTWwvLlzmKwAF99BfqftIiISJZYvXo1UVFRLF++HIBy5coRERHhcioRcYvuIOUEL7/szFIHUKkSaApRERGRf2z79u3ccsstNGvWjOXLl1O0aFFiYmLYsWMH9957r9vxRMQlKkiBbtQoZ6Y6gJkzYc8eV+OIiIjkFnv27GH+/PmEh4fz1FNPsXPnTqKioihYsKDb0UTERSpIge75553tggVw110aWiciInKJjhw5wqxZszKO27dvz6RJk4iLi+OZZ56hRIkSLqYTkUChghToQkNhwADo1s3tJCIiIjnS8ePHGTt2LNWrV6dPnz5s2LAh49rgwYOpUKGCi+lEJNBokoZA4fFAaip8/TV89hkUKOCcT0x0N5eIiEgOlZyczLRp0xg/fjyHDx8GoGPHjoSFhbmcTEQCmQqS2+bOhQkTYO3ac7+mVq3syyMiIpILzJ49mxEjRrB7924AWrRoQWxsLG3btnU5mYgEOhUkt/3vf7B5M1x/PYSHQ/36cMMN0LTp6deEhrqXT0REJAdasWIFu3fvpkGDBkRHR3PjjTdqHSMR8YkKklsSEpz1jPbuhbJlYfFitxOJiIjkWCtWrCA4OJjWrVsDMHr0aNq2bcudd95JcHCwy+lEJCdRQXLD/v3OekanNGzoXhYREZEcbN26dQwbNoylS5fSoEEDNmzYQHBwMJUrV6Z3795uxxORHEiz2LkhPt7ZtmgBK1eCd+VuERER8c3PP//MbbfdRtOmTVm6dClFihThtttuIy0tze1oIpLD6Q5SdvvxR7jvPmf/qaegWTN384iIiOQgv/32G1FRUcyYMYP09HTCwsIYOHAgQ4cOpWTJkm7HE5FcQAUpu91wA+zZ4+zXqOFuFhERkRwmX758LFy4EID777+fUaNGUSnzsHURkX9IQ+yyk7Vw5Ai0beuse1S/vtuJREREAtqJEyeYMGECCQkJABQqVIi33nqLzZs3M336dJUjEclyKkjZ5d13ISjIWfi1fn3QVKMiIiLnlJKSwtSpU4mIiODJJ5/k+eefz7jWsWNHateu7WI6EcnNNMTOnzweeP11+PZbeOMN51zXrjBkiLu5REREApTH4+Hdd99l5MiR7Nq1C4BmzZplTN8tIuJvKkj+tG4d3H//6eNBg2DyZPfyiIiIBLAVK1YwaNAgfvjhBwDq1q1LdHQ0N910kxZ5FZFso4LkL2lpcOWVzv68ec6doxD9dYuIiJzL8ePH+eGHH6hSpQpPP/00vXv31iKvIpLt9AySv/z8s7OtX9+ZuU7lSERE5C82btzI1KlTM447d+7M7Nmz2bZtG3379lU5EhFXqCD5S+fOznbAAAgPdzeLiIhIAImLi6Nnz540atSIQYMGsXXrVgCMMfTs2ZNw/bspIi7SbQ1/SUyEhg1PLworIiKSxx08eJBx48bx6quvkpaWRr58+Xj44YcpVaqU29FERDKoIPlLcDA0a6ahdSIikudZaxkxYgSTJk3i5MmTBAUFcffddzN69Gguu+wyt+OJiPyFfnrPamlp8PHHcOCA20lEREQCgjGGXbt2cfLkSbp378748eOpW7eu27FERM5KzyBlteHD4aabnP3y5d3NIiIi4oLU1FSmTZvGl19+mXEuJiaGlStXMn/+fJUjEQlouoOU1X77zdl+9x00b+5uFhERkWzk8Xh47733GDlyJDt27ODKK69k5cqVGGOoWrUqVatWdTuiiMgF+VyQjDEFrLWJ/gyT46Wnw/r1ULYsXHWV22lERESyhbWWJUuWMGzYMDZs2ABArVq1ePLJJ11OJiJy8S44xM4Y09IYswXY6j2+whjzkt+T5USvvQZr10L+/G4nERERyRZxcXG0a9eOTp06sWHDBipWrMhrr73G5s2bueWWWzDGuB1RROSi+PIM0iTgOuAogLV2I9DGn6FypORkmDbN2f/gA3eziIiIZJMSJUqwadMmSpQowbPPPsv27du59957CdEsriKSQ/k0SYO1du8Zp9L9kCXneuUVZzHY9eud43r13M0jIiLiJ7t27WLQoEEkJSUBULx4cRYtWsTOnTt5/PHHya9RFCKSw/lSkPYaY1oC1hgTaox5AvjJz7lyjpQUeOghZ79VK2eShtBQdzOJiIhksUOHDjFw4EBq167NlClTmDp1asa1Nm3aULRoURfTiYhkHV8K0oNAf6AisB9oBDzsz1A5yldfOdunn4avv4bixd3NIyIikoX++OMPRowYQUREBC+++CJpaWn06dOH7t27ux1NRMQvfBkgXNta2zPzCWNMK+Ab/0TKQQ4dgmuvdfarVXM3i4iISBZ78803eeyxx/jNu4RFly5diI6OpkGDBi4nExHxH1/uIE3x8Vzec//9zvauu6B3b3eziIiIZLGwsDB+++032rRpwzfffMOiRYtUjkQk1zvnHSRjTAugJVDaGPNYpktFgGB/B8sRTpyAggWdSRpERERyMI/Hw/z58/nll18YMmQIAD169KBcuXK0bdtW03WLSJ5xvjtI+YBCOCWqcKavP4FbfPlwY0xHY8w2Y8wOY8zQc7ymhzFmizFmszHmnYuLHwAaN3ZmsBMREcmBrLV88sknNGvWjB49ejB8+HB++eUXAIKCgmjXrp3KkYjkKee8g2St/QL4whgz01q7+2I/2BgTDEwFOgD7gNXGmA+ttVsyvaYmEAW0stYeM8aUueg/gRs8HvjpJzh+HMLC3E4jIiJySVauXElUVBSff/45ABUqVGD06NFUrFjR5WQiIu7xZZKGRGPMBKA+kHGrxFp7zQXe1wzYYa3dCWCMmQN0BbZkes39wFRr7THvZ8ZfRPbsN2MGDB4MCQmnz3Xo4F4eERGRS5CSksLtt9/OwoULAShWrBhRUVEMGDCAAgUKuJxORMRdvhSkt4G5QGecKb/vAg778L6KQOYFZvcBzc94TS0AY8w3OM81jbHWLjnzg4wx/YB+AFWqVPHhW/tBWhrcd9/p45494eaboWlTd/KIiIhconz58uHxeMifPz+DBw9myJAhFNcyFSIigG8FqaS1doYx5pFMw+5WZ+H3rwm0AyoBXxpjLrfW/p75Rdba6cB0gMjISJtF3/vSjBsHI0a4GkFERORixMfHExMTw2233UaLFi0AmDx5Mvny5aN8+fIupxMRCSy+FKRU7/agMeYG4ABQwof37QcqZzqu5D2X2T5gpbU2FdhljPkZpzBlVQETERHJs/78808mTpzIc889R0JCAhs3bsx43uiyyy5zOZ2ISGDypSCNN8YUBR7HWf+oCDDYh/etBmoaY6rhFKPbgTvPeM0HwB3AG8aYUjhD7nb6mF1ERETOIikpiZdffpno6GiOHj0KwA033EB0dLTLyUREAt8FC5K19v+8u38A/wIwxrTy4X1pxpgBwFKc54tet9ZuNsaMBdZYaz/0Xvu3MWYLkA4MsdYevbQ/ioiIiHz//ff06NGDvXudx4BbtWpFbGwsrVu3djmZiEjOcL6FYoOBHjiTLSyx1v5ojOkMDAPyA40v9OHW2sXA4jPOjcq0b4HHvF8iIiLyD1WvXp3ffvuNyy+/nNjYWDp16qR1jERELsL57iDNwHmGaBXwgjHmABAJDLXWfpAd4UREROT8PvvsM6ZNm8bs2bMJDQ2lTJkyfPfdd9SvX5+goPOtBy8iImdzvoIUCTS01nqMMeHAr0BEnh0CN3as2wlEREQyrFmzhqioKJYtWwZAhw4duM+7HMXll1/uZjQRkRztfL9aSrHWegCstUnAzjxbjgDefNPZamFYERFx0datW7n11lu58sorWbZsGUWLFiUmJoY77rjD7WgiIrnC+e4g1THGbPLuGyDCe2xwHh9q6Pd0geK332DPHujTB5qfudatiIhI9hg5ciQxMTF4PB7Cw8MZNGgQTz31FCVK+LL6hoiI+OJ8BalutqUIZJs2wRVXOPuFC7ubRURE8rRq1aphjOGBBx5g5MiRVKxY0e1IIiK5zjkLkrV2d3YGCVjTpzvbq66C2Fh3s4iISJ6RkJDApEmTCAoKYvjw4QD06dOHNm3aUKNGDZfTiYjkXr4sFJu3WQulSsF337mdRERE8oDk5GSmT5/OuHHjOHz4MPnz5+fBBx+kZMmShISEqByJiPiZ5v8UEREJAOnp6bz11lvUrl2bQYMGcfjwYa666ioWL15MyZIl3Y4nIpJn+HQHyRiTH6hird3m5zwiIiJ5zrFjx2jdujWbN28GoF69esTExNClSxct8ioiks0ueAfJGHMjsAFY4j1uZIz50N/BRERE8orixYtTvnx5LrvsMt588002bdpE165dVY5ERFzgyxC7MUAz4HcAa+0GoJofM4mIiORq69evp1OnTqxfvz7j3Jtvvsm2bdvo06cPwcHBLqYTEcnbfBlil2qt/eOM32JZP+UJLElJkJLidgoREckltm/fzsiRI5k7dy4ABQoUYN68eQBUqFDBzWgiIuLlS0HabIy5Ewg2xtQEBgHf+jdWANiwAZo1g9RUKFfO7TQiIpKDHThwgLFjx/Laa6+Rnp5OWFgY/fv3Jyoqyu1oIiJyBl8K0kBgOJAMvAMsBcb7M1RA2L/fKUeDBkHnzm6nERGRHGrBggX07NmTpKQkgoKCuPfeexk9ejSVK1d2O5qIiJyFLwWpjrV2OE5Jynt69YIrr3Q7hYiI5FDNmjUD4JZbbmHcuHHUqVPH5UQiInI+vhSk54wx5YB5wFxr7Y9+ziQiIpIjpaSk8Oqrr/LBBx+wZMkSgoODqVSpEnFxcXrGSEQkh7jgLHbW2n8B/wIOA9OMMT8YY0b4PZlbrIXmzeGWW5zjIK2lKyIi5+fxeHj77bepU6cOAwYMYNmyZfzf//1fxnWVIxGRnMOnn/6ttb9aa18AHsRZE2mUX1O5bdUqaNQIxoyBhg3dTiMiIgHKWsvHH39M48aN6dWrF7t27aJOnTrMnz+fLl26uB1PREQuwQWH2Blj6gK3ATcDR4G5wON+zuW+jh1h9Gi3U4iISAC79dZbmT9/PgCVK1dmzJgx9OnTh5AQX0awi4hIIPLl/+Cv45Si66y1B/ycx10eDxw75nYKEREJYNZaTq0N2K5dO7744guGDRvGQw89RHh4uMvpRETkn/LlGaQW1trnc305mjcPgoOhVCnnODTU3TwiIhJQ4uLi6NmzJzExMRnn+vXrR1xcHI8++qjKkYhILnHOO0jGmPestT2MMT8ANvMlwFprc8/DOWlpEBvr7HfvDtdcAz16uJtJREQCwsGDBxk/fjzTp08nLS2N0qVL88QTTxAWFka+fPnIly+f2xFFRCQLnW+I3SPebe5eJXXWLOjTx9kvUwa8Y8lFRCRv+/333/nvf//L5MmTSUxMJCgoiL59+zJmzBjCwsLcjiciIn5yzoJkrT3o3X3YWvtU5mvGmP8AT/39XTnMvn2ny1FEBCxb5m4eEREJCLt27aJp06Yc8z6XetNNNzF+/Hjq16/vcjIREfE3X6b57nCWc9dndRBXHD/ubGNjYccOqFrV1TgiIuIea0+PJq9atSp169alXbt2fPfddyxcuFDlSEQkjzhnQTLGPOR9/qi2MWZTpq9dwKbsi+hHTZs622rV3M0hIiKu8Xg8zJ07l3r16rF161YAjDEsXryYzz77jKuuusrlhCIikp3OdwfpHeBG4EPv9tRXU2ttr2zI5l/WwsmTUKEC3HCD22lERCSbWWtZsmQJkZGR3H777WzdupWpU6dmXC9atGjGdN4iIpJ3nK8gWWvtL0B/4HimL4wxJfwfzc+6dnW2DzwAhQq5m0VERLLVd999x7/+9S+uv/561q9fT4UKFZg+fToTJ050O5qIiLjsfLPYvYMzg91anGm+M/8azQLV/ZjL/7Zscbb33ONuDhERyVYTJ07k8ccfB6BEiRJERUXRv39/8ufP73IyEREJBOe8g2St7ezdVrPWVvduT33l7HIEEBQEd9wBlSq5nURERPwsPT09Y79Lly4UK1aMESNGsHPnTp544gmVIxERyXC+O0gAGGNaARustSeMMb2AJsDz1to9fk8nIiLyD8THxxMdHc26dev48ssvMcZQo0YN9u/fT4ECBdyOJyIiAciXab5fBhKNMVcAjwNxwCy/phIREfkH/vjjD0aNGkX16tV54YUX+Oabb1izZk3GdZUjERE5F18KUpp1FofoCrxorZ0KFPZvLBERkYuXlJTEc889R0REBOPGjePEiRPceOONbNy4kSuvvNLteCIikgNccIgdcNwYEwX0BlobY4KAUP/GEhERuTjWWpo3b86mTc5Sfa1bt+aZZ56hZcuWLicTEZGcxJeCdBtwJ3CPtfZXY0wVYIJ/Y4mIiFyYtZb09HRCQkIwxtCzZ0+CgoKIiYmhY8eOWsdIREQu2gWH2FlrfwXeBooaYzoDSdbat/yeTERE5DyWLVtGs2bNeOGFFzLOPfroo6xdu5brr79e5UhERC7JBQuSMaYHsAq4FegBrDTG3OLvYCIiImezatUq2rdvT4cOHVizZg1vvPEGzqOyEBoaSlCQL4/XioiInJ0vQ+yGA1daa+MBjDGlgWXAPH8GExERyeynn35ixIgRLFiwAIBixYoxdOhQBg4cqLtFIiKSZXwpSEGnypHXUXyb/U5ERCRLrFq1ihYtWuDxeMifPz+PPPIITz75JMWLF3c7moiI5DK+FKQlxpilwLve49uAxf6LlA2OHYPt2yEy0u0kIiJyDklJSYSHhwMQGRlJZGQkTZs2ZeTIkZQvX97ldCIikltdsCBZa4cYY7oDV3tPTbfWLvRvLD976CFnW1jLOYmIBJrjx48zceJEpkyZwurVq6lWrRpBQUF88803hIT48ns9ERGRS3fOoXLGmJrGmEXGmB9xJmh4zlr7WI4vRwDHjzvbZ591N4eIiGRITk5m8uTJVK9enTFjxnD06FEWLjz9T47KkYiIZIfzPUv0OvB/wM3AWmBKtiTKLpGRuoMkIhIA0tPTmTlzJrVq1WLw4MEcOXKEli1b8sUXX/DYY4+5HU9ERPKY8/06rrC19lXv/jZjzLrsCCQiInnLo48+ypQpzu/gGjRoQExMDJ07d9bMdCIi4orzFaRwY0xj4NS/UPkzH1trVZhEROSSJCQkUKhQIQAeeOABFi9ezJgxY7jjjjsIDg52OZ2IiORl5ytIB4GJmY5/zXRsgWv8FUpERHKntWvXEhUVRXJyMitWrMAYQ/369fn555+1wKuIiASEcxYka+2/sjOIiIjkXtu2bWPkyJG8//77ABQpUoTdu3dTtWpVAJUjEREJGPoXSURE/Gbfvn3cf//91K9fn/fff5/w8HCGDBnCzp07M8qRiIhIIMl7c6YuWgSLF2uRWBERP0tKSqJx48YcOXKE4OBg7r//fkaNGkWlSpXcjiYiInJOea8gff65s+3f390cIiK5UEJCAmFhYYSGhhIeHk7//v356aefGDduHLVq1XI7noiIyAVdcIidcfQyxozyHlcxxjTzfzQ/KloU+vZ1O4WISK6RkpLCiy++SEREBDNmzMg4P3r0aObOnatyJCIiOYYvzyC9BLQA7vAeHwem+i2RiIjkGOnp6cyaNYvatWszcOBA4uPjWbp0acZ1rWUkIiI5jS8Fqbm1tj+QBGCtPQbk82sqEREJaNZaPvzwQxo1akSfPn345ZdfqFevHgsXLmTBggVuxxMREblkvjyDlGqMCcZZ+whjTGnA49dU/rJvH0yeDAUKuJ1ERCRHW7RoEd26dQOgSpUqjB07ll69emmRVxERyfF8KUgvAAuBMsaYaOAWYIRfU/nLhx862yZN3M0hIpIDxcfHU6ZMGQA6d+5MmzZt6N69Ow8++CBhYWEupxMREckaFyxI1tq3jTFrgfaAgf9v786jq6oOt49/dxJAEKQKCg6gUgIoYBVSKxVFC4pWRXFAsLZVKYKobRGsylQkSECKVStVW1FwaLGKAzi29SdKcYKqBQREHBDB1gGICgRMst8/cuGllCEgycm9+X7Wct17zzn33gfXWZAne599OCvGuLDCk1WEGMsenf4hSeW2ZMkShg0bxrRp03jnnXc44IADyMnJYcaMGV5jJEnKOOVZxa4psBaYDkwD1qS2pZ+lS5NOIElpY8WKFfTr14/DDjuMKVOmUFJSwssvv7xpv+VIkpSJyjPF7knKrj8KwB7AocDbQOsKzLX7zZkD48aVPa/pGhOStC2rVq1i7Nix3Hrrraxbt46srCwuueQSfv3rX9O0aXr+fkySpPIqzxS7tpu/DiG0A/pXWKKKsnJl2ePYsWX3QZIkbVXv3r159NFHATjnnHPIz8/nsMMOSziVJEmVozwjSP8lxvh6COF7FRGmUnTsmHQCSapSvv76a1auXEmjRo0AGDx4MF988QWjR4/m6KPT+77gkiTtrB0WpBDCVZu9zALaASsqLJEkqVKUlpby4IMPMmzYMHJzc3n66acByMvL4+9//3vC6SRJSkZ5RpDqbfa8mLJrkqZWTBxJUkWLMfL0008zePBg/vWvfwGQnZ3NqlWr2HvvvRNOJ0lSsrZbkFI3iK0XYxxUSXkkSRVo1qxZXHfddcycOROAgw46iBEjRvDTn/6UnJydnnUtSVLG2ea/hiGEnBhjcQjh2MoMJEmqGJ9//jldunShqKiIBg0acN1119G/f39q166ddDRJkqqM7f268DXKrjd6M4QwDXgIWLNxZ4zRu61KUhX3wQcf0KRJE7Kzs2nQoAHXXHMNpaWlDBw4kPqu6ClJ0v8oz3yKPYDPgR/w/++HFIH0KUgxwm23JZ1CkirNv//9b0aNGsWdd97JxIkT+clPfgLAiBEjkg0mSVIVt72CtF9qBbv5/P9itFGs0FS727JlMH162fODDko2iyRVoNWrVzNu3Dhuvvlm1q5dSwiBhQsXJh1LkqS0sb2ClA3U5b+L0UbpVZBKSsoeJ00C7wIvKQOtW7eO2267jYKCAlatWgXAmWeeyahRo2jTpk3C6SRJSh/bK0gfxxhHVloSSdIuu/fee/nVr34FQKdOnSgoKKBDhw4Jp5IkKf1kbWff1kaOJElVQGlpmJrlAgAAIABJREFU6X9Nnbv44os566yzeOaZZ3j++ectR5Ik7aLtFaTOlZZCklQuMUaeffZZ8vLy+N73vsdnn30GQM2aNXn00Ufp2rUrIfj7LUmSdtU2C1KMcWVlBpEkbd8rr7zCD37wA0455RTeeOMN6tWrx+LFi5OOJUlSRtneCJIkqQp466236N69Ox06dGDGjBnsvffejB07lnfeeYfvf//7SceTJCmjVGhBCiGcEkJ4O4SwJIRw7XaOOyeEEEMIeRUSZNKkCvlYSaoM/fv357HHHqNOnToMHjyY9957j1/96lfUqVMn6WiSJGWc8twodpeEELKBCcBJwEfA7BDCtBjjgi2Oqwf8Ani1orIwMrUYX25uhX2FJO0un3zyCevXr6dJkyYA3HDDDUyZMoWhQ4fSuHHjhNNJkpTZKnIE6WhgSYzxvRjjBmAKcOZWjssHxgJFFZYkOxuGDAGnokiqwr744guGDx9Os2bNGDBgwKbtHTt25LbbbrMcSZJUCSqyIB0ILNvs9UepbZuEENoBTWKMT1ZgDkmq0oqKirjpppto1qwZ+fn5rFmzhvXr17Nhw4ako0mSVO0ktkhDCCELuAkYWI5jLw0hzAkhzPn0008rPpwkVYLi4mLuvvtuWrRowcCBA/n888/p2LEjM2fOZPr06dSsWTPpiJIkVTsVWZCWA002e31QattG9YA2wIwQwgfAMcC0rS3UEGP8Q4wxL8aYt+++++5cipKSsv8kqYpZtmwZffv2ZdmyZRxxxBE8+eSTvPjii3Ts2DHpaJIkVVsVtkgDMBvIDSEcSlkx6glcsHFnjLEQaLjxdQhhBjAoxjhnt6b44Q/LHmvU2K0fK0m74qWXXqJDhw6EEDj00EMZMWIEzZo14/zzzycryzsvSJKUtAr71zjGWAxcATwLLAT+EmN8K4QwMoTQraK+93988EHZY9++lfaVkrSl2bNnc9JJJ3HsscfyyCOPbNo+ZMgQevXqZTmSJKmKqMgRJGKMTwFPbbFt+DaOPaFCQmRlQY8e4OpPkhKwaNEihg4dytSpUwGoX78+hYWFCaeSJEnbktm/snzySVi0KOkUkqqhZcuW8bOf/YzWrVszdepU9thjD6655hree+89LrnkkqTjSZKkbajQEaRE3XMPbPwhpHPnZLNIqnb+/Oc/M3HiRLKzs+nbty/Dhg3jwAMP3PEbJUlSojK3IM2fX/b4zDPQtWuyWSRlvC+//JL58+fToUMHAK644gref/99rrrqKnJzcxNOJ0mSyiuzp9jVrWs5klSh1q9fz6233sq3v/1tzjjjjE3XF9WpU4fbb7/dciRJUprJ3IL0+edQXJx0CkkZqqSkhMmTJ9OyZUt+8Ytf8Omnn9KiRQu8mbUkSektMwvStGkweTKEkHQSSRkmxsjjjz/OEUccwUUXXcTSpUtp06YNjz/+OLNmzaJ58+ZJR5QkSd9AZl6DtGJF2eM99ySbQ1LGiTFy/fXXs2DBAg455BBGjhzJBRdcQHZ2dtLRJEnSbpCZBWmjTp2STiApA7z++us0bNiQpk2bkpWVxfjx43nrrbfo06cPtWrVSjqeJEnajTJzip0k7QaLFy/m/PPPp3379gwdOnTT9hNPPJErrrjCciRJUgbK7BEkSdoFy5cvZ+TIkUycOJGSkhJq1apF48aNiTESvLZRkqSMZkGSpJSVK1cyZswYfve731FUVERWVhY/+9nPGD58OE2aNEk6niRJqgSZV5A+/xwuuyzpFJLS0Mcff8z48eMpLS3lvPPOIz8/n5YtWyYdS5IkVaLMuwbpzTfLHps0gYYNk80iqUrbsGEDDz30EDFGAFq3bs348eOZPXs2f/nLXyxHkiRVQ5lXkDZ64AHIybwBMknfXElJCffffz+tWrWiR48ePPPMM5v2/fKXvyQvLy/BdJIkKUk2CEnVRoyRJ598ksGDBzNv3jwADjvsMGrWrJlwMkmSVFVk7giSJG1m1qxZHHfccZxxxhnMmzePpk2bcs899zBv3jw6d+6cdDxJklRFZNYIUnGxCzRI2qq//e1vzJo1i4YNGzJ06FD69evnfYwkSdL/yKyCtGIFvPNO2fNWrZLNIilR7777Lu+//z5dunQB4KqrrqJmzZpceeWV1KtXL+F0kiSpqsrMKXYTJ8K++yadQlICPv74Y/r370+rVq348Y9/zJo1awDYa6+9GDx4sOVIkiRtV2aNIEmqtlatWsWNN97ILbfcwrp168jKyuLUU0+lqKiIPffcM+l4kiQpTWRWQbr11qQTSKpkGzZs4Le//S1jxoxh9erVAHTv3p1Ro0Zx+OGHJ5xOkiSlm8wqSOPHlz22bZtsDkmVJisri8mTJ7N69WpOPPFECgoK+N73vpd0LEmSlKYyqyDVrAlXXQXf/W7SSSRVkNLSUh588EE6derEAQccQE5ODhMmTKC4uJguXboQQkg6oiRJSmOZuUiDpIwTY+Tpp5+mXbt2XHDBBeTn52/ad+KJJ3LSSSdZjiRJ0jeWWSNIkjLSSy+9xHXXXceLL74IwEEHHeQ0OkmSVCEsSJKqrMWLFzNo0CCmT58OwD777MPgwYPp378/tWvXTjidJEnKRBYkSVXWl19+yfTp09lzzz0ZMGAAgwYNon79+knHkiRJGSxzrkEqKIANG5JOIekb+M9//sOECRM2vW7fvj0TJ07k3XffJT8/33IkSZIqXOaMIN1/f9nj6acnm0PSTissLGTcuHHcfPPNrFmzhtatW3PCCScAcMkllyQbTpIkVSuZUZBWrYIFC+Ccc+DYY5NOI6mc1q1bx4QJEygoKGDlypUAdOvWjcaNGyecTJIkVVfpX5BihLPPLnu+117JZpFUbvfeey+DBw9m+fLlABx//PGMGTOGDh06JJxMkiRVZ+lfkMaPhxkzICcHbrkl6TSSymn+/PksX76cI488koKCArp27ep9jCRJUuLSvyD95z9lj2+8AfXqJZtF0lbFGPnb3/5GcXExP/zhDwG45ppraNeuHT169CArK3PWi5EkSektM34qqVMH2rRJOoWkrXj11Vfp3LkzXbt2pX///qxfvx6ABg0a0LNnT8uRJEmqUvzJRFKFWLBgAd27d+eYY47h+eefZ++996Z///7EGJOOJkmStE3pP8VOUpWyevVqBgwYwL333ktpaSm1a9dmwIABXH311XzrW99KOp4kSdJ2WZAk7VZ16tThhRdeICsri379+jF06FD233//pGNJkiSViwVJ0jfyxRdfcPPNN9O/f38aNmxIzZo1mTx5MgcccADf/va3k44nSZK0UyxIknZJUVERd9xxBzfccAOfffYZhYWFjB8/HoDjjjsu4XSSJEm7xoIkaacUFxdz33338etf/5ply5YBcOyxx9K9e/eEk0mSJH1zFiRJ5fbCCy9w2WWXsXDhQgDatm3L6NGjOe2007zJqyRJyggWJEnllp2dzcKFCzn00EPJz8+nZ8+eZGdnJx1LkiRpt0nv+yC9+CL85jdQVJR0EikjzZkzhxtuuGHT644dO/L444+zaNEifvSjH1mOJElSxknvgrRoUdnj6NHJ5pAyzKJFizjvvPP47ne/y9ChQ5k9e/amfd26daNmzZoJppMkSao4mTHF7sc/TjqBlBGWLVvG9ddfzz333ENpaSl77LEHV155pct1S5KkaiN9C9Knn0LfvkmnkDLG4MGDuemmm1i/fj3Z2dlceumlDB8+nAMPPDDpaJIkSZUmfQvS/Pllj7m50KhRslmkDFBYWMj69es5//zzGTlyJC1atEg6kiRJUqVL34K00R//CF4oLu2U9evX84c//IFDDz2U008/HYBhw4bRu3dv2rVrl3A6SZKk5KR/QZJUbiUlJTzwwAMMHz6cpUuX0qJFC0455RRycnJo3LgxjRs3TjqiJElSoixIUjUQY2TatGkMGTKEt956C4DDDz+c0aNHu1S3JEnSZixIUob74IMPuOCCC3j55ZcBOPjgg7n++uu58MILLUeSJElbsCBJGa5Ro0YsXbqUfffdl6FDh9K3b19q1aqVdCxJkqQqKb1vFCvpf7zzzjv07t2bwsJCAGrXrs20adN49913+fnPf245kiRJ2g4LkpQhVqxYQb9+/TjssMO4++67+c1vfrNpX/v27alXr16C6SRJktKDU+ykNLdy5UrGjh3LrbfeSlFREVlZWfTu3ZtLL7006WiSJElpx4IkpbH77ruPK6+8ctN0unPPPZf8/HxatWqVcDJJkqT0ZEGS0th+++1HYWEhXbp0YfTo0Xz3u99NOpIkSVJaS9+CtGhR0gmkSlVaWsqUKVOYP38+o0ePBuDkk0/m1Vdf5eijj044nSRJUmZIz4K0ciX071/2vG7dZLNIFSzGyFNPPcXgwYOZO3cuIQR+9KMf0bp1a0IIliNJkqTdKD1XsVu7tuyxTx9o1y7ZLFIF+sc//sHxxx/P6aefzty5c2nSpAl33XUXLVu2TDqaJElSRkrPEaS33y57PPpoCCHZLFIFKCkp4eyzz2batGkANGjQgCFDhnDZZZexxx57JJxOkiQpc6VnQTr99LLHvfZKNodUQbKzs9lnn32oW7cuAwcO5KqrrmIvz3dJkqQKF2KMSWfYKXl5eXHOG2/AiSfCX/8KWek5S1Da3Mcff8yoUaM444wzOOWUUwD497//TVZWFvvtt1/C6SRJktJLCOGfMca8XXlv+o0grV8PpaXQvr3lSGlv9erV3Hjjjdxyyy2sXbuWV155ha5duxJCoHHjxknHkyRJqnbSryC9917Z4957J5tD+gbWrl3LbbfdxpgxY1i1ahUAZ511FqNGjSJ4XZ0kSVJi0q8glZZCixZw9dVJJ5F2yT//+U+6devGihUrADjhhBMoKCjgmGOOSTiZJEmS0q8ghQCtW0N2dtJJpF3SokULNmzYQLt27SgoKOCkk05y1EiSJKmK8CIeqQLFGHnmmWc49dRTWbNmDQD16tXj5ZdfZvbs2Zx88smWI0mSpCrEgiRVkJdffpkTTzyRU089lWeeeYY777xz077mzZuT5SIjkiRJVY4/oUm72fz58znzzDP5/ve/zwsvvMDee+/NuHHjuOyyy5KOJkmSpB1Iv2uQpCpsxIgRjBw5khgjderUYcCAAQwaNIhvfetbSUeTJElSOViQpN3oyCOPJCcnh759+zJkyBDvZSRJkpRmLEjSLiosLGT8+PGsWbOG8ePHA3DmmWfy7rvv0qRJk4TTSZIkaVdYkKSdVFRUxIQJExg9ejQrV64kJyeHgQMHcsABBxBCsBxJkiSlsQpdpCGEcEoI4e0QwpIQwrVb2X9VCGFBCGFuCOG5EMLBFZlH+iaKi4uZOHEiubm5DBo0iJUrV3LcccfxwgsvcMABByQdT5IkSbtBhY0ghRCygQnAScBHwOwQwrQY44LNDnsDyIsxrg0hXAbcCJxfUZmkXbVmzRry8vJYtGgRAN/5zncoKCjglFNO8T5GkiRJGaQip9gdDSyJMb4HEEKYApwJbCpIMcbnNzv+FeDCCswj7bI999yTtm3b8vXXX5Ofn8/555/vfYwkSZIyUEX+hHcgsGyz1x+ltm1Lb+DpCswjldtrr71G586dmTFjxqZtt99+OwsXLqRXr16WI0mSpAxVJRZpCCFcCOQBnbax/1LgUoDv1KhRiclU3SxcuJChQ4fyyCOPAFCjRg1OOOEEABo0aJBgMkmSJFWGivw1+HJg8+W8Dkpt+y8hhC7AEKBbjHH91j4oxviHGGNejDEvJ6dKdDplmA8//JBLLrmENm3a8Mgjj1C7dm2uvfZa/vznPycdTZIkSZWoItvGbCA3hHAoZcWoJ3DB5geEEI4C7gROiTF+UoFZpG2aPn065557Lhs2bCAnJ4dLL72UYcOGuTKdJElSNVRhBSnGWBxCuAJ4FsgG7o4xvhVCGAnMiTFOA8YBdYGHUiuBfRhj7FZRmaSNYoybVp879thjqVOnDueccw4jR46kefPmCaeTJElSUip0vlqM8SngqS22Dd/seZeK/H5pS0VFRdxxxx088MAD/OMf/6BWrVrss88+LFmyxGuMJEmSVLE3ipWqipKSEiZNmkTLli0ZMGAAc+bMYerUqZv2W44kSZIEFiRluBgjjz76KEcccQQXX3wxH374IW3atGH69On06tUr6XiSJEmqYlwSThntggsuYMqUKQAccsgh5Ofn06tXL7KzsxNOJkmSpKrIESRlnNLS0k3PTz/9dBo1asRtt93G22+/zYUXXmg5kiRJ0jZZkJQx3n77bXr06MHVV1+9aVuvXr1YsmQJl19+OTVr1kwwnSRJktKBBUlp76OPPqJPnz60bt2ahx56iIkTJ/LVV18BkJWVRd26dRNOKEmSpHRhQVLa+vzzzxk0aBDNmzfnrrvuAqBPnz7Mnz/fUiRJkqRd4iINSksff/wxrVq14osvvgCgR48e5Ofn06JFi4STSZIkKZ1ZkJQ2SkpKNi2wsP/++3Pcccfx9ddfM3r0aNq3b59wOkmSJGUCp9ipyispKeG+++4jNzeX1157bdP2v/zlLzz77LOWI0mSJO02FiRVWTFGpk+fzpFHHslPfvIT3n//ff74xz9u2l+nTp0E00mSJCkTWZBUJb344ot07NiRbt26MX/+fJo2bcqkSZO44447ko4mSZKkDOY1SKpyfv/733P55ZcDsO+++zJkyBD69etHrVq1Ek4mSZKkTOcIkqqEDRs2bHp+9tln07hxY66//nreffddfvGLX1iOJEmSVCnSbwRp3bqkE2g3WrFiBfn5+cycOZM333yTnJwcGjduzNKlS6lZs2bS8SRJklTNpOcI0iWXJJ1A39CqVau49tprad68OXfccQcLFy5k1qxZm/ZbjiRJkpSE9CtIe+wBp5+edArtorVr1zJmzBiaNWvG2LFjWbduHWeffTbz58+nU6dOSceTJElSNZd+U+yUtmKMnHDCCcyePRuAH/zgBxQUFHD00UcnnEySJEkqY0FShSotLWXDhg3ssccehBDo06cPpaWljBkzhi5duiQdT5IkSfov6TfFTmkhxshTTz3FUUcdxQ033LBp+yWXXMLs2bMtR5IkSaqSLEja7WbNmkWnTp047bTTmDt3Lg8//DAlJSUAZGdnE0JIOKEkSZK0dRYk7Tbz5s2jW7dudOzYkZkzZ9KgQQPGjx/PG2+8QXZ2dtLxJEmSpB3yGiTtFvPnz+c73/kOMUb23HNPrrrqKgYOHEj9+vWTjiZJkiSVmwVJu+zLL7+kXr16ALRu3ZquXbvSokULhgwZwn777ZdwOkmSJGnnOcVOO2316tUMGTKEAw44gLlz5wIQQuDJJ5/klltusRxJkiQpbVmQVG7r1q1j3LhxNGvWjNGjR/PVV1/xxBNPbNqfleXpJEmSpPTmFDvtUHFxMffccw8jRoxgxYoVAHTq1ImCggI6dOiQcDpJkiRp97EgaYeGDBnCjTfeCMBRRx1FQUEBJ598sst1S5IkKeM4J0r/I8bIypUrN73u378/bdq0YcqUKcyZM4euXbtajiRJkpSRHEHSf3nllVe47rrrWLVqFa+//jpZWVkcfPDBzJ0711IkSZKkjOcIkgB46623OOuss+jQoQMzZszgww8/ZPHixZv2W44kSZJUHViQqrmlS5dy0UUX0bZtWx5//HHq1KnD4MGDee+992jVqlXS8SRJkqRK5RS7aqy4uJiOHTvy0UcfkZOTQ9++fRk6dCiNGzdOOpokSZKUCAtSNfPFF1+Qk5NDnTp1yMnJ4Ve/+hWvvvoqI0eOpFmzZknHkyRJkhLlFLtqoqioiJtuuolmzZpxyy23bNp+5ZVXcv/991uOJEmSJBxBynjFxcXce++9jBgxgmXLlgHw0ksvEWN04QVJkiRpC44gZagYI1OnTqVt27b07t2bZcuWccQRR/Dkk08ybdo0y5EkSZK0FY4gZajnnnuOc889F4BmzZqRn59Pz549ycqyE0uSJEnbYkHKIMuWLaNJkyYAdO7cmbPOOouTTz6Z3r17U7NmzYTTSZIkSVWfwwkZYNGiRZx77rk0a9aMd955Byi7seujjz7KZZddZjmSJEmSysmClMaWLVvGz372M1q3bs3UqVPJycnhn//8Z9KxJEmSpLRlQUpDn332GQMHDiQ3N5eJEycSQqBfv368++679OzZM+l4kiRJUtryGqQ0NGjQICZPngxAz549GTlyJLm5uQmnkiRJktKfBSkNrF+/nv/85z80bdoUgCFDhvDZZ5+Rn5/PUUcdlXA6SZIkKXM4xa4KKykpYfLkybRs2ZIePXoQYwQgNzeXJ554wnIkSZIk7WYWpCooxshjjz3GEUccwUUXXcTSpUv56quv+OSTT5KOJkmSJGU0C1IVM2PGDDp06ED37t1ZsGABhxxyCPfeey//+te/aNSoUdLxJEmSpIzmNUhVyJdffslZZ51FYWEh++23H8OGDaNPnz7UqlUr6WiSJElStWBBStg777xD06ZNqVWrFvXq1WPkyJF88cUX/PKXv6Ru3bpJx5MkSZKqFafYJWT58uX07duXww47jDvvvHPT9p///OcMHTrUciRJkiQlwBGkSrZy5UrGjBnD7373O4qKisjKymLFihVJx5IkSZKEBanSrFmzhptvvplx48ZRWFgIwHnnnUd+fj4tW7ZMOJ0kSZIksCBVmunTpzN06FAATjrpJEaPHk1eXl7CqSRJkiRtzoJUQUpKSnjzzTdp3749AD169OCZZ57hxz/+MZ07d044nSRJkqStcZGG3SzGyBNPPMFRRx3F97//fT788EMAsrKymDRpkuVIkiRJqsIsSLvRzJkz6dixI2eccQbz5s2jcePGLF26NOlYkiRJksrJgrQbvPnmm5x22mkcf/zxvPTSSzRs2JDf/va3vP322xx33HFJx5MkSZJUTl6DtBsMHz6cp556irp16zJo0CAGDBjAXnvtlXQsSZIkSTvJgrQLPv74Y7766ityc3MBGDVqFM2aNWPIkCHsu+++CaeTJEmStKtCjDHpDDslr3btOGfdukS+e9WqVdx4443ccsstHHPMMTz33HOEEBLJIkmSJGnrQgj/jDHu0j11HEEqh7Vr1/K73/2OMWPGsHr1agD23ntv1q1bR506dRJOJ0mSJGl3cZGG7fj666+54447aN68Oddeey2rV6/mxBNP5JVXXmHq1KmWI0mSJCnDOIK0HStXrmTgwIGsXbuW9u3bU1BQQJcuXZxWJ0mSJGUoC9JmYow899xzdOrUiRo1atCoUSNuvPFG9ttvP8455xyyshxwkyRJkjKZP/GnzJo1i06dOnHSSScxadKkTdsvv/xyzjvvPMuRJEmSVA1U+5/6582bR7du3ejYsSMzZ85kn332sQxJkiRJ1VS1nWL3/vvvM3z4cB544AFijOy5554MGDCAQYMGUb9+/aTjSZIkSUpAtS1If//737n//vupUaMG/fr1Y8iQITRq1CjpWJIkSZISVG0KUmFhIbNnz6ZLly4AXHzxxSxevJj+/ftz6KGHJpxOkiRJUlUQYoxJZ9gpebVrxznr1pX7+HXr1jFhwgQKCgooKirivffec6RIkiRJymAhhH/GGPN25b0ZuxpBcXExd911F7m5uVx99dWsXLmSvLw8CgsLk44mSZIkqYrKuIIUY+Thhx+mdevW9OnTh+XLl3PkkUfy9NNPM2PGDFq0aJF0REmSJElVVEZeg3T77bezePFivv3tbzNq1Ch69Ojh0t2SJEmSdigjCtKrr75K/fr1adWqFSEExo4dy5w5c+jduzc1atRIOp4kSZKkNFGhwyohhFNCCG+HEJaEEK7dyv5aIYQHU/tfDSEcsjOfv2DBArp3784xxxzD1VdfvWl7Xl4e/fr1sxxJkiRJ2ikVVpBCCNnABOBU4HCgVwjh8C0O6w2sijE2B34LjC3PZy9dupSLL76Ytm3b8thjj1G7dm3atGlDSUnJ7vwjSJIkSapmKnKK3dHAkhjjewAhhCnAmcCCzY45ExiRev4wcFsIIcTtrD2+7OuvadGiBRs2bCAnJ4e+ffsybNgw9t9//4r5U0iSJEmqNipyit2BwLLNXn+U2rbVY2KMxUAh0GB7H/ppSQkbNmzgggsuYNGiRfz+97+3HEmSJEnaLdJikYYQwqXApamX64H5f/rTn/jTn/6UYCpVEw2Bz5IOoWrFc06VyfNNlcnzTZWp5a6+sSIL0nKgyWavD0pt29oxH4UQcoD6wOdbflCM8Q/AHwBCCHN29a640s7yfFNl85xTZfJ8U2XyfFNlCiHM2dX3VuQUu9lAbgjh0BBCTaAnMG2LY6YBP009Pxf4v+1dfyRJkiRJFanCRpBijMUhhCuAZ4Fs4O4Y41shhJHAnBjjNGAicF8IYQmwkrISJUmSJEmJqNBrkGKMTwFPbbFt+GbPi4DzdvJj/7Aboknl5fmmyuY5p8rk+abK5PmmyrTL51twRpskSZIklanIa5AkSZIkKa1U2YIUQjglhPB2CGFJCOHareyvFUJ4MLX/1RDCIZWfUpmiHOfbVSGEBSGEuSGE50IIByeRU5lhR+fbZsedE0KIIQRXfdIuK8/5FkLokfo77q0QgvfQ0DdSjn9Tm4YQng8hvJH6d/WHSeRU+gsh3B1C+CSEMH8b+0MI4dbUuTg3hNCuPJ9bJQtSCCEbmACcChwO9AohHL7FYb2BVTHG5sBvgbGVm1KZopzn2xtAXozxCOBh4MbKTalMUc7zjRBCPeAXwKuVm1CZpDznWwghF7gOODbG2Br4ZaUHVcYo599xQ4G/xBiPomyBrt9XbkplkEnAKdvZfyqQm/rvUuD28nxolSxIwNHAkhjjezHGDcAU4MwtjjkTmJx6/jDQOYQQKjGjMscOz7cY4/MxxrWpl69Qdl8vaVeU5+83gHzKfvFTVJnhlHHKc771ASbEGFcBxBg/qeSMyizlOecisFfqeX1gRSXmUwaJMb5I2UorW9c0AAAG0UlEQVTY23ImcG8s8wrwrRDC/jv63KpakA4Elm32+qPUtq0eE2MsBgqBBpWSTpmmPOfb5noDT1doImWyHZ5vqSkATWKMT1ZmMGWk8vz91gJoEUKYFUJ4JYSwvd/GSjtSnnNuBHBhCOEjylY7vrJyoqka2tmf8YAKXuZbyjQhhAuBPKBT0lmUmUIIWcBNwEUJR1H1kUPZ9JMTKBsdfzGE0DbGuDrRVMpkvYBJMcbxIYQOlN0Ts02MsTTpYBJU3RGk5UCTzV4flNq21WNCCDmUDdF+XinplGnKc74RQugCDAG6xRjXV1I2ZZ4dnW/1gDbAjBDCB8AxwDQXatAuKs/fbx8B02KMX8cY3wcWU1aYpF1RnnOuN/AXgBjjy8AeQMNKSafqplw/422pqhak2UBuCOHQEEJNyi7gm7bFMdOAn6aenwv8X/SmTto1OzzfQghHAXdSVo6cn69vYrvnW4yxMMbYMMZ4SIzxEMqueesWY5yTTFylufL8e/oYZaNHhBAaUjbl7r3KDKmMUp5z7kOgM0AI4TDKCtKnlZpS1cU04Cep1eyOAQpjjB/v6E1VcopdjLE4hHAF8CyQDdwdY3wrhDASmBNjnAZMpGxIdgllF2f1TC6x0lk5z7dxQF3godRaIB/GGLslFlppq5znm7RblPN8exY4OYSwACgBro4xOiNDu6Sc59xA4I8hhAGULdhwkb/k1q4IIfyZsl/wNExd0/ZroAZAjPEOyq5x+yGwBFgLXFyuz/V8lCRJkqQyVXWKnSRJkiRVOguSJEmSJKVYkCRJkiQpxYIkSZIkSSkWJEmSJElKsSBJkjYJIZSEEN7c7L9DtnPsV7vh+yaFEN5PfdfrIYQOu/AZd4UQDk89H7zFvpe+acbU52z8/zI/hDA9hPCtHRx/ZAjhh7vjuyVJlctlviVJm4QQvoox1t3dx27nMyYBT8QYHw4hnAz8JsZ4xDf4vG+caUefG0KYDCyOMd6wneMvAvJijFfs7iySpIrlCJIkaZtCCHVDCM+lRnfmhRDO3Mox+4cQXtxshOW41PaTQwgvp977UAhhR8XlRaB56r1XpT5rfgjhl6lte4YQngwh/Cu1/fzU9hkhhLwQwhigdirHA6l9X6Uep4QQTtss86QQwrkhhOwQwrgQwuwQwtwQQt9y/G95GTgw9TlHp/6Mb4QQXgohtAwh1ARGAuenspyfyn53COG11LH/8/9RklQ15CQdQJJUpdQOIbyZev4+cB7QPcb4RQihIfBKCGHaFne9vwB4NsZ4QwghG6iTOnYo0CXGuCaEcA1wFWXFYVvOAOaFENpTdrfz7wEBeDWE8ALQDFgRYzwNIIRQf/M3xxivDSFcEWM8ciuf/SDQA3gyVWA6A5cBvYHCGON3Qwi1gFkhhL/GGN/fWsDUn68zMDG1aRFwXIyxOITQBRgdYzwnhDCczUaQQgijgf+LMV6Smp73Wgjh7zHGNdv5/yFJSoAFSZK0uXWbF4wQQg1gdAjheKCUspGTRsC/N3vPbODu1LGPxRjfDCF0Ag6nrHAA1KRs5GVrxoUQhgKfUlZYOgOPbiwPIYRHgOOAZ4DxIYSxlE3Lm7kTf66ngVtSJegU4MUY47rUtL4jQgjnpo6rD+RSVg43t7E4HggsBP622fGTQwi5QARqbOP7Twa6hRAGpV7vATRNfZYkqQqxIEmStudHwL5A+xjj1yGEDyj74X6TGOOLqQJ1GjAphHATsAr4W4yxVzm+4+oY48MbX4QQOm/toBjj4hBCO+CHwKgQwnMxxu2NSG3+3qIQwgygK3A+MGXj1wFXxhif3cFHrIsxHhlCqAM8C1wO3ArkA8/HGLunFrSYsY33B+CcGOPb5ckrSUqO1yBJkranPvBJqhydCBy85QEhhIOB/8QY/wjcBbQDXgGODSFsvKZozxBCi3J+50zgrBBCnRDCnkB3YGYI4QBgbYzxfmBc6nu29HVqJGtrHqRs6t7G0SgoKzuXbXxPCKFF6ju3Ksa4Fvg5MDCEkEPZ/5/lqd0XbXbol0C9zV4/C1wZUsNpIYSjtvUdkqRkWZAkSdvzAJAXQpgH/ISya262dALwrxDCG5SNztwSY/yUssLw5xDCXMqm17UqzxfGGF8HJgGvAa8Cd8UY3wDaUnbtzpvAr4FRW3n7H4C5Gxdp2MJfgU7A32OMG1Lb7gIWAK+HEOYDd7KD2RWpLHOBXsCNQEHqz775+54HDt+4SANlI001UtneSr2WJFVBLvMtSZIkSSmOIEmSJElSigVJkiRJklIsSJIkSZKUYkGSJEmSpBQLkiRJkiSlWJAkSZIkKcWCJEmSJEkpFiRJkiRJSvl/sy5kVCq/nGcAAAAASUVORK5CYII=",
            "text/plain": [
              "<Figure size 1008x576 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "def get_roc_curve(model, X, y):\n",
        "    pred_proba = model.predict_proba(X)[:, 1]\n",
        "    fpr, tpr, _ = roc_curve(y, pred_proba)\n",
        "    return fpr, tpr\n",
        "\n",
        "fpr, tpr = get_roc_curve(grid_svm.best_estimator_, X_test, y_test)\n",
        "plt.figure(figsize=(14,8))\n",
        "plt.plot(fpr, tpr, color=\"red\")\n",
        "plt.plot([0, 1], [0, 1], color='black', lw=2, linestyle='--')\n",
        "plt.xlim([0.0, 1.0])\n",
        "plt.ylim([0.0, 1.05])\n",
        "plt.xlabel('False Positive Rate')\n",
        "plt.ylabel('True Positive Rate')\n",
        "plt.title('Roc curve')\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a926c1YZ7-GL"
      },
      "source": [
        "### Prediction"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "86AUM56j7_e9",
        "outputId": "e1dcd58f-3ab2-4acc-aee4-ebd3ff0f1fcf"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "not negative\n"
          ]
        }
      ],
      "source": [
        "pred = grid_svm.predict([\"flying with @united is always a great experience.\"])\n",
        "print('negative' if pred == np.array([1]) else 'not negative')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1EmDkGE28DNe",
        "outputId": "8593665f-90d5-4377-9f2e-bb57487e286f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "negative\n"
          ]
        }
      ],
      "source": [
        "pred = grid_svm.predict([\"flying with @united is always a great experience. If you don't lose your luggage\"])\n",
        "print('negative' if pred == np.array([1]) else 'not negative')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7Ddf7SuP8QyE"
      },
      "source": [
        "**It easily distinguishes the text based on context!!**"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "Jb7w_dsF1aYX",
        "axAITZne3B-G"
      ],
      "name": "00_Pre_Neural_NLP.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}