BERT_Vocabulary.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Inspect BERT Vocabulary.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SaifAlmaliki/Bert-NLP/blob/main/BERT_Vocabulary.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "whjogPl1KL4-"
      },
      "source": [
        "## Load the Model\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "O0wWu4oEKYtc"
      },
      "source": [
        "Install the huggingface implementation."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PzCjfNB6jksJ"
      },
      "source": [
        "!pip install pytorch-pretrained-bert"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nwpkz_WYjscB",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "03faff17-92e2-4533-e6b9-8e234767b933"
      },
      "source": [
        "import torch\n",
        "from pytorch_pretrained_bert import BertTokenizer\n",
        "\n",
        "# Load pre-trained model tokenizer (vocabulary)\n",
        "tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "100%|██████████| 231508/231508 [00:00<00:00, 17240113.82B/s]\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B8FD0tDpKVBF"
      },
      "source": [
        "## Inspect BERT Vocabulary\n",
        "--------------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7UB5FRPOhinf"
      },
      "source": [
        "### Vocab Dump\n",
        "--------------\n",
        "\n",
        "Retrieve the entire list of \"tokens\" and write these out to text files so we can peruse them."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NRf7NXJGgsnG"
      },
      "source": [
        "with open(\"vocabulary.txt\", 'w') as f:\n",
        "    \n",
        "    # For each token...\n",
        "    for token in tokenizer.vocab.keys():\n",
        "        \n",
        "        # Write it out and escape any unicode characters.            \n",
        "        f.write(token + '\\n')\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c8geTQ3Iidsv"
      },
      "source": [
        "From perusing the vocab, I'm seeing that:\n",
        "\n",
        "* The first 999 tokens (1-indexed) appear to be reserved, and most are of the form [unused957].\n",
        "    * 1   - [PAD]\n",
        "    * 101 - [UNK]\n",
        "    * 102 - [CLS]\n",
        "    * 103 - [SEP]\n",
        "    * 104 - [MASK]\n",
        "* Rows 1000-1996 appear to be a dump of individual characters. \n",
        "    * They don't appear to be sorted by frequency (e.g., the letters of the alphabet are all in sequence).\n",
        "* The first word is \"the\" at position 1997.\n",
        "    * From there, the words appear to be sorted by frequency. \n",
        "    * The top ~18 words are whole words, and then number 2016 is ##s, presumably the most common subword.\n",
        "    * The last whole word is at 29612, \"necessitated\"\n",
        "\n",
        "Some funny inclusions:\n",
        "* starbucks\n",
        "* triassic\n",
        "* abolitionist\n",
        "* 1679"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uviYdpTZYMmd"
      },
      "source": [
        "### Single Characters\n",
        "---------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bu-KbpbR1L4q"
      },
      "source": [
        "The following code prints out all of the single character tokens in vocabulary, as well as all of the single-character tokens preceded by '##'.\n",
        "\n",
        "It turns out that these are matching sets--for every standalone character there is also a '##' version. There are 997 single character tokens."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q6ysF8LD1fbl"
      },
      "source": [
        "The following cell iterates over the vocabulary, pulling out all of the single character tokens."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YHpH2NEPy3t9"
      },
      "source": [
        "one_chars = []\n",
        "one_chars_hashes = []\n",
        "\n",
        "# For each token in the vocabulary...\n",
        "for token in tokenizer.vocab.keys():\n",
        "    \n",
        "    # Record any single-character tokens.\n",
        "    if len(token) == 1:\n",
        "        one_chars.append(token)\n",
        "    \n",
        "    # Record single-character tokens preceded by the two hashes.    \n",
        "    elif len(token) == 3 and token[0:2] == '##':\n",
        "        one_chars_hashes.append(token)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jCzfsjHrzZYn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1c48868f-0e49-4f80-f9f5-7cb23f434b5d"
      },
      "source": [
        "print('Number of single character tokens:', len(one_chars), '\\n')\n",
        "\n",
        "# Print all of the single characters, 40 per row.\n",
        "\n",
        "# For every batch of 40 tokens...\n",
        "for i in range(0, len(one_chars), 40):\n",
        "    \n",
        "    # Limit the end index so we don't go past the end of the list.\n",
        "    end = min(i + 40, len(one_chars) + 1)\n",
        "    \n",
        "    # Print out the tokens, separated by a space.\n",
        "    print(' '.join(one_chars[i:end]))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of single character tokens: 997 \n",
            "\n",
            "! \" # $ % & ' ( ) * + , - . / 0 1 2 3 4 5 6 7 8 9 : ; < = > ? @ [ \\ ] ^ _ ` a b\n",
            "c d e f g h i j k l m n o p q r s t u v w x y z { | } ~ ¡ ¢ £ ¤ ¥ ¦ § ¨ © ª « ¬\n",
            "® ° ± ² ³ ´ µ ¶ · ¹ º » ¼ ½ ¾ ¿ × ß æ ð ÷ ø þ đ ħ ı ł ŋ œ ƒ ɐ ɑ ɒ ɔ ɕ ə ɛ ɡ ɣ ɨ\n",
            "ɪ ɫ ɬ ɯ ɲ ɴ ɹ ɾ ʀ ʁ ʂ ʃ ʉ ʊ ʋ ʌ ʎ ʐ ʑ ʒ ʔ ʰ ʲ ʳ ʷ ʸ ʻ ʼ ʾ ʿ ˈ ː ˡ ˢ ˣ ˤ α β γ δ\n",
            "ε ζ η θ ι κ λ μ ν ξ ο π ρ ς σ τ υ φ χ ψ ω а б в г д е ж з и к л м н о п р с т у\n",
            "ф х ц ч ш щ ъ ы ь э ю я ђ є і ј љ њ ћ ӏ ա բ գ դ ե թ ի լ կ հ մ յ ն ո պ ս վ տ ր ւ\n",
            "ք ־ א ב ג ד ה ו ז ח ט י ך כ ל ם מ ן נ ס ע ף פ ץ צ ק ר ש ת ، ء ا ب ة ت ث ج ح خ د\n",
            "ذ ر ز س ش ص ض ط ظ ع غ ـ ف ق ك ل م ن ه و ى ي ٹ پ چ ک گ ں ھ ہ ی ے अ आ उ ए क ख ग च\n",
            "ज ट ड ण त थ द ध न प ब भ म य र ल व श ष स ह ा ि ी ो । ॥ ং অ আ ই উ এ ও ক খ গ চ ছ জ\n",
            "ট ড ণ ত থ দ ধ ন প ব ভ ম য র ল শ ষ স হ া ি ী ে க ச ட த ந ன ப ம ய ர ல ள வ ா ி ு ே\n",
            "ை ನ ರ ಾ ක ය ර ල ව ා ก ง ต ท น พ ม ย ร ล ว ส อ า เ ་ ། ག ང ད ན པ བ མ འ ར ལ ས မ ა\n",
            "ბ გ დ ე ვ თ ი კ ლ მ ნ ო რ ს ტ უ ᄀ ᄂ ᄃ ᄅ ᄆ ᄇ ᄉ ᄊ ᄋ ᄌ ᄎ ᄏ ᄐ ᄑ ᄒ ᅡ ᅢ ᅥ ᅦ ᅧ ᅩ ᅪ ᅭ ᅮ\n",
            "ᅯ ᅲ ᅳ ᅴ ᅵ ᆨ ᆫ ᆯ ᆷ ᆸ ᆼ ᴬ ᴮ ᴰ ᴵ ᴺ ᵀ ᵃ ᵇ ᵈ ᵉ ᵍ ᵏ ᵐ ᵒ ᵖ ᵗ ᵘ ᵢ ᵣ ᵤ ᵥ ᶜ ᶠ ‐ ‑ ‒ – — ―\n",
            "‖ ‘ ’ ‚ “ ” „ † ‡ • … ‰ ′ ″ › ‿ ⁄ ⁰ ⁱ ⁴ ⁵ ⁶ ⁷ ⁸ ⁹ ⁺ ⁻ ⁿ ₀ ₁ ₂ ₃ ₄ ₅ ₆ ₇ ₈ ₉ ₊ ₍\n",
            "₎ ₐ ₑ ₒ ₓ ₕ ₖ ₗ ₘ ₙ ₚ ₛ ₜ ₤ ₩ € ₱ ₹ ℓ № ℝ ™ ⅓ ⅔ ← ↑ → ↓ ↔ ↦ ⇄ ⇌ ⇒ ∂ ∅ ∆ ∇ ∈ − ∗\n",
            "∘ √ ∞ ∧ ∨ ∩ ∪ ≈ ≡ ≤ ≥ ⊂ ⊆ ⊕ ⊗ ⋅ ─ │ ■ ▪ ● ★ ☆ ☉ ♠ ♣ ♥ ♦ ♭ ♯ ⟨ ⟩ ⱼ ⺩ ⺼ ⽥ 、 。 〈 〉\n",
            "《 》 「 」 『 』 〜 あ い う え お か き く け こ さ し す せ そ た ち っ つ て と な に ぬ ね の は ひ ふ へ ほ ま み\n",
            "む め も や ゆ よ ら り る れ ろ を ん ァ ア ィ イ ウ ェ エ オ カ キ ク ケ コ サ シ ス セ タ チ ッ ツ テ ト ナ ニ ノ ハ\n",
            "ヒ フ ヘ ホ マ ミ ム メ モ ャ ュ ョ ラ リ ル レ ロ ワ ン ・ ー 一 三 上 下 不 世 中 主 久 之 也 事 二 五 井 京 人 亻 仁\n",
            "介 代 仮 伊 会 佐 侍 保 信 健 元 光 八 公 内 出 分 前 劉 力 加 勝 北 区 十 千 南 博 原 口 古 史 司 合 吉 同 名 和 囗 四\n",
            "国 國 土 地 坂 城 堂 場 士 夏 外 大 天 太 夫 奈 女 子 学 宀 宇 安 宗 定 宣 宮 家 宿 寺 將 小 尚 山 岡 島 崎 川 州 巿 帝\n",
            "平 年 幸 广 弘 張 彳 後 御 德 心 忄 志 忠 愛 成 我 戦 戸 手 扌 政 文 新 方 日 明 星 春 昭 智 曲 書 月 有 朝 木 本 李 村\n",
            "東 松 林 森 楊 樹 橋 歌 止 正 武 比 氏 民 水 氵 氷 永 江 沢 河 治 法 海 清 漢 瀬 火 版 犬 王 生 田 男 疒 発 白 的 皇 目\n",
            "相 省 真 石 示 社 神 福 禾 秀 秋 空 立 章 竹 糹 美 義 耳 良 艹 花 英 華 葉 藤 行 街 西 見 訁 語 谷 貝 貴 車 軍 辶 道 郎\n",
            "郡 部 都 里 野 金 鈴 镇 長 門 間 阝 阿 陳 陽 雄 青 面 風 食 香 馬 高 龍 龸 ﬁ ﬂ ！ （ ） ， － ． ／ ： ？ ～\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "62MD-M1N0R-W",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "86e8ac4c-b770-4353-b8d9-73698f827993"
      },
      "source": [
        "print('Number of single character tokens with hashes:', len(one_chars_hashes), '\\n')\n",
        "\n",
        "# Print all of the single characters, 40 per row.\n",
        "\n",
        "# Strip the hash marks, since they just clutter the display.\n",
        "tokens = [token.replace('##', '') for token in one_chars_hashes]\n",
        "\n",
        "# For every batch of 40 tokens...\n",
        "for i in range(0, len(tokens), 40):\n",
        "    \n",
        "    # Limit the end index so we don't go past the end of the list.\n",
        "    end = min(i + 40, len(tokens) + 1)\n",
        "    \n",
        "    # Print out the tokens, separated by a space.\n",
        "    print(' '.join(tokens[i:end]))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of single character tokens with hashes: 997 \n",
            "\n",
            "s a e i n o d r y t l m u h k c g p 2 z 1 b 3 f 4 6 7 x v 8 5 9 0 w j q ° ₂ а и\n",
            "² ₃ ı ₁ ⁺ ½ о ه ي α е د ن ν ø р ₄ ₀ ر я ³ ι ł н ᵢ ₙ ß ة ς م − т ː ل ь к ♭ η ی в\n",
            "ا × ¹ ы ה ɛ л ! \" # $ % & ' ( ) * + , - . / : ; < = > ? @ [ \\ ] ^ _ ` { | } ~ ¡\n",
            "¢ £ ¤ ¥ ¦ § ¨ © ª « ¬ ® ± ´ µ ¶ · º » ¼ ¾ ¿ æ ð ÷ þ đ ħ ŋ œ ƒ ɐ ɑ ɒ ɔ ɕ ə ɡ ɣ ɨ\n",
            "ɪ ɫ ɬ ɯ ɲ ɴ ɹ ɾ ʀ ʁ ʂ ʃ ʉ ʊ ʋ ʌ ʎ ʐ ʑ ʒ ʔ ʰ ʲ ʳ ʷ ʸ ʻ ʼ ʾ ʿ ˈ ˡ ˢ ˣ ˤ β γ δ ε ζ\n",
            "θ κ λ μ ξ ο π ρ σ τ υ φ χ ψ ω б г д ж з м п с у ф х ц ч ш щ ъ э ю ђ є і ј љ њ ћ\n",
            "ӏ ա բ գ դ ե թ ի լ կ հ մ յ ն ո պ ս վ տ ր ւ ք ־ א ב ג ד ו ז ח ט י ך כ ל ם מ ן נ ס\n",
            "ע ף פ ץ צ ק ר ש ת ، ء ب ت ث ج ح خ ذ ز س ش ص ض ط ظ ع غ ـ ف ق ك و ى ٹ پ چ ک گ ں ھ\n",
            "ہ ے अ आ उ ए क ख ग च ज ट ड ण त थ द ध न प ब भ म य र ल व श ष स ह ा ि ी ो । ॥ ং অ আ\n",
            "ই উ এ ও ক খ গ চ ছ জ ট ড ণ ত থ দ ধ ন প ব ভ ম য র ল শ ষ স হ া ি ী ে க ச ட த ந ன ப\n",
            "ம ய ர ல ள வ ா ி ு ே ை ನ ರ ಾ ක ය ර ල ව ා ก ง ต ท น พ ม ย ร ล ว ส อ า เ ་ ། ག ང ད\n",
            "ན པ བ མ འ ར ལ ས မ ა ბ გ დ ე ვ თ ი კ ლ მ ნ ო რ ს ტ უ ᄀ ᄂ ᄃ ᄅ ᄆ ᄇ ᄉ ᄊ ᄋ ᄌ ᄎ ᄏ ᄐ ᄑ\n",
            "ᄒ ᅡ ᅢ ᅥ ᅦ ᅧ ᅩ ᅪ ᅭ ᅮ ᅯ ᅲ ᅳ ᅴ ᅵ ᆨ ᆫ ᆯ ᆷ ᆸ ᆼ ᴬ ᴮ ᴰ ᴵ ᴺ ᵀ ᵃ ᵇ ᵈ ᵉ ᵍ ᵏ ᵐ ᵒ ᵖ ᵗ ᵘ ᵣ ᵤ\n",
            "ᵥ ᶜ ᶠ ‐ ‑ ‒ – — ― ‖ ‘ ’ ‚ “ ” „ † ‡ • … ‰ ′ ″ › ‿ ⁄ ⁰ ⁱ ⁴ ⁵ ⁶ ⁷ ⁸ ⁹ ⁻ ⁿ ₅ ₆ ₇ ₈\n",
            "₉ ₊ ₍ ₎ ₐ ₑ ₒ ₓ ₕ ₖ ₗ ₘ ₚ ₛ ₜ ₤ ₩ € ₱ ₹ ℓ № ℝ ™ ⅓ ⅔ ← ↑ → ↓ ↔ ↦ ⇄ ⇌ ⇒ ∂ ∅ ∆ ∇ ∈\n",
            "∗ ∘ √ ∞ ∧ ∨ ∩ ∪ ≈ ≡ ≤ ≥ ⊂ ⊆ ⊕ ⊗ ⋅ ─ │ ■ ▪ ● ★ ☆ ☉ ♠ ♣ ♥ ♦ ♯ ⟨ ⟩ ⱼ ⺩ ⺼ ⽥ 、 。 〈 〉\n",
            "《 》 「 」 『 』 〜 あ い う え お か き く け こ さ し す せ そ た ち っ つ て と な に ぬ ね の は ひ ふ へ ほ ま み\n",
            "む め も や ゆ よ ら り る れ ろ を ん ァ ア ィ イ ウ ェ エ オ カ キ ク ケ コ サ シ ス セ タ チ ッ ツ テ ト ナ ニ ノ ハ\n",
            "ヒ フ ヘ ホ マ ミ ム メ モ ャ ュ ョ ラ リ ル レ ロ ワ ン ・ ー 一 三 上 下 不 世 中 主 久 之 也 事 二 五 井 京 人 亻 仁\n",
            "介 代 仮 伊 会 佐 侍 保 信 健 元 光 八 公 内 出 分 前 劉 力 加 勝 北 区 十 千 南 博 原 口 古 史 司 合 吉 同 名 和 囗 四\n",
            "国 國 土 地 坂 城 堂 場 士 夏 外 大 天 太 夫 奈 女 子 学 宀 宇 安 宗 定 宣 宮 家 宿 寺 將 小 尚 山 岡 島 崎 川 州 巿 帝\n",
            "平 年 幸 广 弘 張 彳 後 御 德 心 忄 志 忠 愛 成 我 戦 戸 手 扌 政 文 新 方 日 明 星 春 昭 智 曲 書 月 有 朝 木 本 李 村\n",
            "東 松 林 森 楊 樹 橋 歌 止 正 武 比 氏 民 水 氵 氷 永 江 沢 河 治 法 海 清 漢 瀬 火 版 犬 王 生 田 男 疒 発 白 的 皇 目\n",
            "相 省 真 石 示 社 神 福 禾 秀 秋 空 立 章 竹 糹 美 義 耳 良 艹 花 英 華 葉 藤 行 街 西 見 訁 語 谷 貝 貴 車 軍 辶 道 郎\n",
            "郡 部 都 里 野 金 鈴 镇 長 門 間 阝 阿 陳 陽 雄 青 面 風 食 香 馬 高 龍 龸 ﬁ ﬂ ！ （ ） ， － ． ／ ： ？ ～\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "u-TFrVP9005S",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7ecf2296-0ff5-4c6f-e49b-e694707bf95e"
      },
      "source": [
        "print('Are the two sets identical?', set(one_chars) == set(tokens))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Are the two sets identical? True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uPnW2RetYc97"
      },
      "source": [
        "### Subwords vs. Whole-words\n",
        "Let's gather some statistics on the vocabulary.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LjvrWcNTdkAW",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 446
        },
        "outputId": "bf9eab63-309e-489f-c8bd-8a8170768ad3"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "import numpy as np\n",
        "\n",
        "sns.set(style='darkgrid')\n",
        "\n",
        "# Increase the plot size and font size.\n",
        "sns.set(font_scale=1.5)\n",
        "plt.rcParams[\"figure.figsize\"] = (10,5)\n",
        "\n",
        "# Measure the length of every token in the vocab.\n",
        "token_lengths = [len(token) for token in tokenizer.vocab.keys()]\n",
        "\n",
        "# Plot the number of tokens of each length.\n",
        "sns.countplot(token_lengths)\n",
        "plt.title('Vocab Token Lengths')\n",
        "plt.xlabel('Token Length')\n",
        "plt.ylabel('# of Tokens')\n",
        "\n",
        "print('Maximum token length:', max(token_lengths))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
            "  FutureWarning\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Maximum token length: 18\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoUAAAFjCAYAAABL3HHWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVzU1f4/8NcAA4iggA3uC3kbQDYFpUCuppiiuaCCiiu5G2lqmXjTX7e0L6XAdcFcML2GuAuiuafmzQV3JZNrqWiaIaPIJjIzwOf3Bw8+1xGwGeXDqPN6Ph49Hs4553PeZ8zw1ed8FpkgCAKIiIiIyKSZGXsBRERERGR8DIVERERExFBIRERERAyFRERERASGQiIiIiICQyERERERgaGQiF4xycnJcHFxwcmTJ429FJGLiwuioqKMvYxX1pIlS+Di4oLbt28beylELzULYy+AiF5sU6ZMwb59+7B9+3a4ublVOUYQBAQFBSE/Px9Hjx6FtbV1La+y5owYMQKnTp3Sa2x0dDQGDBgg8Yqk07VrV9jY2OD777839lL+0smTJ3Hq1CmMGjUK9erVM/ZyiF5JDIVE9FShoaHYt28ftm3bhtmzZ1c5Ji0tDX/88QcGDx78UgdCAJg4cSJCQ0PFzw8ePEB0dDTat2+PQYMG6Yz18fGp7eWZrFOnTiE+Ph79+/dnKCSSCEMhET1VYGAgGjdujJ07d+KTTz6BpaVlpTHJyckAoBOmXlYdO3bU+Xz79m1ER0ejefPm6Nevn5FWRUQkPV5TSERPZWZmhv79+yM3NxeHDh2q1F9YWIj9+/dDqVTCy8sLAJCTk4PPP/8cnTt3hoeHBzp37ozPP/8cDx48qHS8RqNBQkIC+vXrB29vb/j6+mLAgAFYt26dOObu3bv46quv0K9fP3To0AGenp7o1asXVq5cidLS0irXXVpaiiVLlqBLly7w8PBAnz59sGvXrhr6XSm3ZcsW9O/fH15eXvD19cXo0aNx5swZvY795Zdf0LFjR/Tq1Qt37twBUL4Nv379egwYMADe3t5o164dRowYgbS0NJ1jb9++DRcXFyxZsgSHDx/GwIED4enpicDAQHz99dcoKSmp0e+p0WiwfPlyvPvuu/D09ET79u0xceJEXL58WWfcyZMn4eLiguTkZGzbtg3vvvsuPDw80KVLFyQkJFQ59/r169GjRw94eHige/fuWLduXaXrQqOiohAfHw8ACAoKgouLi/j9n1xnXFwcOnXqBA8PD/Tt2xdHjhypVHP79u0IDQ1F+/bt0bZtWwQFBeGjjz5CTk5OTfx2Eb20eKaQiP7SgAEDsGzZMiQnJyM4OFinb9euXSguLsbAgQMBAAUFBQgPD8fNmzcxcOBAtGnTBhkZGdiwYQPS0tKwZcsW2NraAij/S3zMmDE4deoUAgMD0bdvX1hZWeHXX3/F/v37MXz4cADAlStXsH//frzzzjto0aIFtFotfvrpJ8TGxuL27dv44osvKq05JiYGRUVFCA8PB1B+NnP69OlQq9U1ch3gggULsGrVKnh5eWH69OkoLCzE5s2bMWrUKHzzzTfo3Llztcf+9NNPmDJlClxcXLB8+XLY29sDAGbMmIFdu3ahR48eGDBgADQaDXbu3InRo0djyZIlCAoK0pnnyJEjWL9+PYYMGYKBAwfi4MGDWL16NerXr4+JEyc+93cEAK1WizFjxuD8+fPo168fhg0bJn7X8PBwrFu3Dp6enjrHbNy4Effu3UNoaCjq1auHHTt2ICYmBo0aNUKfPn3EcStXrkRsbCzc3d3x0Ucf4dGjR/j222/h4OCgM9/gwYNRWFiIAwcOYNasWWK/i4uLzrioqChYWFhg9OjR0Gq1WLt2LSIjI7F37140a9YMQHkgnDlzJtq3b48pU6bA2toaf/75J44cOYL79+/D0dGxRn7fiF5KAhGRHkaOHCm4ubkJd+/e1WkfNGiQ4O7uLty/f18QBEGIi4sTlEqlsG7dOp1x69atE5RKpfCvf/1LbFu5cqWgVCqF2NjYSvVKS0vFXz969EgoKyurNObjjz8WXF1ddda0bds2QalUCm+//baQn58vtufn5wtvv/220KFDB+HRo0d6f+9bt24JSqVSmDlzpth27do1wcXFRRgyZIigVqvF9qysLMHX11fo0qWLUFJSIrY/fnxKSorg7u4uTJo0SWcd+/fvF5RKpbBx40ad+lqtVujfv7/QpUsX8fegYk3e3t7CrVu3xLFlZWXCu+++K3Ts2FGv79alSxfh3XfffeqYNWvWCEqlUvjPf/6j015QUCB07txZGD58uNiWlpYmKJVKoWPHjjq/90VFRcKbb74pDBo0SGx78OCB4OnpKfTu3VsoLi4W27OzswUfHx9BqVQKaWlpYvvixYsFpVKp832f7Bs/frzOn5OLFy8KSqVSiImJEdsiIyOFdu3aCVqt9qnfm8gUcfuYiPQSGhqK0tJSbN++XWy7du0aLly4gK5du4pnWA4cOABHR0cMHjxY5/jBgwfD0dERP/zwg9i2c+dO1K9fH5GRkZXqmZn978eTtbU1ZDIZgPKzi7m5ucjJyUFgYCDKyspw6dKlSseHh4fDzs5O/GxnZ4chQ4YgLy/vuR9Xc/DgQQiCgLFjx+pcY9mwYUMMGDAAf/zxR6WtVaD8zFhUVBQGDBiAJUuW6NyUs2PHDtStWxfdunVDTk6O+E9+fj66du2KP/74Azdu3NCZLygoSDwDBgAymQxvvvkmVCoVHj58+Fzf8fF1vf7663B3d9dZl0ajQUBAAM6ePYvi4mKdYwYOHKjze1+nTh20bdtWZ/3Hjx+HWq1GeHg4rKysxHaFQqFzNtEQI0eOFP+cAICXlxdsbGxw8+ZNsc3Ozg7FxcX48ccfIQjCM9UhelVx+5iI9NK9e3fUq1cPycnJGD9+PABg27ZtACBuHQPl17t5eHjAwkL3x4uFhQVatWqlE5Zu3rwJNzc3nVBQlZKSEqxcuRKpqam4efNmpb/M8/PzKx3z+uuvV2pr3bq1uMbnUXH8G2+8Uamvou3WrVs626r79+/Hw4cPMWjQoCq3u69du4aHDx8iICCg2rr379+Hs7Oz+Ll58+aVxlRsRefm5qJu3bp6fqPqXbt2DcXFxfD39692zIMHD9C4cWPx8+NB9fF15ebmip8rfg8f/z4VqmrTR1W/Hw4ODjrXsk6YMAGnT59GZGQk7O3t4efnh06dOqFnz57iZQ1EpoqhkIj0YmVlhd69e2P9+vU4d+4cvL29sWPHDjRq1Ah///vfJa391VdfITExEb169cLEiRPh6OgIuVyOX375BTExMSgrK5O0fk3w8vLCH3/8gX379mHQoEGVrsMTBAGOjo6IjY2tdo4nQ6i5uXm1Y2vqLJggCFAqlZg1a1a1Y568Du9p65LS42eXq9OqVSvs3r0bJ06cwIkTJ3Dq1CnMnj0bixcvRlJSElq0aFELKyV6MTEUEpHeQkNDsX79eiQnJyMvLw8qlQoTJ07U+cu4efPmyMzMRElJic7ZwpKSEty4cUPnbE6rVq1w/fp1aDSaKh91UyE1NRUdOnTAv/71L532x7cFn3T9+vVKbdeuXQNQ9ZksQ1R8h99++61SiLh69arOmAqNGjXC119/jZEjR+K9997DqlWr0LZtW7G/ZcuWuHHjBry9vWvkDF9NadmyJR48eIC33npLr9Clr6ZNmwIAMjMzK52FzMzMrDT+8W3h52VpaYnOnTuLNwMdOXIE48ePx5o1a/DZZ5/VWB2ilw2vKSQivbm7u8PNzQ27d+9GUlISZDJZpWcTVlwTt2XLFp32zZs3IycnB926dRPb+vTpg7y8PHzzzTeVaj1+psvMzKzSma+ioiL8+9//rnatGzZsQEFBgfi5oKAAGzduRL169eDn56fX961O165dIZPJ8O2330Kr1Yrt2dnZSE5ORtOmTdGmTZtKxzVs2BDr1q2Dk5MTRo8ejbNnz4p9ISEhKCsrQ1xcXJU1792791xrflYhISFQqVRYs2ZNlf3Puq6AgABYWlpiw4YNUKvVYrtKpcLOnTsrjbexsQEA5OXlPVO9ClU9dqbi39Xzzk30suOZQiIySGhoKObOnYuffvoJfn5+lc6IjR07Fnv37sUXX3yBy5cvw83NDRkZGdi6dSucnZ0xduxYcezIkSNx+PBhLFu2DD///DMCAwNhaWmJq1evIjMzUwx9PXr0wKZNmzB16lQEBATg3r172LZtm3j9XFUcHBwQFhYmPn4mOTkZd+7cwbx581CnTp3n+j14/fXXMWbMGKxatQrDhw9Hz5498fDhQ2zevBlFRUWIiYmpdgtVoVAgMTERERERGDt2LFasWAE/Pz8EBweLz2f85Zdf0KVLFzg4OCArKwsXLlzAzZs3cfDgwedad1VycnKqDOVA+bWiI0eOxPHjxzF//nykpaXhrbfegq2tLe7cuYO0tDRYWloiMTHR4LoODg744IMPEBcXh/DwcPTt2xePHj3C5s2b0apVK1y6dEnn7KC3tzeA8kcN9enTB1ZWVnjjjTegVCoNqjtmzBjY2dmhffv2aNy4MfLz85GSkgKZTMaHk5PJYygkIoP06dMH8+fPh1qt1rnBpIKdnR02bNiAxYsX49ChQ0hOTkaDBg0wZMgQTJ48WedifktLS6xevRqrV6/G999/j7i4OFhZWaFly5Y6zxKcNWsW6tati7179+LgwYNo3LgxBg8eDE9PT0RERFS5zo8//hhnzpzB+vXrce/ePTg7O4uBoibMmDEDLVu2xPr16xEbGwu5XA5vb2/Exsaiffv2Tz22QYMG+O677/Dee+9h/PjxWLZsGfz9/REdHY0333wTmzdvxooVK6DVaqFQKNCmTRt89NFHNbLuJ92/fx+LFi2qsi8gIAANGzbEihUrsH79eqSmpooPjHZycoKnpyf69+//zLUnTJgAW1tbfPfdd4iJiUGTJk0wZswYCIKAS5cu6dyd7evri48//hgbN27EnDlzUFJSgg8++MDgUBgeHo49e/Zg06ZNyMvLg729Pdzc3DB79my89dZbz/xdiF4FMoH35BMR0Qtk7ty5WLduHY4ePQqFQmHs5RCZDF5TSERERvH4tYQVsrOzsX37diiVSgZColrG7WMiIjKKkydPYsGCBXjnnXfQqFEj/PHHH+J1mVJtlxNR9RgKiYjIKFq2bInmzZtj8+bNyM3NhZWVFTw8PDBhwoSnPsSbiKTBawqJiIiIiNcUEhERERFDIRERERGB1xTWmAcPHqKsjDvxRERE9OIyM5PBwaHqV2kyFNaQsjKBoZCIiIheWtw+JiIiIiKGQiIiIiJiKCQiIiIiMBQSERERERgKiYiIiAgMhUREREQEhkIiIiIiAkMhEREREYGhkIiIiIhgxDeanDx5EiNHjqyyb/fu3WjdurX4+dy5c1iwYAEuX74MW1tb9OzZEx999BHq1Kmjc5xGo8GiRYuQmpqK/Px8uLq6Ytq0afD3969UQ985iV5U9e3lsJRbS1pDoy1GXq5W0hpERPRiMPpr7kaNGgV3d3edtoYNG4q/zsjIQEREBP72t78hKioKWVlZWL16NW7fvo3ly5frHBcVFYX9+/dj5MiRaNmyJVJSUjBu3DgkJiaiXbt2zzQn0YvKUm6N2A09JK3xUfg+AAyFRESmwOih0M/PD926dau2Py4uDvb29khMTETduuUvcG7WrBlmz56NEydOiGcB09PTsWvXLsyaNQsREREAgJCQEPTu3RsxMTFISkoyeE4iIiIiU/FCXFNYWFiIkpKSKtuPHz+OkJAQMbwBQL9+/WBjY4M9e/aIbXv37oVcLkdYWJjYZmVlhdDQUJw9exbZ2dkGz0lERERkKoweCmfMmAFfX194e3tj9OjRuHLlith35coVlJSUwMPDQ+cYS0tLuLm5ISMjQ2zLyMiAs7OzTtADAC8vLwiCII41ZE4iIiIiU2G07WO5XI4ePXqgU6dOcHBwwJUrV7B69WoMHToUW7duhbOzM1QqFQBAoVBUOl6hUODChQviZ5VKpXMt4uPjAIhnCg2Z0xANGtg+03FELzqFws7YSyAiolpgtFDo4+MDHx8f8XNQUBC6du2KgQMHIj4+HrGxsSguLgZQfhbvSVZWVmI/ABQXF0Mul1c5DgDUarU4Tt85DXH/fiHKyoRnOpboWdRWWFOpCmqlDhERSc/MTFbtiSyjbx8/ztXVFf7+/khLSwMAWFuXP25Do9FUGqtWq8X+irFabeW7JCvCYEU4NGROIiIiIlPxQoVCAGjcuDHy8vIA/G+Lt2LL93EqlQpOTk7iZ4VCIW4RPzkOgDjWkDmJiIiITMULFwpv3boFBwcHAIBSqYSFhQUuXbqkM0aj0SAjIwNubm5im6urKzIzM/Hw4UOdsRcvXhT7DZ2TiIiIyFQYLRTm5ORUajtz5gxOnjyJwMBAAICdnR38/f2RmpqqE/ZSU1NRVFSE4OBgsS04OBharRZbtmwR2zQaDZKTk+Hj4yPehGLInERERESmwmg3mkydOhV16tRBu3bt4ODggN9++w2bNm2Cg4MDJk+eLI6bNm0ahgwZghEjRiAsLAxZWVlYs2YNOnXqhICAAHGct7c3goODERMTA5VKhRYtWiAlJQV37txBdHS0Tm195yQiIiIyFTJBEIxyy+x3332HnTt34vfff0dhYSEcHR0RGBiIyZMno0mTJjpjz5w5g5iYGPE9xb169cL06dNhY2OjM06tVmPhwoXYuXMn8vLy4OLigunTp1cZ9PSdU1+8+5hqm0JhVyuvuePdx0REr46n3X1stFD4qmEopNrGUEhERIZ6aR5JQ0RERETGwVBIRERERAyFRERERMRQSERERERgKCQiIiIiMBQSERERERgKiYiIiAgMhUREREQEhkIiIiIiAkMhEREREYGhkIiIiIjAUEhEREREYCgkIiIiIjAUEhEREREYComIiIgIDIVEREREBIZCIiIiIgJDIRERERGBoZCIiIiIAFgYewFELzN7eznkcmtJa2i1xcjN1Upag4iIiKGQ6DnI5dZI/HcPSWuMiNgHgKGQiIikxe1jIiIiImIoJCIiIiKGQiIiIiICQyERERERgaGQiIiIiMBQSERERERgKCQiIiIiMBQSERERERgKiYiIiAh8owkRPYN69pawkltJWkOtVSM/VyNpDSIi+h+GQiIymJXcCu+lBEtaY03/vQAYComIagu3j4mIiIiIoZCIiIiIGAqJiIiICAyFRERERASGQiIiIiLCCxYKExIS4OLign79+lXqO3fuHMLDw+Ht7Y2OHTti3rx5ePToUaVxGo0GCxYsQGBgILy8vDBo0CCcOHGiynr6zklERET0qnthQqFKpcKyZctgY2NTqS8jIwMRERFQq9WIiopCaGgoNm3ahGnTplUaGxUVhbVr16Jv37749NNPYWZmhnHjxuH8+fPPPCcRERHRq+6FeU5hbGwsPDw8IAgC8vPzdfri4uJgb2+PxMRE1K1bFwDQrFkzzJ49GydOnIC/vz8AID09Hbt27cKsWbMQEREBAAgJCUHv3r0RExODpKQkg+ckIiIiMgUvxJnC9PR07NixA7NmzarUV1hYiOPHjyMkJEQMbwDQr18/2NjYYM+ePWLb3r17IZfLERYWJrZZWVkhNDQUZ8+eRXZ2tsFzEhEREZkCo4dCQRAwd+5chISEwM3NrVL/lStXUFJSAg8PD512S0tLuLm5ISMjQ2zLyMiAs7OzTtADAC8vLwiCII41ZE4iIiIiU2D07ePt27fj6tWrWLp0aZX9KpUKAKBQKCr1KRQKXLhwQWdsw4YNqxwHQDxTaMic+mrQwNbgY4j0pVDYsTYREUnKqKGwsLAQsbGxGD9+PJycnKocU1xcDKD8LN6TrKysxP6KsXK5vMpxAKBWqw2eU1/37xeirEww+Dh6udVWaFGpClibiIiem5mZrNoTWUbdPl62bBnkcjnee++9asdYW1sDKH/UzJPUarXYXzFWq9VWOQ74Xzg0ZE4iIiIiU2C0M4XZ2dlYu3YtPvzwQ9y7d09sV6vV0Gq1uH37Nuzs7MQt3oot38epVCqdM4wKhULcIn5yHABxrCFzEhEREZkCo50pvH//PrRaLWJiYhAUFCT+c/HiRVy7dg1BQUFISEiAUqmEhYUFLl26pHO8RqNBRkaGzs0prq6uyMzMxMOHD3XGXrx4UewHYNCcRERERKbAaGcKmzVrVuXNJQsXLkRRURH+8Y9/oFWrVrCzs4O/vz9SU1MxYcIE8c7i1NRUFBUVITg4WDw2ODgYq1evxpYtW8TnFGo0GiQnJ8PHx0e8CcWQOYmIiIhMgdFCoZ2dHbp161apfe3atTA3N9fpmzZtGoYMGYIRI0YgLCwMWVlZWLNmDTp16oSAgABxnLe3N4KDgxETEwOVSoUWLVogJSUFd+7cQXR0tE4dfeckIiIiMgVGf06hPtzd3bFmzRpYWloiOjoaW7ZswaBBg7Bo0aJKY+fPn48RI0YgNTUV8+bNQ0lJCVauXAlfX99nnpOIiIjoVWf05xQ+KTExscr29u3bY+PGjX95vJWVFWbOnImZM2f+5Vh95yQiIiJ61b0UZwqJiIiISFoMhURERETEUEhEREREDIVEREREBIZCIiIiIgJDIRERERGBoZCIiIiIwFBIRERERDDg4dUPHjxATk4OWrduLbbdunUL//73v5Gbm4uQkBD8/e9/l2SRRERERCQtvUPhl19+iRs3bmDr1q0AgIcPH2LYsGHIzs4GAOzZswdr165Fhw4dpFkpEREREUlG7+3jCxcuoHPnzuLn3bt3Izs7GytXrsRPP/2E1q1bY9WqVZIskoiIiIikpXcovH//Pho1aiR+/umnn+Dh4YFOnTpBoVCgf//+uHz5siSLJCIiIiJp6R0KLSwsoFarxc+nTp3S2Sq2s7NDbm5uza6OiIiIiGqF3qGwVatW2LdvHwRBwMGDB5GXlwd/f3+xPysrC/Xr15dkkUREREQkLb1vNBk2bBiioqLQoUMHFBcXo3nz5jqh8MyZM3BxcZFkkUREREQkLb1DYUhICADg4MGDsLW1xcSJEyGXywGUP66moKAA4eHh0qySiIiIiCSldygEyoNhRTh8nIODA5KTk2tsUURERERUu/hGEyIiIiIy7EzhnTt3sGnTJty4cQO5ubkQBEGnXyaTYe3atTW6QCIiIiKSnt6h8MiRI/jggw+g1WphY2MDe3t7KddFRERERLVI71AYFxcHBwcHLF26FJ6enlKuiYiIiIhqmd7XFF6/fh2jRo1iICQiIiJ6BekdCh0dHcVH0BARERHRq0XvUNivXz/s379fyrUQERERkZHofU1h//79cfLkSUyaNAkjR45Es2bNYG5uXmlckyZNanSBRERERCQ9vUNhz549IZPJIAgCfvzxx2rHZWRk1MS6iIiIiKgW6R0KIyMjIZPJpFwLERERERmJ3qFw8uTJUq6DiIiIiIzIoDeaEBEZm529FazllpLWKNZqUJCrlrQGEdGLxqBQWFhYiH//+984duwY7t+/j6+//hrt2rVDTk4O1q9fj549e6J169ZSrZWICNZyS/TaPkfSGrtD5qIADIVEZFr0DoU5OTkIDw/H7du30aJFC9y6dQvFxcUAyp9huH37dhQUFGDWrFmSLZaIiIiIpKF3KFy4cCHu3buHzZs3o3HjxggICNDpDwoKwokTJ2p8gUREREQkPb0fXn348GEMHToU7u7uVd6F3Lx5c2RlZdXo4oiIiIiodugdCh88eIAWLVpU2y+TyaBW8xocIiIiopeR3qFQoVDg1q1b1fZnZGSgcePGNbIoIiIiIqpdeofCTp06YevWrcjOzq7Ud/HiRWzfvh1BQUE1ujgiIiIiqh16h8IPPvgA5ubm6N+/P+Li4iCTybB9+3ZMnz4dw4YNg5OTE8aNG6d34Z9//hmRkZHo0qULvLy80LFjR4wZMwbnzp2rNPbcuXMIDw+Ht7c3OnbsiHnz5uHRo0eVxmk0GixYsACBgYHw8vLCoEGDqr35Rd85iYiIiEyBQdvHmzdvhpeXF7Zt2wZBEJCamoo9e/YgMDAQ69evh729vd6Fb926hdLSUoSFhWHOnDkYM2YMcnJyMHz4cBw7dkwcl5GRgYiICKjVakRFRSE0NBSbNm3CtGnTKs0ZFRWFtWvXom/fvvj0009hZmaGcePG4fz58zrjDJmTiIiIyBQY9PDqxo0bY9myZSgsLMT169cBAC1atBDDYGFhIWxtbfWaq1evXujVq5dOW3h4OLp164bvvvsOHTt2BADExcXB3t4eiYmJqFu3LgCgWbNmmD17Nk6cOAF/f38AQHp6Onbt2oVZs2YhIiICABASEoLevXsjJiYGSUlJYh195yQiIiIyFXqfKZw3b574a1tbW3h5ecHLy0snEI4dO/a5FlOnTh04OjoiPz9fnPP48eMICQkRwxsA9OvXDzY2NtizZ4/YtnfvXsjlcoSFhYltVlZWCA0NxdmzZ8VrIQ2Zk4iIiMhU6B0K161bh1WrVlXZV1RUhHHjxiEjI8PgBRQWFiInJwfXr19HXFwcfv31V/FM3ZUrV1BSUgIPDw+dYywtLeHm5qZTLyMjA87OzjpBDwC8vLwgCII41pA5iYiIiEyF3tvHH374IWJjY+Hk5IS+ffuK7cXFxZgwYQIuXbqE+Ph4gxfwj3/8A/v27QMAyOVyDBkyBBMnTgQAqFQqAOXXMz5JoVDgwoUL4meVSoWGDRtWOQ6AeKbQkDmJiIiITIXeoXDSpEnIysrCp59+itdeew0BAQFQq9WYOHEizp8/j8WLF6Nz584GLyAyMhKDBw9GVlYWUlNTodFooNVqYWlpKb5b2dLSstJxVlZWYj9QHk7lcnmV4wCID9Y2ZE5DNGig37WUVPNKSzQwt6j87/Nlq/E0CoUda5tQbSIiYzDoRpPPPvsM9+7dw+TJk5GQkIClS5fizJkziI2NRdeuXZ9pAS4uLnBxcQEA9O3bFwMHDsSsWbOwePFiWFtbAyh/1MyT1Gq12A8A1tbW0Gq1VY4D/hcODZnTEPfvF6KsTHimY+n5KBR22Pdtr78e+Bx6jNkNlaqgytq1gbVfjNpERC87MzNZtSey9L6msHwiM4YYdycAACAASURBVMTFxUGpVGL48OE4efIk5s+fjx49etTIQuVyOYKCgrB//34UFxeLW7wVW76PU6lUcHJyEj8rFIoqH6xdcWzFWEPmJCIiIjIV1YbC06dPV/lPeno6Ro8ejTp16mDAgAFQKBQ6/c+ruLgYgiDg4cOHUCqVsLCwwKVLl3TGaDQaZGRkwM3NTWxzdXVFZmYmHj58qDP24sWLYj8Ag+YkIiIiMhXVbh+PGDECMpms2gMFQcDmzZuxZcsW8bNMJtP77t2cnBw4OjrqtBUWFmLfvn1o3LgxGjRoAADw9/dHamoqJkyYIN5ZnJqaiqKiIgQHB4vHBgcHY/Xq1diyZYv4nEKNRoPk5GT4+PiIN6HY2dnpPScRERGRqag2FEZHR0taeOrUqbCyskK7du2gUCjw559/Ijk5GVlZWYiLixPHTZs2DUOGDMGIESMQFhaGrKwsrFmzBp06dUJAQIA4ztvbG8HBwYiJiYFKpUKLFi2QkpKCO3fuVPou+s5JREREZCqqDYX9+/eXtHDfvn2RmpqKxMRE5Ofnw87ODm3btsX8+fPh5+cnjnN3d8eaNWsQExOD6Oho2NraYtCgQZg+fXqlOefPn4+FCxciNTUVeXl5cHFxwcqVK+Hr66szzpA5iYiIiEyBQXcf16TQ0FCEhobqNbZ9+/bYuHHjX46zsrLCzJkzMXPmzBqbk4iIiMgUGBQKi4qKsGrVKhw4cAC3b98GUP7O4O7du2PMmDGwsbGRZJFEREREJC29Q2Fubi6GDRuGa9euwdHRUbxL98aNG1i6dCn27t2LpKQk8V3IRERERPTy0DsULl68GNevX8ecOXMwZMgQmJubAwBKS0uxadMmzJs3D/Hx8Zg9e7ZkiyUiIiIiaej98OpDhw4hLCwMw4YNEwMhAJibm2Po0KEYOHAgfvjhB0kWSURERETS0jsU3rt376kPdm7Tpg3u3btXI4siIiIiotqldyh87bXXnvpg6oyMDLz22ms1sigiIiIiql16h8IuXbpg69at2LhxI8rKysT2srIybNq0Cdu2bUPXrl0lWSQRERERSeupN5q4ublh/vz56NOnD6ZMmYLjx4/j888/x5IlS+Ds7AwAyMzMRE5ODlq0aIHJkyfXyqKJiIiIqGY99UyhIAjirx0cHLBt2zaMHz8e9vb2+Pnnn/Hzzz/DwcEB48ePx7Zt2+Dg4CD5gomIiIio5hn08GpbW1tMmzYN06ZNk2o9RERERGQEel9TSERERESvrr88U3j9+nWcPn1a7wk7dOjwXAsiIiIiotr3l6Fw+fLlWL58ud4TPu2xNURERET0YvrLUNitWze4uLjUxlqIiIiIyEj+MhR2794dffr0qY21EBEREZGR8EYTIiIiImIoJCIiIiKGQiIiIiLCX1xTePDgQTg6OtbWWoiIiIjISJ4aCps2bVpb6yAiIiIiI+L2MRERERExFBIRERGRHs8pJCKicnb21rCWyyWtUazVoiC3WNIaRERVqTYUxsfHo3v37lAqlQCAO3fuwNHREdbW1rW2OCKiF4m1XI53k7+RtMauAe+jAAyFRFT7nhoKW7ZsKYbCoKAgzJ8/n283oSo51LeEhaWVpDVKNGo8yNNIWoOIiMhUVRsK69Wrh/z8fPGzIAi1siB6OVlYWiF9WV9Ja3hN2gGAoZCIiEgK1YZCNzc3fPvttygpKUH9+vUBAGfOnEFpaelTJwwJCanZFRIRERGR5KoNhbNmzcIHH3yA6OhoAIBMJsOmTZuwadOmaieTyWQMhUREREQvoWpDoaurK/bt24dbt25BpVJhxIgRmDhxIgICAmpzfURERERUC576SBpzc3O0atUKrVq1QocOHfDmm2/Cz8+vttZGRERERLVE7+cUJiYmSrkOIiIiIjIigx5eXVZWhpSUFBw4cAC3b98GADRr1gzdu3dHSEgIzMz4ghQiIiKil5HeobC4uBjjxo3DmTNnIJPJoFAoAAD/+c9/cOTIEWzfvh0JCQmwspL2WXVEREREVPP0PrW3bNkynD59Gu+99x5OnDiBI0eO4MiRI0hLS8Po0aNx6tQpLFu2TMq1EhEREZFE9A6Fu3fvRs+ePfHJJ5+Izy0Eyh9yPWPGDPTs2RO7du2SZJFEREREJC29Q2FWVtZT7zzu0KEDsrKyamRRRERERFS79A6F9erVw++//15t/++//4569erVyKKIiIiIqHbpHQoDAgKQlJSEn376qVLf0aNHsWHDBgQGBupdOD09HZ9//jl69eqFtm3b4u2338a0adNw8+bNSmPPnTuH8PBweHt7o2PHjpg3bx4ePXpUaZxGo8GCBQsQGBgILy8vDBo0CCdOnKiyvr5zEhEREZkCve8+njp1Ko4ePYrx48fDzc0Nb7zxBgDgt99+Q0ZGBhwcHDBlyhS9C69atQrnzp1DcHAwXFxcoFKpkJSUhJCQEGzduhWtW7cGAGRkZCAiIgJ/+9vfEBUVhaysLKxevRq3b9/G8uXLdeaMiorC/v37MXLkSLRs2RIpKSkYN24cEhMT0a5dO3GcIXMSERERmQK9Q2HTpk2xbds2xMbG4vDhw7h8+TIAoG7dunj33Xcxffp0NGnSRO/CERERiImJgaWlpdjWq1cv9OnTBwkJCfjqq68AAHFxcbC3t0diYiLq1q0LoPzZiLNnz8aJEyfg7+8PoPzM465duzBr1ixEREQAAEJCQtC7d2/ExMQgKSlJrKPvnERERESmwqCnTTdp0gSxsbE4e/Ysjh07hmPHjuHMmTOIiYkxKBACgI+Pj04gBIBWrVrhjTfewLVr1wAAhYWFOH78OEJCQsTwBgD9+vWDjY0N9uzZI7bt3bsXcrkcYWFhYpuVlRVCQ0Nx9uxZZGdnGzwnERERkal4pleQyGQyNGjQAA0aNIBMJquxxQiCgHv37sHBwQEAcOXKFZSUlMDDw0NnnKWlJdzc3JCRkSG2ZWRkwNnZWSfoAYCXlxcEQRDHGjInERERkal4od5Lt2PHDty9exc9e/YEAKhUKgAQ357yOIVCIZ79qxjr5ORU5TgA4lhD5iQiIiIyFQa9+1hK165dwxdffAFfX1/069cPQPmr9QBU2mYGyreGK/orxsrl8irHAYBarTZ4TkM0aGD7TMeRYRQKO9ZmbdYmIpLACxEKVSoVJkyYgPr162PRokUwMys/gWltbQ2g/FEzT1Kr1WJ/xVitVlvlOOB/4dCQOQ1x/34hysqEZzr2VVBbf4mpVAWszdomWZuIqCaYmcmqPZFl9FBYUFCAcePGoaCgABs2bNDZ1q34dcWW7+Oe3C6ubuu34tiKsYbMSURERGQqjHpNoVqtxsSJE3Hjxg2sWLECr7/+uk6/UqmEhYUFLl26pNOu0WiQkZEBNzc3sc3V1RWZmZl4+PChztiLFy+K/YbOSURERGQqjBYKS0tLMXXqVFy4cAGLFi1C27ZtK42xs7ODv78/UlNTdcJeamoqioqKEBwcLLYFBwdDq9Viy5YtYptGo0FycjJ8fHzQsGFDg+ckIiIiMhV6bx8XFhbi/fffR1RUFNq0afPchb/66iscOnQIXbp0QW5uLlJTU8W+unXrolu3bgCAadOmYciQIRgxYgTCwsKQlZWFNWvWoFOnTggICBCP8fb2RnBwMGJiYqBSqdCiRQukpKTgzp07iI6O1qmt75xEREREpkLvUKjVanHq1Cnk5eUBAIqKijB37lyMHTtWfCWdIf773/8CAA4fPozDhw/r9DVt2lQMhe7u7lizZg1iYmIQHR0NW1tbDBo0CNOnT6805/z587Fw4UKkpqYiLy8PLi4uWLlyJXx9fXXGGTInERERkSl4aiicMmUKfHx80K5dOzRq1EinT61WY/v27ejbt+8zhcLExES9x7Zv3x4bN278y3FWVlaYOXMmZs6cWWNzEhEREZmCp4bCR48eYenSpSgoKICFhQVkMhn27NkDGxsbNGvWDIJguo9gISIiInqVPDUUJiQkQBAEXLlyBceOHcOCBQuwc+dObN68GTY2NpDJZPjxxx9Rv359uLm51egr74iIiIio9vzl3ccymQyurq4YMGAAAOCbb75Bamoqxo0bB0EQkJSUhIEDB8LPzw8TJkyQfMFEREREVPOeeqZwzJgx8PX1ha+vL5o3bw6gPCS6uLhAoVBg0aJFWLFiBerVq4fTp0/jzJkztbJoIiKqPXb2dWAtl/ZdB8XaEhTkPpK0BhE93VP/K7e0tERiYiIWL14Mc3NzyGQypKSkAID4oGlzc3N4enrC09MTo0ePln7FRERUq6zlFui79XtJa+wI7Q2+3I/IuJ4aCpctWwYAuHHjBo4dO4a5c+fi8OHDSE1NhZWVFWQyGfbv3w9ra2t4eHjAwsLob80jIiIiomeg1xtNWrVqhV69egEAFi1ahD179iAyMhKCICAlJQVDhgxBhw4dEBERIeVaiYiIiEgiz/SaO2dnZ4SFhQEov/Fk165dmDFjBhwdHWt0cURERERUO/Te77WyskL//v3h5ORUqa9169Zo3bo1hg4dWqOLIyIiIqLaoXcotLGx0XmH8NNCIhERERG9XJ75zpAnQyIRERERvbye6ZpCIiIiInq18BkyREQvATt7a1jL5ZLWKNZqUZBbLGkNInpxMRQSEb0ErOVy9N6aJGmN70OHoQAMhUSmitvHRERERMRQSEREREQMhUREREQEhkIiIiIiAkMhEREREYGhkIiIiIjAUEhEREREYCgkIiIiIjAUEhEREREYComIiIgIDIVEREREBIZCIiIiIgJDIRERERGBoZCIiIiIwFBIRERERGAoJCIiIiIwFBIRERERGAqJiIiICAyFRERERASGQiIiIiICQyERERERgaGQiIiIiMBQSEREREQwcijMzs5GTEwMRowYgXbt2sHFxQUnT56scuzBgwfRv39/eHp64u2330Z8fDxKSkoqjcvPz8ecOXPw1ltvoW3bthg5ciQyMjKea04iIiKiV51RQ2FmZiYSEhJw9+5duLi4VDvuyJEjiIyMRP369TFnzhx069YNS5cuRXR0tM64srIyjB8/Hrt27cLw4cMxY8YM3L9/HyNGjMDvv//+THMSERERmQILYxZ3d3dHWloaHBwc8MMPPyAyMrLKcfPnz0ebNm3w7bffwtzcHABQt25drFy5EiNGjECrVq0AAHv37sX58+exdOlSdOvWDQDQs2dP9OjRA/Hx8Zg/f77BcxIRERGZAqOeKbS1tYWDg8NTx1y9ehVXr17F4MGDxfAGAEOHDkVZWRn2798vtu3btw9OTk4ICgoS2xwdHdGzZ0/88MMP0Gq1Bs9JREREZApe+BtNLl++DADw8PDQaW/YsCEaNWok9gNARkYG3N3dIZPJdMZ6enri4cOH4hayIXMSERERmQKjbh/rQ6VSAQAUCkWlPoVCgezsbJ2xb731VqVxTk5OAMpvbGndurVBc+qrQQNbg48hwykUdqzN2qzN2kQkgRc+FBYXFwMALC0tK/VZWVnh0aNHOmOrGlfRVjGXIXPq6/79QpSVCXCsbw1zS7nBxxuiVKNFTl6xpDUMVVs/zFWqAtZmbdY2odpEVLPMzGTVnsh64UOhtbU1AECj0VTqU6vVYn/F2KrGVbRVjDVkTkOZW8qhWrbumY/Xh2LScAAvVigkIiKil9sLf01hxRZvxZbv41Qqlbg1XDG2qq3firaKsYbMSURERGQKXvhQ6ObmBgC4dOmSTvvdu3eRlZUl9gOAq6srfvnlFwiCoDM2PT0dNjY2aNGihcFzEhEREZmCFz4UvvHGG3j99dexadMmlJaWiu0bNmyAmZkZunfvLrYFBwcjOzsbBw8eFNtycnKwd+9eBAUFQS6XGzwnERERkSkw+jWF33zzDQDg2rVrAIDU1FScPXsW9erVw/DhwwEAn3zyCSZNmoQxY8agV69e+PXXX5GUlITBgwfD2dlZnKtHjx5o27YtPvnkE4wePRoODg7YsGEDysrKMHnyZJ26+s5JREREZAqMHgoXLVqk83nbtm0AgKZNm4qhsEuXLoiPj0d8fDzmzp0LR0dHTJo0Ce+//77Osebm5li5ciXmz5+PxMREqNVqeHp64uuvv0bLli11xuo7JxEREZEpMHoovHLlil7junXrJr667mnq16+PL7/8El9++WWNzUlERET0qnvhrykkIiIiIukxFBIRERERQyERERERMRQSERERERgKiYiIiAgMhUREREQEhkIiIiIiAkMhEREREYGhkIiIiIjAUEhEREREYCgkIiIiIjAUEhEREREYComIiIgIDIVEREREBIZCIiIiIgJDIRERERGBoZCIiIiIAFgYewFUcxzrW8Hc0lLSGqUaDXLy1JLWICKqYGdvA2u5uaQ1irWlKMgtkrQG0cuAofAVYm5piaxvPpO0RqP3PwfAUEhEtcNabo4B29IkrZE88C0USFqB6OXA7WMiIiIiYigkIiIiIoZCIiIiIgJDIRERERGBoZCIiIiIwFBIRERERGAoJCIiIiIwFBIRERERGAqJiIiICAyFRERERASGQiIiIiIC331MRERUpXr2NrCSm0taQ60tRX5ukaQ1iPTFUEhERFQFK7k5pqTckrTG4v7NJZ2fyBDcPiYiIiIihkIiIiIiYigkIiIiIjAUEhEREREYComIiIgIJh4KNRoNFixYgMDAQHh5eWHQoEE4ceKEsZdFREREVOtM+pE0UVFR2L9/P0aOHImWLVsiJSUF48aNQ2JiItq1a2fs5RERkYmyt68LuVza8zZabRlycx9KWoNeLiYbCtPT07Fr1y7MmjULERERAICQkBD07t0bMTExSEpKMu4CiYjIZMnlZtiwTSVpjfCBCknnp5ePyW4f7927F3K5HGFhYWKblZUVQkNDcfbsWWRnZxtxdURERES1y2TPFGZkZMDZ2Rl169bVaffy8oIgCMjIyICTk5Pe85mZyf73a7u6TxlZMx6v9zhzO3uj1Zbb6f/7VdO1rW2NV7uubUOj1a5X13i1G9gYr7aTjfH+nDvZ2BmxtvF+tjjZ1DFabYWNldFqO9pI+5q7p9WuayP9eZuqatevZwMLibeuS7RlyMvn6/2Mobo/bwAgEwRBqMW1vDB69+6Nhg0b4ttvv9Vpv3r1Kt59913MmzdP5ywiERER0avMZLePi4uLIZfLK7VbWZX/H6lara7tJREREREZjcmGQmtra2i12krtFWGwIhwSERERmQKTDYUKhaLKm0lUqvK7vQy5npCIiIjoZWeyodDV1RWZmZl4+FD3GU0XL14U+4mIiIhMhcmGwuDgYGi1WmzZskVs02g0SE5Oho+PDxo2lP7OSiIiIqIXhck+ksbb2xvBwcGIiYmBSqVCixYtkJKSgjt37iA6OtrYyyMiIiKqVSb7SBqg/KaShQsXYufOncjLy4OLiwumT5+OgIAAYy+NiIiIqFaZdCgkIiIionIme00hEREREf0PQyERERERme6NJsaUnZ2N7777DhcvXsSlS5dQVFSE7777Dm+++aakddPT05GSkoKTJ0/izp07sLe3R7t27TB16lS0bNlS0to///wzli9fjsuXL+P+/fuws7ODq6srIiMj4ePjI2ntqiQkJCAmJgaurq5ITU2VrM7JkycxcuTIKvt2796N1q1bS1a7Qnp6OuLj43H+/HmUlJSgefPmiIiIwIABAySrGRUVhZSUlGr7//Of/0h6h/+NGzewcOFCnDt3Dvn5+WjSpAlCQkIQEREBS0tLyeoCwIULF/Cvf/0L6enpMDMzw5tvvomoqCi0aNGiRusY8nPk4MGDiI+Px9WrV9GgQQOEhoZi4sSJsLB4tr8C9K29YcMGpKWlIT09HXfu3EH//v3x1VdfPVNNQ2o/ePAA27Ztw6FDh3D9+nWUlJSgdevWiIiIQM+ePSWtLQgCPvvsM5w/fx5//vknSktL0bx5c4SGhiI8PLzKN2nVVO0n/fHHH+jVqxeKi4uxfft2uLm5SVq7a9eu+OOPPyodP27cOHz88ceS1gaAgoICLF26FPv27YNKpUKDBg3g6+uLuLg4yWo/7Wc8AEydOhWTJk2SpDZQfm/EmjVrkJqaKv693r59e3zwwQdwdnY2uC5DoRFkZmYiISEBLVu2hIuLC86fP18rdVetWoVz584hODgYLi4uUKlUSEpKQkhICLZu3SppQLl16xZKS0sRFhYGhUKBgoIC7Ny5E8OHD0dCQgI6duwoWe0nqVQqLFu2DDY2NrVWc9SoUXB3d9dpq43HHh05cgSRkZHw8/PDhx9+CAsLC9y4cQN//vmnpHUHDx4Mf39/nTZBEPDPf/4TTZs2lfS73717F2FhYbCzs8Pw4cNRv359nDlzBrGxsfjtt9+wYMECyWqnp6dj+PDhaNq0KSZPnoyysjKsX78eQ4cOxfbt2/Haa6/VWC19f45U/Bl46623MGfOHPz6669YunQpHjx4gDlz5khaOyEhAYWFhfD09BRfDPC89Kl94cIFLFy4EJ06dcKkSZNgYWGBffv2YerUqbh+/ToiIyMlq11WVoZffvkFgYGBaNasGczNzXHhwgX83//9Hy5duoT58+dLVvtJX3/9NczMnn9D0JDa7u7uGDVqlE6bUqmUvHZ+fj6GDRuG/Px8hIWFoVGjRlCpVDh9+rSktVu3bl3lv9MdO3bg6NGjz/x3m77fe8aMGTh48CAGDRqENm3aICsrC0lJSTh69Ch2796NBg0aGFZYoFpXUFAg5OTkCIIgCAcOHBCUSqWQlpYmed2zZ88KarVapy0zM1Pw8PAQZs6cKXn9JxUVFQkBAQHC+PHja7XuzJkzhREjRgjDhw8X+vbtK2mttLQ0QalUCgcOHJC0TlXy8/MFf39/Ye7cubVeuyqnT58WlEqlsGzZMknrrFixQlAqlcKvv/6q0z558mShTZs2gkajkaz2mDFjBD8/PyE3N1dsu3v3rtC2bVth3rx5NVpL358jvXr1Evr37y+UlJSIbXFxcYKrq6uQmZkpae3bt28LZWVlgiAIgq+vb438nNGn9u+//y7cvn1bp62srEwYOXKk4OXlJTx69Eiy2tWZO3eu4OLiIty/f79WaqelpQnu7u5CXFycoFQqhcuXLz9TXUNqd+nSRZg0adIz13me2nPmzBG6du0qjq3N2lV55513hO7du0taW6VSCUqlUvjqq6902g8dOiQolUph69atBtflNYVGYGtrCwcHh1qv6+PjU2nrrFWrVnjjjTdw7dq1Wl9PnTp14OjoiPz8/FqrmZ6ejh07dmDWrFm1VrNCYWEhSkpKaq3ezp07kZ+fjw8//FCsLxjxYQPff/89ZDIZevfuLWmdircUPfl/yK+99hosLCxgbm4uWe1z584hMDAQ9evXF9ucnJzg5+eHPXv21GgtfX6OXL16FVevXsXgwYN1vvfQoUNRVlaG/fv3S1YbAJo2bQqZTPZMNZ6ndvPmzdG0aVOdNplMhm7duqG4uLjKLc6aql2dJk2aQBAEFBQUSF67tLQUX375JYYPH14jlwYZ+r01Gg0ePXr03HX1rZ2fn4+UlBSMGTMGDg4OUKvV0Gg0tVK7Kunp6bh58yb69Okjae3CwkIAqLQDUfHZ2tra4LoMhSZOEATcu3ev1kJqYWEhcnJycP36dcTFxeHXX3+ttM0oFUEQMHfuXISEhDzztTXPasaMGfD19YW3tzdGjx6NK1euSF7zxIkTeP3113HkyBF07twZvr6+8PPzQ0xMDEpLSyWv/zitVos9e/agXbt2aNasmaS1OnToAAD49NNP8d///hd//vknduzYgZSUFIwbN65GttOqo9FoYGVlVand2toaKpWqyvetS+ny5csAAA8PD532hg0bolGjRmK/qbh37x4A1MrPO61Wi5ycHPz55584cOAAVq9ejebNm0v+5x8ANm7ciLt37+L999+XvNaTjh07hrZt26Jt27bo1q0bNm3aJHnNM2fOQKPR4LXXXkNERAS8vb3Rtm1bjB49Gr///rvk9Z+0Y8cOAHiuUKiPZs2aoXHjxlizZg0OHTqErKwsXLhwAV9++SVat26NoKAgg+fkNYUmbseOHbh79y6mTZtWK/X+8Y9/YN++fQAAuVyOIUOGYOLEibVSe/v27bh69SqWLl1aK/WA8u/Yo0cPdOrUCQ4ODrhy5QpWr16NoUOHYuvWrc90IbC+bt68iaysLERFRWHs2LFo06YNDh8+jISEBKjVanz66aeS1X7S0aNHkZubK/kPSQAIDAzEhx9+iBUrVuDQoUNi+5QpU575WjJ9OTs748KFCygrKxPDp0ajQXp6OoDyi8ednJwkXcPjKq7jUygUlfoUCkWth1Rjys3NxZYtW+Dn5wdHR0fJ6x09elTnZ5uHhweio6MlPVMNlH/PxYsXY/LkyahXr56ktZ6kVCrRvn17tGrVCg8ePMDmzZvx//7f/0NeXh7Gjx8vWd2K4Ddnzhx4eHggLi4O2dnZiI+Px6hRo7Bz507Y2tpKVv9xpaWl2LNnD7y8vCS/gdPCwgKLFy/GRx99pHMzS9u2bbFu3bpnOlPIUGjCrl27hi+++AK+vr7o169frdSMjIzE4MGDkZWVhdTUVGg0Gmi1WsnvCC0sLERsbCzGjx9fq38p+/j46NxdHRQUhK5du2LgwIGIj49HbGysZLWLioqQl5eHjz76SPyB3L17dxQVFWHDhg2YNGlSrfzlCJRvHcvl8ue689MQzZo1g5+fH9555x3Y29vjxx9/xJIlS+Do6Ijw8HDJ6g4dOhT//Oc/MXv2bIwePRplZWVYtmyZGM6Ki4slq12VinpV/fdlZWVVY1t8L7qysjJ8/PHHKCgowOzZs2ulpre3N9asWYOCggKkpaUhIyMDRUVFktddvHgxHB0dMWTIEMlrPWn58uU6nwcMGIChQ4fim2++QXh4OOzs7CSpW3HJiEKhQEJCgvg/ZM7Ozhg/fjy2bdtW6eYXqZw4cQL37t37/+3deUyTgkUfgAAAEP5JREFU9x8H8DdUMICccypDmcdcFQhgPDg35FBQU5EgolwDO4FpPGCToNmGUTONeMyUgehw4AIIcqgMoiKMIoiYybyGynQiFBQ55cZq+/vDtFnXqlxP+YmfV0Jiv0+f5/N9avP083yvByEhIUqJp6Ojg9mzZ2PJkiUwNzdHTU0N4uPjsXnzZiQkJAz4t5W6j99TjY2NCAkJga6uLg4fPsxol9q/sdls2NnZwdPTEwkJCfjrr7+UMr4vLi4OampqCAoKYjzW28yaNQs2Nja4cuUKo3Ekd4n/HcPH4XAgFApx69YtRuNLdHV1oaCgAPb29krptsvNzUVUVBR2796NVatWYfHixfjhhx/g4eGBffv24dmzZ4zFXrNmDUJDQ3H27FksW7YMHA4HNTU14HK5AAAtLS3GYisi+Q4oGl/V19c3qJaEd9GuXbtQUlKCPXv2gM1mKyWmgYEBbG1t4erqiqioKDg7OyMoKGjYZmErUlVVhZMnTyIyMnLQyw0NJxaLhS+++AI9PT2MrrIh+R67ubnJ/JY5ODhAV1cXFRUVjMX+r5ycHLBYLCxdupTxWB0dHfD19cXcuXMRHh4OFxcXrF27FjweD1evXsXp06cHfExKCt9DHR0dWLduHTo6OvDzzz8r7FpSBjU1NTg7O+PChQuMtqA8ffoUSUlJ8PHxQVNTEwQCAQQCAfr6+iAUCiEQCBhNFBQxNDRkPKbk//V1g5CVdc4XL15ET0+PUrqOASAlJQWmpqZyy944OTmhu7sbd+/eZTR+WFgYSktLkZycjLNnzyIzMxNisRgqKiqYMmUKo7H/S/IdUJSINDY2KrXVfKTExMQgJSUFW7duZXyS05u4ubmhu7sbBQUFjMU4ePAgTExMMGPGDOl1rrW1FcCr6yDTS1EpMmnSJADMXm9ed60DoNTJjL29vcjPz4eNjc2wLj/1OufPn0dTUxOcnJxkyhcsWIBx48YNKhke+VsJolR9fX0IDQ1FdXU1EhMTMX369BGtT29vL8RiMbq6uhhrtWhuboZQKMT+/fuxf/9+ue3Ozs5DWlx1MGpraxlvNTM1NcXly5fR0NAgk4w8efIEAJTWdZyTkwNNTU25CxdTmpqaFJ6bUCgEAKVMstHV1cW8efOkry9fvgxzc3OljWuSkEyoun37tsw6mQ0NDXjy5InSJ1wpW3JyMng8HgIDA6WttSNFcuM72NnH/fH48WPcvXtX4QSD4OBgjB8/HqWlpYzFV6S2thYAs9cbyXe7oaFBplwkEqGxsVFujVimFBYWoqurS2k3wM3NzQBenee/icViiESiQa12QUnhe+Tly5fYsmULrl+/jtjYWFhaWiotdktLi9xFobOzE+fPn4ehoeHAF9gcgMmTJyucXPLjjz+iu7sb27dvx9SpUxmJrei8//jjD5SXl2PFihWMxJRwc3PDsWPHkJGRIZ1IJBaLcerUKWhqairl/7+lpQVlZWVYtmwZNDQ0GI8HvBpHVFpaipqaGpmniOTm5oLFYimt+1AiLy8Pt27dGvRTFYZi5syZmD59OtLS0rBy5UrpJIfU1FSoqqpi8eLFSq+TsuTl5WH37t3gcDiIjIxUWty2tjZoa2vLTSg5deoUAPmZ4MNp27Zt0mVKJK5cuYJff/0V27ZtY7QRoK2tDTo6OjLdt319fUhISICWlhaj15sZM2bg008/RU5ODkJDQ6UrAOTl5aGzs1NpK1zk5ORAQ0MDixYtUko8ye9Wbm6uzEzzgoICdHd3w8TEZMDHpKRwhMTGxgKAdH3AM2fO4Nq1a9DR0YGfnx8jMffu3YvCwkI4Ojqira1N5vFuWlpacHFxYSQu8OpRP2PHjsWcOXPw4Ycf4vHjx8jKysKTJ08Y/7HU1tZWeG5JSUlgsViMn7eGhgbmzJkDfX19/P3330hLS4O+vj42btzIWFzg1Y/PihUrEB8fj+bmZpiYmIDP56OkpARbt25VSqtVXl4eXrx4obQ7ZwDgcrkoLi7GmjVr4OvrC11dXRQVFaG4uBirV69m9AakrKwM8fHxsLOzg56eHq5fv47s7GxwOBwsW7Zs2OP15zoSERGBr776ClwuF0uXLkVVVRWSk5Ph7e09pNnv/YldWFgo7a5//vw57t27J93P3d1dbi3B4Yp98+ZNREREQE9PDzY2NtIlQiTs7OwG3b33ttiFhYWIi4vDokWLYGxsjJ6eHpSUlKCkpAQLFy4cUoLyttjW1tZy+0i6Tq2srIbUMtyf8z5y5AhcXV1hZGSEtrY2ZGdno7q6Gjt27BjSeNr+fNciIyOxbt06+Pj4wN3dHY2NjUhKSoKJiQmWL1/OaGzgVVJ86dIlLF68eNjGDr8ttqOjI2bOnAkejweBQAALCwtUV1cjOTkZEydOHNSjTFXEI7ma7Xvsda0VRkZGMstoDCd/f39cvXpV6XEBICMjA2fOnMH9+/fR3t4ObW1t6TpSCxYsYCzum/j7+6O9vZ3RZx+fOHECOTk5qKmpQWdnJwwMDGBvb4+NGzfio48+YiyuxPPnzxEbG4vTp0+jqakJkydPRmBgoNJmJnp7e6O2thaXLl1ifCmOf7t58yZ4PB7u3LmDtrY2GBkZwdPTE1wul9F6VFdXY+fOnaisrERXVxemTp0KLy8v+Pn5MTKZq7/XkYsXLyImJgYPHjyAgYEBPD09sX79+iFNRuhP7Dc9A3soz3t/W+ysrKw3TmBjMnZVVRXi4+Px559/oqmpCaqqqpg2bRo4HA78/f0H/ezj/sRWRPJZDOXZx/2Jffv2bcTExKCyshItLS1QV1eHqakp1q5dC0dHx0HH7U9sieLiYvB4PNy7dw+amppwdnbGN998M6ShOv2NffLkSURFRSEuLm7Yhsr0J/azZ88QGxuLoqIi1NfXQ0tLC3Z2dggPDx/UTRclhYQQQgghhGYfE0IIIYQQSgoJIYQQQggoKSSEEEIIIaCkkBBCCCGEgJJCQgghhBACSgoJIYQQQggoKSSEEEIIIaCkkBDyHisvLwebzUZWVtZIV2XU8vf3V9pzrwkhQ0OPuSOEvDMG8tzigoICTJ48mcHaMEcgEMDZ2Rm+vr74/vvvR7o6b5WVlYX29nYEBgaOdFUIIUNASSEh5J2xb98+mdfXrl1DWloavL29MXfuXJltBgYGyqzaey07Oxt1dXWUFBLyjqOkkBDyznB3d5d5/fLlS6SlpcHS0lJuGyGEkIGhMYWEkFGnu7sbBw4cgIuLC8zMzGBnZ4eIiAjU1dX1a//s7GyYmppi06ZN6OvrAwB0dHQgOjoaixYtgpmZGaytrREeHo7a2lqZfbOyssBms1FWVoaEhARpHVxdXZGdnT3s5/r06VNERUVh4cKFMDMzg729Pb777js0NzfLvI/H44HNZuOff/7BwYMH8fnnn8PMzAzLly8Hn8+XO25PTw/27NkDe3t7mJubY9WqVSgrK0NkZKRMN76TkxOuXr2Kuro6sNls6V95ebnM8RoaGhAeHo758+fDwsICXC4XDx8+HPbPgxAyeNRSSAgZVYRCIbhcLioqKuDq6oqgoCA8evQIqampKC0tRWZmJiZNmvTa/Y8cOYJDhw7B19cX3377LVRVVdHR0YHVq1ejvr4enp6emDlzJhobG5GSkgIvLy9kZmbCyMhI5jiHDh1Cb28vvL29oa6ujtTUVERGRsLY2Fiuq3uw6uvr4e3tDaFQiJUrV8LY2Fh6ruXl5cjMzIS2trbMPpGRkRgzZgzWrl0LoVCIpKQkbNiwAefOnZMZg7l582bw+Xy4uLjA1tYWAoEAGzZskBunuX37dhw4cACtra3Ytm2btHzGjBnSf3d3d8PPzw8WFhYICwuDQCDAiRMnsH79evz2229gsVjD8nkQQoaGkkJCyKiSnZ2NiooKcLlcRERESMttbW0REhKCAwcOIDo6Wm4/kUiEXbt2ISUlBWFhYQgNDZVuO3z4MGpra5Geno5Zs2ZJyz08PMDhcMDj8bB3716Z4z1//hwZGRlQV1cHALi5ucHZ2RnJycnDlhTu2rULL168wOnTp2USXTc3N3h7eyMxMREbN26U2UdfXx9HjhyBiooKAMDKygpeXl5IS0vD119/DQDg8/ng8/nw8vLC7t27pftaW1sjODhY5nguLi5ISkpCX1/fa7vwW1tbweVysW7dOmmZgYEBoqOjcfnyZXz22WdD+yAIIcOCuo8JIaNKfn4+VFVVERISIlO+cOFCzJ49GwUFBRCJRDLb+vr6sGnTJqSnp2Pv3r0yCaFYLEZOTg7mz5+PCRMmoKWlRfqnoaEBS0tLlJSUyNXDx8dHmhACwMSJEzFt2jRUV1cPy3l2dHSgqKgITk5OUFdXl6mXkZERjI2NUVpaKrdfQECANCEEAHNzc2hqauLRo0fSssLCQgBAUFCQzL4ODg4yLYD9paqqioCAAJkya2trAJCJSwgZWdRSSAgZVQQCASZMmABdXV25bZ988gnu3LmD1tZWfPDBB9Ly6OhodHV1Yf/+/eBwODL7tLS0oK2tDSUlJbCxsVEYU1VV/v56ypQpcmV6enr9Htf4Ng8fPoRIJEJGRgYyMjIUvkdRHRSV6evro7W1VfpaIBBAVVUVxsbGcu+dNm0aHjx4MKC6TpgwAWPHjpUp09PTAwC0tbUN6FiEEOZQUkgIee+5uLjgwoULSEhIgL29PfT19aXbxGIxgFfdz//u/nwbRYnicJLUa/ny5fDw8FD4nv8mYgOt179bFIfiTWMGJedBCBl5lBQSQkaVKVOm4NKlS2hvb4eOjo7MtgcPHmDcuHEySR/wqivT09MToaGhCAgIQGJiorQl0cDAADo6Oujs7IStra3SzuNtjI2NoaKiAqFQOOz1MjIygkgkwqNHj+S6i2nGMCGjF40pJISMKi4uLhCJRDh69KhMOZ/PR2VlJZycnBS2lllZWeHYsWOoq6tDQEAAGhsbAbxqWeNwOLh58ybOnTunMOZ/l39RBn19fTg4OCA/Px/Xr1+X2y4Wi9HS0jKoY0seS5eYmChTzufzFXYda2lp4dmzZ9TqR8g7jloKCSGjioeHB7Kzs6UJ3rx581BTU4OUlBSMHz8e4eHhr9133rx5OH78OL788kv4+/sjKSkJEydORFhYGCoqKrBlyxYsWbIEFhYWUFNTQ319PYqLi2Fqaio3+3g43L59G7GxsXLlY8aMQXBwMHbs2AEfHx/4+fnB3d0dJiYmEIlEqK2tRUFBAVasWCE3+7g/HBwcYG9vj/T0dLS2tsLGxgYCgQDp6elgs9m4d++ezPstLCzw+++/Y+fOnZgzZw5YLBasra1lxm0SQv7/UVJICBlV1NTUkJCQgLi4OOTl5SE/Px/a2tpwc3PDli1bYGho+Mb9LS0t8csvv4DL5UoTQ0NDQ6SmpuL48eM4d+4cCgoKwGKxMGnSJMydOxdeXl6MnMuNGzdw48YNuXJ1dXUEBwfD0NAQmZmZOHbsGAoLC3H27FmMHTsWhoaGcHR0xJIlSwYVV0VFBTweD4cOHUJubi6Ki4vBZrMRExOD1NRUuRnDgYGBqK2txfnz53Hy5EmIRCKcOHGCkkJC3jEqYmrvJ4QQ0k8cDgdCofC1XemEkHcXjSkkhBAip7e3V66sqKgIVVVVsLOzG4EaEUKYRt3HhBBC5Pz000+orKyElZUVtLW1cefOHWRlZUFPT29AS/MQQt4d1H1MCCFEDp/Px9GjR3H//n10dnZCV1cX1tbW2Lx5Mz7++OORrh4hhAGUFBJCCCGEEBpTSAghhBBCKCkkhBBCCCGgpJAQQgghhICSQkIIIYQQAkoKCSGEEEIIKCkkhBBCCCEA/gdYKeXnll+mJQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QQ8YUh75Usqj"
      },
      "source": [
        "Let's look at just the tokens which begin with '##'."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "69Vf46p-Ym97"
      },
      "source": [
        "num_subwords = 0\n",
        "\n",
        "subword_lengths = []\n",
        "\n",
        "# For each token in the vocabulary...\n",
        "for token in tokenizer.vocab.keys():\n",
        "    \n",
        "    # If it's a subword...\n",
        "    if len(token) >= 2 and token[0:2] == '##':\n",
        "        \n",
        "        # Tally all subwords\n",
        "        num_subwords += 1\n",
        "\n",
        "        # Measure the sub word length (without the hashes)\n",
        "        length = len(token) - 2\n",
        "\n",
        "        # Record the lengths.        \n",
        "        subword_lengths.append(length)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lakK1gF-S1Ej"
      },
      "source": [
        "How many '##' tokens are there vs. the full vocab?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PJh6wl2bSzaC",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3c0628f9-5170-4279-9d11-2e7861715f09"
      },
      "source": [
        "vocab_size = len(tokenizer.vocab.keys())\n",
        "\n",
        "print('Number of subwords: {:,} of {:,}'.format(num_subwords, vocab_size))\n",
        "\n",
        "# Calculate the percentage of words that are '##' subwords.\n",
        "prcnt = float(num_subwords) / vocab_size * 100.0\n",
        "\n",
        "print('%.1f%%' % prcnt)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of subwords: 5,828 of 30,522\n",
            "19.1%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fj_AcObWS-1S"
      },
      "source": [
        "Plot the subword lengths (not including the two '##' characters)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fO1GeFbhSyhD",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 446
        },
        "outputId": "87de54be-d842-461a-bcb1-0a1d221c86fa"
      },
      "source": [
        "sns.countplot(subword_lengths)\n",
        "plt.title('Subword Token Lengths (w/o \"##\")')\n",
        "plt.xlabel('Subword Length')\n",
        "plt.ylabel('# of ## Subwords')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
            "  FutureWarning\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Text(0, 0.5, '# of ## Subwords')"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAoUAAAFjCAYAAABL3HHWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde1zO9/8/8MfV6UqlVLucQhHXVSoV6cTHMYfMhphDU87GsvOQ74fPbx8+nxmLDeVsNhS1kZhTymRaB6dl6LPow4RZF5111vv3h1vvj8tVuaKrxON+u+22vV/v1/v1fr6vK+vpdXpLBEEQQERERESvNJ2mDoCIiIiImh6TQiIiIiJiUkhERERETAqJiIiICEwKiYiIiAhMComIiIgITAqJmlxwcDAUCkVTh9EgAgICMGjQoEa/r0KhQHBwcKPf91Wxbt06KBQK3Lp1q1HuJwgCJkyYgE8++aRR7veqiouLg6OjI27cuNHUodALgkkhkYaysrKwZMkSDB8+HM7Ozujduzd8fX2xcOFCJCcnN3V4zUZAQAAUCoVG/+zbt6+pw30ugwYNwsiRI5s6DI2kpKRg3bp1KCgoaOpQ8OOPP+LSpUt47733Grztbdu2wdHREUVFRQ3SXkpKChQKBVJSUgA8+s6f9heU48ePQ6FQ4K+//gIAHD58GHZ2dhp99leuXIFCocD58+cBABcvXoRCocB//vMftbrBwcHiX9KejBMAfHx8IJfLERISotnD0ktPr6kDIGoOfvvtNwQEBEBPTw+jR49G165dUVpaij/++AOJiYkwNjaGp6dnU4fZLMyZMwfjxo0Tj3Nzc7F8+XK4ublh/PjxKnV79uzZ2OG9slJTUxEaGooxY8bA1NS0SWMJCwvDgAEDYGNj0+BtHz9+HF5eXjAxMWnwtjV1/vx5dOjQAW3atAEAnDt3Dl27dtXocz9//jykUikcHR0BAGfPnoWJiQnkcvkzxRIYGIiFCxfi6tWr6Nat2zO1QS8PJoVEGggLC0NJSQliYmJgZ2endl6pVDZBVI3r4cOHKC8vR4sWLZ6rnT59+qgc37p1C8uXL0fHjh0xatSo52qbmr+kpCRcv35dK0PHSqUSaWlp+Oyzzxq87fq4cOGCyl94zp07B1dXV42vdXR0hIGBgXits7MzdHSebeBvyJAh+Oyzz7Bnzx4sWbLkmdqglweHj4k0cOPGDbRq1arGhBAAZDKZ+N+3bt2CQqHAunXr1OrVNTcrJycHCxYsgIeHB1xcXDBlyhRcvnxZpc6gQYMwefJklbJNmzZBoVBg7ty5KuVffvklFAoF7t27p3KPf/7zn+jfvz8cHR3Rv39//POf/0Rubq7Ktfv27YNCocAvv/yCsLAw+Pj4oEePHjhy5AgAID8/H4sXLxZjDQgIwKVLl2r8bJ7H999/jzFjxqBHjx7o1asXpk+fjrNnz2p07eXLl9GnTx+MGDECd+7cAfBorlpERAT8/Pzg7OwMV1dXBAQEqA3/P/4d/vTTTxg7diycnJzQt29frFixApWVlQ36nOXl5di4cSNef/11ODk5wc3NDXPmzMGVK1dU6lUPAe7btw979+7F66+/DkdHRwwcOBBbtmypse2IiAgMGzYMjo6OGDp0KHbt2iV+v9VDicHBwQgNDQUADB48WBy+f/JnuLy8HKtXr0a/fv3g6OiIN998EwkJCWr33L9/P8aNGwc3Nze4uLhg8ODB+OSTT5CTk/PUz+LIkSPQ1dVV+8tDTfNVf/zxRygUCrz55ptqz6xQKJCWlqZSHh8fLz5jtcrKSmzevBkjRoyAk5MTPDw8EBQUhN9///2psWqqoqICOTk5yMnJgVKpxOXLlyGXy5GTk4M7d+4gIyMD3bp1E+tUVVWJ11ZVVYnlOTk5uHDhAuzs7MTj8+fPQ6FQiMcVFRX1is3Y2Bi9evXCsWPHGux5qfliTyGRBjp16oTr168jNjYWQ4cO1co9Zs6cCTMzM8ybNw/37t3Drl27MHnyZERGRopDQ56enjhw4ABKS0thaGgI4FHPio6ODs6cOYOHDx9CV1cXAJCcnIyuXbvitddeAwAUFhZi0qRJ+OOPPzB27Fh0794d6enp2L17N5KTk/H999+rDalVJ0Djx4+HsbExOnfujIqKCsyYMQO//fYbRo0aBWdnZ/znP//BtGnT0KpVqwb7PL788kts3boVPXr0wMcff4yioiJERUVhypQpWL9+Pfr371/rtT///DPef/99KBQKbNy4UYxr/vz5OHToEIYNGwY/Pz+Ul5fj4MGDmD59OtatW6eSLABAQkICIiIiMHHiRIwdOxbx8fH45ptvYGZmhjlz5jTIc1Z/nhcuXMCoUaPw9ttvi886adIk7Nq1C05OTirX7NmzB/fu3cO4ceNgamqKAwcOICQkBG3btsUbb7wh1tu8eTNWrVoFBwcHfPLJJygpKcG2bdtgbm6u0t6ECRNQVFSE48ePY9GiReL5JxdABQcHQ09PD9OnT0dFRQW+++47BAUF4ejRo+jQoQOARwnhwoUL4ebmhvfffx+Ghob4888/kZCQgPv378PCwqLOz+PMmTPo2rUrjIyMVMo9PT2xdu1a3Lx5E506dQLwv5/9jIwM5OTkiG0nJyfDxMREHGKtdvz4cTg7O4t/JgDg008/xZEjR9CnTx9MmjQJ9+7dQ3h4OCZOnIjw8HB07969zng1cf78eQQGBqqUhYSEqMzl+/e//41///vfAB4lr9Wf5507d9R+LsPDwxEeHi4ef/PNN/jmm28AADt27ICHh0e94nN1dcXp06eRmZkJW1vbel1LLxmBiJ7q/PnzgoODgyCXy4WhQ4cKwcHBQnh4uHDt2jW1ullZWYJcLhfWrl2rdm7t2rWCXC4XsrKyxLKFCxcKcrlcCAoKEqqqqsTy3377TVAoFML06dPFspiYGEEulwunT58WBEEQysrKhB49egiffvqpIJfLhbS0NEEQBKGgoECwt7cXli5dKl67evVqQS6XC7t27VKJadeuXYJcLhe++uorsWzv3r3isxYXF6vU37NnjyCXy4U1a9aolG/fvl2Qy+XCwIEDa/8ga1D9eS1cuFAsy8zMFBQKhTBx4kShrKxMLL97967Qq1cvYeDAgUJlZaVY/vj10dHRgoODgzB37lyhpKRErBMbGyvI5XJhz549KvevqKgQxowZIwwcOFD8/KtjcnZ2VvmuqqqqhNdff13o06ePRs82cOBA4fXXX6+zTvXndurUKZXywsJCoX///sLkyZPFsuTkZEEulwt9+vQRCgoKxPLi4mLBw8NDGD9+vFiWm5srODk5CSNHjhRKS0vF8uzsbKFnz56CXC4XkpOTxfKafjafPDd79myVn9G0tDRBLpcLISEhYllQUJDg6uoqVFRU1PncNamsrBTs7OyEoKAgtXNnz54V5HK5EBkZKZYNGjRI/Nk/dOiQIAiPviMPDw/hnXfeUbm+sLBQcHBwELZu3SqWnT59WpDL5cIHH3yg8lzp6emCvb29MGnSpHo/Q03y8vKExMREITExUfjggw8ELy8v8TgoKEgYMGCAeJyYmKjyfZWWlorly5YtE7p37y6cPHlSSExMFP7xj38ITk5OwunTp8U6eXl59Y5v//79glwuF44ePdogz0vNF4ePiTTg6uqKvXv3YsyYMSgsLMS+ffvwz3/+EyNGjMDbb7+NrKys577HzJkzIZFIxGNHR0f06dMHSUlJePDgAQCIi1mqhzsvXLiA0tJSzJw5Ey1btkRSUhKAR4sGHj58qLL45fjx47CwsMCECRNU7jthwgRYWFggLi5OLaZJkyapzSGMi4uDrq4upk+frlLu7+/fYJP34+PjIQgCZs6cKc6dAoA2bdrAz88Pt2/fVhtaBR71jAUHB8PPzw/r1q0Te1MB4MCBAzA2NoaPj4/KcFxBQQEGDRqE27dvq23NMXjwYLHHBgAkEgk8PDygVCrF7+R5HThwAF26dIGDg4NKXOXl5fD29sa5c+dQWlqqcs3YsWPRsmVL8bhFixZwcXFRif+XX35BWVkZJk2aBKlUKpbLZDKV3sT6CAwMVPkZ7dGjB4yMjPDHH3+IZS1btkRpaSlOnjwJQRDq1X5eXh6qqqpgZmamdq76XtU/+7dv38atW7cwcuRIyOVysfz3339Hbm6u2sKvkydPoqKiAj4+PmLZ8ePHATxa/PT4c9nZ2WHgwIE4d+6cRkPeT2NmZgZvb294e3vjzz//FP/b29sbt2/fRt++fVXKHv++pFKpWJ6Tk4MePXqgf//+8Pb2xv379+Hm5oY+ffqIdWr67J6muif9/v37z/2s1Lxx+JhIQwqFAl988QWAR7+Qzpw5g++//x5nz57Fu+++i71796okMPVV07CNra0tTp8+jTt37qBbt25o3bo1unTpIv4CTE5Ohkwmg0KhQO/evZGcnIx33nkHycnJ0NHRgbu7u9jWrVu34OjoCD091T/2enp6sLGxqTHJ6ty5s1pZVlYWZDKZWgJoYGCAjh07NsiWJtVzLmtaDVldlpWVpTKsGhsbiwcPHmD8+PFYunSp2nWZmZl48OABvL29a73v/fv3VZ65Y8eOanWqf4Hm5eXB2NhYwyeqXWZmJkpLS+Hl5VVrndzcXLRr1048fjxRfTyuvLw88bj6M6zpO6ypTBM1fR7m5uYqc1LfeecdnDlzBkFBQWjVqhXc3d3Rr18/+Pr6PvUvDdWJWU3JpL6+Pnr16iXOg0xKSoKenh7c3Nzg4eGBU6dOAfjfX5ieTAqPHz8OuVwOa2trsezWrVvQ0dGp8c9e165dERcXh1u3bj11yLsuFRUVKCwsBAA8ePAAv/32G9544w3k5OSgsLAQ6enp8Pf3F5PPx+9VVVWl8p0mJiZi7NixYt2UlBRMnjxZPG7VqtUzLzgBoJIY06uJSSHRM7CysoKVlRVGjRoFf39/nD9/HhcvXoSbm1ud/2NtiAUKnp6eiIyMRGFhIZKTk8X5Q56enli9ejXKy8uRnJwMe3v7Z+o1eNzjPW0vuh49euD27ds4duwYxo8frzYPTxAEWFhYYNWqVbW28WQSWj0/syb17QWrqx25XI5FixbVWufJpKSuuLRJk4TDxsYGhw8fRlJSEpKSkpCamorFixdj7dq1CA8PF+cD1qQ6qcnPz6/xvKenJ37++WdcvXoVycnJcHJyEreD2rlzJ+7cuYPk5GSYm5urzIcsLy/HqVOnMGXKlPo/9HOqaT7hsmXLsGzZMvF48eLFWLx4MQCoLHCpaT7htm3bsG3bNvF4/fr1WL9+PQDVuYj1UZ14Pk/ySy8HJoVEz0EikcDZ2Rnnz59HdnY2AIiJWE2/2Op6I0RmZiZcXFzUynR1ddG+fXuxzNPTExERETh58iR+++03+Pn5AQC8vLxQWlqK+Ph4XL16FdOmTVNpq2PHjrh+/ToqKytVegsrKytx48aNGnuBatKxY0ckJiaiqKhIpeenvLwcWVlZz52IVt8DAK5evaqWRFy7dk2lTrW2bdtixYoVCAwMxLRp07B161aVz9Pa2ho3btyAs7Nzg/TwNRRra2txuPN5enmeZGVlBQC4fv26Wi/k9evX1eo3ZC+RgYEB+vfvLy4GSkhIwOzZs7F9+3b8v//3/2q9rrrX7vHh6MdV9/4lJSUhOTlZ3O/Sw8MDurq6SExMxNmzZ+Ht7a3yPImJiSguLsaQIUNU2uvYsSOqqqqQmZmptrNAZmYmgJp7ZevDzs4O27dvBwDs2rULGRkZYk/2tm3bcOfOnVq3gpHJZOK1R48excGDBxEWFgYAiImJwYkTJ7BmzRqV+s/i5s2bAGrumadXC+cUEmkgMTGxxl6+0tJSJCYmAvjf8K+JiQlkMhmSk5NVepOysrJqnLdXbevWrSr1L1++jF9++QVeXl4qSYyHhwckEgk2bNiAiooK8RelXC6HpaUlQkNDIQiC2vBZ9Vy677//XqU8KioKOTk5KnOt6jJ48GA8fPhQXO1YLSIiosHeEjFo0CBIJBJs27ZNZYuN7Oxs7Nu3D1ZWVjWuCm3Tpg127dqF1q1bY/r06Th37px4bvTo0aiqqsLq1atrvOfjW/c0ptGjR0OpVIq//J/0rHF5e3vDwMAAu3fvRllZmViuVCpx8OBBtfrVq31r66XTVE1z8Kq/K03adnd3R2ZmZo0/S927d4eZmRn27NkDpVIp/oy3bNkS3bt3x7fffovCwsIah46trKzg4OCgUl79M79582aVP3sZGRk4ceIEevXq9dy9Z4/PJ8zOzoanp6d4fPfuXZW5hE9ObXh8PmF+fj5cXFzE49zcXPTq1avWuYj18euvv+K1115Dly5dnutZqfljTyGRBpYvX468vDwMGjQIcrkchoaGuHv3Lg4ePIgbN25g9OjRKsNVb7/9Nr7++mvMnDkTPj4+yM7Oxp49e9CtWzf89ttvNd7jzp07mDFjBgYNGgSlUoldu3bB0NAQ8+fPV6lXvV9ieno6rKysVHrMPDw8cPjwYejr68PNzU3lupkzZ+Lo0aNYunQprly5Ant7e6Snp+OHH35A586dMXPmTI0+Cz8/P0RFRSEsLAy3bt2Ci4sL0tPTcfToUXTq1AkPHz7U9GOtVZcuXTBjxgxs3boVkydPhq+vLx48eICoqCgUFxcjJCSk1iFUmUyGnTt3YurUqZg5cyY2bdoEd3d3DB8+HH5+fti1axcuX76MgQMHwtzcHHfv3sWvv/6KP/74Q9zHriHl5OSIw3tPGjt2LAIDA/HLL79g5cqVSE5OhqenJ0xMTMShUAMDA+zcubPe9zU3N8e8efOwevVqTJo0CW+++SZKSkoQFRUFGxsbXLp0SaU3zdnZGcCjrVLeeOMNSKVSdOvWrd5vypgxYwZatmwJNzc3tGvXDgUFBYiOjoZEItFoc/Lhw4cjPDwcp06dwogRI1TO6ejooHfv3oiLi4NUKlXZANrT01Pcq/HxpPDhw4f46aef8Prrr6vdq0+fPvD19cWhQ4eQn5+PgQMHQqlUIiIiAlKpVBzSbQhFRUW4cuWKuM9oTk4OMjMzMW/ePI2uP3v2LPz9/QE8mmt4/vz5BtkW6cGDBzh37hzGjh373G1R88ekkEgDwcHBiI+Px7lz53Ds2DEUFhaiZcuWkMvlmDVrljiEW23WrFkoLCzEgQMHkJqaiq5du+Lf//43Ll++XGtSuHXrVixfvhzr1q1DaWkpnJ2dsWDBgho3zPb09ER6erpaj4iXlxcOHz4MR0dHtSHSli1bYvfu3Vi7di1OnDiBffv2wdLSEhMnTsR7772n8cphAwMDfPPNN1i5ciXi4+MRGxsLJycnsez27dsatfM08+fPh7W1NSIiIrBq1Sro6+vD2dkZq1atUkt4n2RpaYkdO3Zg2rRpmD17NjZs2AAvLy8sX74cHh4eiIqKwqZNm1BRUQGZTIbu3btr5Q0awKPFK48P8T3O29sbbdq0waZNmxAREYGYmBhxw+jWrVvDyckJY8aMeeZ7v/POOzAxMcGOHTsQEhKC9u3bY8aMGRAEAZcuXVKZM9qrVy98+umn4pstKisrMW/evHonhZMmTcKRI0cQGRmJ/Px8tGrVCvb29li8eLFGr4J0d3dH165dceDAAbWkEHj0sx8XFwdXV1eVhV1eXl7YsmUL2rRpo9Ljdf78eeTk5KgNHVcLCQlB9+7dER0djS+++AJGRkbo3bs3PvjgA7V9Gp/H+fPn8fDhQ/Tu3RvAozeRCIIgHtclMzMT9+7dE+v+5z//QWFhoUbXPk1sbCxKSkrUdiWgV5NEaKjZ0kRE1CwsW7YMu3btwunTp595Hpo2HTp0CPPnz8ePP/743EOan3/+OWJiYvDLL7802QKdF9mYMWNgZWUlvtGGXm2cU0hE9JJ6fC5htezsbOzfvx9yufyFTAgBiK/7q15U8TxsbW2xePFiJoQ1iIuLw9WrV/Hpp582dSj0gmBPIRHRS+rUqVP48ssvMWTIELRt2xa3b99GVFQU8vLysGHDBgwYMKCpQySiFwjnFBIRvaSsra3RsWNHMRGUSqVwdHTEO++8U+cm3kT0amJPIRERERFxTiERERERMSkkIiIiInBOYYPJzX2AqiqOxBMREdGLS0dHAnPzml/1yaSwgVRVCUwKiYiIqNni8DERERERMSkkIiIiIiaFRERERAQmhUREREQEJoVEREREBCaFRERERAQmhUREREQEJoVEREREBCaFRERERAS+0YToldSqpQH0DaVNHYbGKkrLkFdY3tRhEBG91JgUEr2C9A2lOBw4ranD0NiIHdsBJoVERFrF4WMiIiIiYlJIREREREwKiYiIiAhMComIiIgITAqJiIiICEwKiYiIiAhNnBRmZ2cjJCQEAQEBcHV1hUKhQEpKilq9QYMGQaFQqP0TEhKiVregoABLliyBp6cnXFxcEBgYiPT09BrvHx8fjzFjxsDJyQkDBgxAaGgoKisrG/w5iYiIiF50TbpP4fXr17FlyxZYW1tDoVDgwoULtdZ1cHDAlClTVMrkcrnKcVVVFWbPno2MjAxMnz4d5ubmiIiIQEBAAPbt24dOnTqJdRMSEhAUFARPT08sWbIEGRkZCAsLQ25uLpYsWdKwD0pERET0gmvSpNDBwQHJyckwNzdHXFwcgoKCaq3btm1bjBo1qs72jh49igsXLiAsLAw+Pj4AAF9fXwwbNgyhoaFYuXKlWHflypXo3r07tm3bBl1dXQCAsbExNm/ejICAANjY2Dz/AxIRERE1E006fGxiYgJzc3ON65eXl6OkpKTW88eOHUPr1q0xePBgsczCwgK+vr6Ii4tDRUUFAODatWu4du0aJkyYICaEAODv74+qqirExsY+w9MQERERNV/NZqFJYmIiXFxc4OLiAh8fH0RGRqrVSU9Ph4ODAyQSiUq5k5MTHjx4gJs3bwIArly5AgBwdHRUqdemTRu0bdtWPE9ERET0qmgW7z6Wy+Vwc3ODjY0NcnNzERUVhX/84x/Iz8/H7NmzxXpKpRKenp5q17du3RrAo4Uttra2UCqVAACZTKZWVyaTITs7u94xWlqa1PsaItKcTNayqUMgInqpNYukcOPGjSrHfn5+8Pf3x/r16zFp0iS0bPnol0VpaSkMDAzUrq8uKy0tVfl3TXWlUmmdQ9S1uX+/CFVVQr2voxeTqZkU0hp+Pl5UZeXlKMgv07h+c0ywlMrCpg6BiKjZ09GR1NqR1SySwifp6upiypQp+Oijj3DhwgX069cPAGBoaIjy8nK1+tVlhoaGKv+uqW5ZWZl4nl5dUgMDTN3+QVOHobFvp60BoHlSSERE9KRmM6fwSW3btgUA5Ofni2W1Df1Wl1UPI1cPG1cPIz9OqVSK9YiIiIheFc02KczKygLwaHVxNTs7O1y+fBmCoDqMe/HiRRgZGYn7FNrb2wMALl26pFLvr7/+wt27d8XzRERERK+KFz4pzMvLQ1VVlUpZWVkZtm3bBmNjY7i4uIjlw4cPR3Z2NuLj48WynJwcHD16FIMHD4a+vj4AoFu3bujSpQsiIyPx8OFDse7u3buho6ODoUOHavmpiIiIiF4sTT6ncP369QCAzMxMAEBMTAzOnTsHU1NTTJ48GSdOnMDGjRsxbNgwWFlZIS8vD9HR0bhx4wY+++wzGBsbi20NGzYMLi4uWLBggfhGk927d6Oqqgrvvfeeyn0XLFiAuXPnYsaMGRgxYgQyMjIQHh6OCRMmoHPnzo33ARARERG9AJo8KVyzZo3K8d69ewEAVlZWmDx5MuRyObp06YKYmBjk5OTAwMAADg4OCA4OxsCBA1Wu1dXVxebNm7Fy5Urs3LkTZWVlcHJywooVK2Btba1Sd+DAgQgNDUVoaCiWLVsGCwsLzJ07F++++652H5iIiIjoBSQRnpyAR8+EW9K8XGSyls1u9XF9tmyRyVricOA0LUbUsEbs2M4taYiIGkBdW9K88HMKiYiIiEj7mBQSEREREZNCIiIiImJSSERERERgUkhEREREYFJIRERERGBSSERERERgUkhEREREYFJIRERERGBSSERERERgUkhEREREYFJIRERERGBSSERERERgUkhEREREYFJIRERERGBSSERERERgUkhEREREYFJIRERERGBSSERERERgUkhEREREqEdS+Mcff+DUqVMqZWlpaZgzZw4mTpyIyMjIBg+OiIiIiBqHnqYVQ0JCkJeXh379+gEAcnJyMGvWLBQXF0MqleKzzz6DpaUlfHx8tBYsEREREWmHxj2Fly5dgre3t3h86NAhFBUVYd++fUhKSoKzszO+++47rQRJRERERNqlcVKYk5OD1q1bi8c///wzevbsCblcDgMDA4wYMQKZmZlaCZKIiIiItEvjpLBFixYoLCwEADx8+BDnzp2Dm5ubeN7Q0BBFRUUNHyERERERaZ3GSWG3bt2wf/9+5ObmIioqCsXFxejTp494/vbt27CwsNBKkERERESkXRovNJkxYwbeffddcV6hvb29Sk9hYmIiunfv3vAREhEREZHWaZwUDhgwAN999x3i4+NhYmKCyZMnQyKRAAByc3PRtm1bjB49ul43z87Oxo4dO5CWloZLly6huLgYO3bsgIeHh1gnNzcXe/fuxYkTJ/Df//4XlZWVsLW1xdSpU+Hr66vS3r59+7Bo0aIa73Xx4kVIpVKVsvj4eISGhuLatWuwtLTEuHHjMGfOHOjpafyxEBEREb0U6pX99O7dG71791YrNzc3R2hoaL1vfv36dWzZsgXW1tZQKBS4cOGCWp1ff/0VX3/9Nfr164e5c+dCT08Px44dw4cffoj//ve/CAoKUrvmo48+Qrt27VTK9PX1VY4TEhIQFBQET09PLFmyBBkZGQgLC0Nubi6WLFlS72chIiIias6atEvMwcEBycnJMDc3R1xcXI0JXteuXXHs2DFYWVmJZf7+/pg6dSo2b96MGTNmwNDQUOWa/v37w97evs57r1y5Et27d8e2bdugq6sLADA2NsbmzZsREBAAGxub539AIiIiomai1qQwMDCw3o1JJJJ67VVoYmLy1DodO3as8T4+Pj5ITk7G7du3YWtrq1anqKgIRkZG0NFRX0tz7do1XLt2DUuXLhUTQuBRsrlx40bExsZi9uzZGj8HEeeFtG8AACAASURBVBERUXNXa1J469YttbKSkhLk5uYCAExNTQEABQUFAB4NIRsZGWkjxhrdu3dPvO+T/P39xTetDBgwAMHBwWjfvr14/sqVKwAAR0dHlevatGmDtm3biueJiIiIXhW1JoUnTpxQOc7KykJgYCACAwMxa9YsyGQyAIBSqcTmzZsRHx+Pb7/9VqvBVsvLy8P3338Pd3d3lW1wWrRoAT8/P3h4eMDY2BhpaWn47rvvkJaWhujoaLGuUqkEAPEZHieTyZCdnV3vmCwtn97rSaRNMlnLpg5Bq1725yMiamoazyn8/PPP4erqiv/7v/9TKZfJZPj73/+Oe/fuYfny5Vi/fn2DB/m4qqoqfPrppygsLMTixYtVzvn6+qqsSB4yZAh69+6N2bNn47vvvsNHH30EACgtLQUAGBgYqLUvlUpRUlJS77ju3y9CVZVQ7+voxdQcExClslDjui/78xERUc10dCS1dmRpvHl1ampqjSuPq7m7uyM1NbX+0dXTsmXLcPr0aSxfvhwKheKp9fv3748uXbogKSlJLKtemFJeXq5Wv6ysTG3hChEREdHLTuOkUCKR1Plu42vXrjVIQHUJDQ1FREQE5s+fj5EjR2p8Xbt27ZCfny8ePz70/SSlUqnyjmciIiKiV4HGSWGfPn2wZ88e7N+/H4Lwv2FSQRAQHR2NyMhIldfeNbTw8HCsW7cOU6dOxYwZM+p1bVZWlsqClOrtai5duqRS76+//sLdu3efup0NERER0ctG4zmFixYtwm+//YZFixYhJCRE3Mfvxo0buH//Ptq1a1fr20Se1+HDh/Gvf/0Lb7zxBoKDg2utl5OTo/b+5YMHD+LmzZsYNWqUWNatWzd06dIFkZGRGDdunLgtze7du6Gjo4OhQ4dq5TmIiIiIXlQaJ4Vt27ZFTEwMtmzZgvj4eFy8eBHAo30E/fz8MHPmTHGbmvqoXphSPTQdExODc+fOwdTUFJMnT8bFixexYMECtGrVCl5eXjhw4IDK9X369MFrr70GAJg4cSIcHBzQvXt3mJiY4OLFi9i/fz9sbGwwZcoUlesWLFiAuXPnYsaMGRgxYgQyMjIQHh6OCRMmoHPnzvV+DiIiIqLmTCI8PhZci/LycqSlpUEmkzX4mz5qWyxiZWWFEydO1Pk+YwAq70r+6quvcPLkSdy+fRulpaVo3bo1Bg0ahHnz5qFVq1Zq18bFxSE0NBSZmZmwsLDA2LFj8e677z7Tu4+5+vjlIpO1xNTtHzR1GBr7dtqaeq8+Phw4TYsRNawRO7Zz9TERUQOoa/WxRklhZWUlnJ2dsXDhwmd608mrgEnhy4VJ4YuFSSERUcN47i1p9PT08Nprr0GD/JGIiIiImiGNVx8PHz4cR44cQVVVlTbjISIiIqImoPHkubfeegspKSmYNm0apkyZAmtra7Ro0UKt3uPvGCYiIiKi5kHjpHDkyJGQSCQQBKHON5ekp6c3SGBERERE1Hg0TgqDgoIgkUi0GQsRERERNRGNk8L33ntPm3EQERERURPSeKEJEREREb286rVLc1VVFaKjo3H8+HHcunULANChQwcMHToUo0ePho4Oc0wiIiKi5kjjpLC0tBSzZs3C2bNnIZFIIJPJAACnTp1CQkIC9u/fjy1btkAqlWotWCIiIiLSDo279jZs2IAzZ85g2rRpSEpKQkJCAhISEpCcnIzp06cjNTUVGzZs0GasRERERKQlGieFhw8fhq+vLxYsWAAzMzOx3NTUFPPnz4evry8OHTqklSCJiIiISLs0Tgrv3r0Ld3f3Ws/37t0bd+/ebZCgiIiIiKhxaZwUmpqa4ubNm7Wev3nzJkxNTRskKCIiIiJqXBonhd7e3ggPD8fPP/+sdu706dPYvXs3+vbt26DBEREREVHj0Hj18YcffojTp09j9uzZsLe3R7du3QAAV69eRXp6OszNzfH+++9rLVAiIiIi0h6Nk0IrKyvs3bsXq1atwk8//YQrV64AAIyNjfH666/j448/Rvv27bUWKBERERFpT702r27fvj1WrVoFQRCQk5MDALCwsOA7kYmIiIiaOY2TwqqqKvGNJRKJBJaWlloLioiIiIgal8ZJobu7O9zc3ODp6QlPT0/Y2dlpMy4iIiIiakQaJ4V9+/bFmTNncPLkSUgkEpiZmcHd3V1MErt06aLNOImIiIhIizROCr/++msAQEZGBpKTk8V/YmNjxXche3p6YuXKlVoLloiIiIi0Q+N9CqvJ5XIEBgZi/fr1SElJwZo1a2Bra4vs7GwcPHhQGzESERERkZbVa/UxAOTk5Ii9hElJSbh16xZ0dHTg7OwMT09PbcRIRERERFqmcVL4+eefIzk5GVevXgXwqMdw0KBB8PLygpubG0xMTLQWJBERERFpl8ZJ4Y4dO6Crq4s333wT8+bNQ8eOHbUZFxERERE1Io2TwvHjxyM5ORkxMTE4dOgQHB0dxZXHPXv2hIGBgTbjJCIiIiIt0jgpXLp0KQDgzz//RFJSElJSUhAdHY2NGzdCKpXCxcUFXl5emDNnjtaCJSIiIiLtqPfq43bt2sHPzw8rVqxAQkICvv76a3To0EFciVwf2dnZCAkJQUBAAFxdXaFQKJCSklJj3fj4eIwZMwZOTk4YMGAAQkNDUVlZqVavoKAAS5YsgaenJ1xcXBAYGIj09PTnapOIiIjoZVfv1cc3b94UVx+npKQgJycHgiDAzMwMHh4e9Wrr+vXr2LJlC6ytraFQKHDhwoUa6yUkJCAoKAienp5YsmQJMjIyEBYWhtzcXCxZskSsV1VVhdmzZyMjIwPTp0+Hubk5IiIiEBAQgH379qFTp071bpOIiIjoVaBxUrho0SKkpKTgzz//hCAIMDY2Vnntnb29fb1v7uDggOTkZJibmyMuLg5BQUE11lu5ciW6d++Obdu2QVdXFwBgbGyMzZs3IyAgADY2NgCAo0eP4sKFCwgLC4OPjw8AwNfXF8OGDUNoaKjKxtqatklERET0KtB4+Pjw4cPo1KkTPvjgA+zZswepqanYtGkTpk2b9kwJIQCYmJjA3Ny8zjrXrl3DtWvXMGHCBDF5AwB/f39UVVUhNjZWLDt27Bhat26NwYMHi2UWFhbw9fVFXFwcKioq6t0mERER0atA457CM2fONMkK4ytXrgAAHB0dVcrbtGmDtm3biucBID09HQ4ODpBIJCp1nZycEBkZiZs3b8LW1rZebRIRERG9CjROCp9MCEtLSwEAhoaGDRvRE5RKJQBAJpOpnZPJZMjOzlapW9NbVVq3bg3g0cIWW1vberWpKUtLbt5NTUsma9nUIWjVy/58RERNrV4LTe7fv49169YhLi4O9+/fBwBYWlrCx8cH8+bNw2uvvdbgAVYnnzX1UkqlUpSUlKjUraledVl1W/VpU1P37xehqkoQj1uaGsJQql/vdppSaVkFCgtKmzqMF0JzTECUykKN677sz0dERDXT0ZHU2pGlcVKYlZUFf39/KJVKdO7cGS4uLgCAzMxM7NmzB/Hx8YiIiGjwN51U90SWl5ernSsrK1PpqTQ0NKyxXnVZdd36tPnMcUv14b8g/LnbaUwRK99GIZgUEhERvYo0TgpXrFiBvLw8hIaGiit7qx0/fhwff/wxVqxYgdDQ0AYNsHqIV6lUisPA1ZRKJVxdXVXq1jT0W11WfX192iQiIiJ6FWi8+jgpKQlvv/22WkIIAEOGDMGkSZOQlJTUoMEBEFc2X7p0SaX8r7/+wt27d1VWPtvZ2eHy5csQBEGl7sWLF2FkZCTuU1ifNomIiIheBRonhRKJBNbW1rWet7GxUVv12xC6deuGLl26IDIyEg8fPhTLd+/eDR0dHQwdOlQsGz58OLKzsxEfHy+W5eTk4OjRoxg8eDD09fXr3SYRERHRq0Dj4ePevXsjJSUFkyZNqvF8amoq3N3d6x3A+vXrATyamwgAMTExOHfuHExNTTF58mQAwIIFCzB37lzMmDEDI0aMQEZGBsLDwzFhwgR07txZbGvYsGFwcXHBggULxDea7N69G1VVVXjvvfdU7qtpm0RERESvAonw5FhrLbKyshAYGIhhw4Zh1qxZsLS0BPBoRfLmzZtx/Phx7NixAx06dKhXAAqFosZyKysrnDhxQjyOi4tDaGgoMjMzYWFhgbFjx+Ldd9+Fnp5qXpufn4+VK1ciLi4OZWVlcHJyQnBwMBwcHNTuoWmbmnhy9bFM1rJZLjThCs9HZLKWmLr9g6YOQ2PfTltT79XHhwOnaTGihjVix3b+bBIRNYC6Vh/XmhQ+/laQasXFxcjLywMAmJqaAgAKCgoAAK1atYKxsTHi4uIaJOjmhknhy4VJ4YuFSSERUcN4pi1p2rdvr7WAiIiIiOjFUmtSuHPnzsaMg4iIiIiakMarj4mIiIjo5cWkkIiIiIg035LGzs7uqfsQSiQSXLly5bmDIiIiIqLGpXFSOHr0aLWksLKyEllZWUhLS4NCoeCbQIiIiIiaKY2Twi+++KLWc+fPn8fcuXPx2WefNURMRERERNTIGmROYc+ePeHn54eQkJCGaI6IiIiIGlmDLTSxsbHB5cuXG6o5IiIiImpEDZYUpqamQiqVNlRzRERERNSINJ5TuH///hrL8/LykJSUhFOnTmHcuHENFhgRERERNR6Nk8Lg4GBIJBLU9KpkPT09jBs3DosWLWrQ4IiIiIiocWicFO7YsUOtTCKRwMzMDB06dICRkVGDBkZEREREjUfjpNDd3V2bcRARERFRE3quhSZ//fUXLl68iIKCgoaKh4iIiIiaQJ1JYXp6OrZv347c3FyV8pycHMycORMDBgzAhAkT4O3tjdDQUK0GSkRERETaU2dSuHv3bnz33XcwNzdXKV+8eDFOnz6NDh06YMiQITAzM0NYWBji4uK0GiwRERERaUedcwp//fVX9OvXT6Xs9u3bOHHiBOzs7BAVFQUDAwPk5OTAz88PUVFR8PHx0WrARERERNTw6uwpzM7Oho2NjUpZcnIyAMDf3x8GBgYAAAsLC7z55pu4cuWKdqIkIiIiIq2qMyksLi5Gy5YtVcouXrwIiUQCDw8PlfKOHTsiLy+v4SMkIiIiIq2rMyls27Ytbt68qVJ24cIFmJqawtraWqX84cOHMDY2bvgIiYiIiEjr6kwKHR0dsX//fmRnZwN4lBBmZGTAy8tLre61a9fQunVr7URJRERERFpV50KT2bNn49ixY/D19UXnzp1x7do16OjoIDAwUK3uyZMn1YaUiYiIiKh5qLOn0M7ODqGhoWjfvj0yMjLQoUMHfPXVV+jZs6dKvZ9//hn3799XW6lMRERERM3DU19zN3DgQAwcOLDOOn/7299w4cKFBguKiIiIiBrXc73mjoiIiIheDkwKiYiIiOjpw8cvguDgYERHR9d6/tSpU2jTpg0CAgKQmpqqdn7EiBH46quvVMrKy8uxZs0axMTEoKCgAHZ2dvjoo49qXFlNRERE9LJrFknhhAkT1JI1QRDw2WefwcrKCm3atBHL27dvjw8//FClrpWVlVqbwcHBiI2NRWBgIKytrREdHY1Zs2Zh586dcHV11c6DEBEREb2gmkVS6OrqqpaonT17FiUlJXjjjTdUyk1NTTFq1Kg627t48SIOHTqERYsWYerUqQCA0aNHY+TIkQgJCUF4eHiDxk9ERET0omu2cwp//PFHSCQSjBw5Uu1cZWUlHjx4UOu1R48ehb6+Pt566y2xTCqVYty4cTh37py4WTcRERHRq6JZJoUVFRU4cuQIXF1d0aFDB5VzmZmZcHFxQc+ePdG3b19s3LgRVVVVKnXS09PRuXNntdfy9ejRA4IgID09XevPQERERPQi0Xj4uKioCO+++y6Cg4PRvXt3bcb0VKdPn0ZeXp7a0HHHjh3h4eEBhUKBoqIi/Pjjj/jqq69w584dLF26VKynVCpV5iFWk8lkAPBMPYWWlib1vuZFJJO1bOoQ6Bm97N/dy/58RERNTeOksKKiAqmpqcjPzwcAFBcXY9myZZg5cyZsbW21FmBNfvzxR+jr68PX11el/PPPP1c5HjNmDD744ANERUVh6tSp6NKlCwCgtLQU+vr6au1KpVIAQFlZWb1jun+/CFVVgnjcXH+BKZWFTR3CC6E5fn/1+e5e9ucjIqKa6ehIau3IqnP4+P3338e3336LtLQ0lJeXq5wrKyvD/v37G33+3YMHDxAfH4++ffvC3Nz8qfWnT58OQRCQkpIilhkaGqKiokKtbnUyWJ0cEhEREb0q6uwpLCkpQVhYGAoLC6GnpweJRIIjR47AyMgIHTp0gCAIdV2uFXFxcTWuOq5N27ZtAUDs4QQeDRPXlMwqlUoAQOvWrRsgUiIiIqLmo86kcMuWLRAEAb///jsSExPx5Zdf4uDBg4iKioKRkREkEglOnjwJMzMz2NvbQyKRaD3ggwcPwsjICIMGDdKoflZWFgDAwsJCLLOzs8POnTvx4MEDlcUmaWlp4nkiIiKiV8lTVx9LJBLY2dnBz88PALB+/XrExMRg1qxZEAQB4eHhGDt2LNzd3fHOO+9oNdicnBwkJSVhyJAhaNGihcq5oqIitSHuhw8fYtOmTdDR0VHZ/Hr48OGoqKjA999/L5aVl5dj37596NmzZ42LUIiIiIheZnX2FM6YMQO9evVCr1690LFjRwCPkkSFQgGZTIY1a9Zg06ZNMDU1xZkzZ3D27FmtBnv48GFUVlbWOHR8+fJlfPLJJxg5ciQ6deqE4uJiHDlyBJcuXcKsWbPE+AHA2dkZw4cPR0hICJRKJTp16oTo6GjcuXMHy5cv1+ozEBEREb2I6kwKDQwMsHPnTqxduxa6urqQSCTiO4irV/Lq6urCyckJTk5OmD59ulaDPXjwICwtLeHt7a12rn379ujZsydiY2Nx79496OjooFu3bvjiiy8wZswYtforV67E119/jZiYGOTn50OhUGDz5s3o1auXVp+BiIiI6EVUZ1K4YcMGAMCNGzeQmJiIZcuW4aeffkJMTAykUikkEgliY2NhaGgIR0dH6Olp9615kZGRtZ7r2LEj1q5dq3FbUqkUCxcuxMKFCxsiNCIiIqJmTaMszsbGBmZmZli2bBnWrFmDtm3b4vjx41i9ejWio6MRERGBFi1awNnZGd9++62WQyYiqp2ZaQsYSJvFa91F5WWVyC8oaeowiOgV90z/5+zcuTPeeustrF69GuvXr0fr1q2RkpKi9TmFRERPYyDVw+d//6Gpw6iX//v3uKYOgYhI86RQKpVizJgxNe7hZ2trC1tbW/j7+zdocERERETUODROCo2MjFRW5taVJBIRERFR8/LME2+eTBKJiIiIqPl66ubVRERERPTy0zgpLC4uxqJFi5CZmanNeIiIiIioCWicFJaVlWH//v3Izs4GAJSWliI0NBS3bt3SWnBERERE1DjqTApXrFiB48eP4969ewAAQRDEcyUlJQgLC0NWVpZ2IyQiIiIiratzoUlCQoK4GXWbNm0gkUhw9uxZdOvWDbq6uipJIhERERE1X3X2FB4+fBi//PIL1q1bBx8fHwiCgLCwMPztb3/DpEmTIJFIcOnSJeTl5TVWvERERESkBU+dU2hubg4fHx+8++67AIDQ0FCEhYXB3d0dgiBg1apV8Pb2xhtvvIGlS5dqPWAiIiIianh1Dh//4x//gJubG3r27AljY2MAgLGxMby8vODi4oKoqCiEhIRAT08PZ86c4WvuiIiIiJqpOpPCX3/9FT/88AMEQUCrVq0gkUiQmJiIDh06wMTEBABgaWkJLy8vDB8+vFECJiIiIqKGV2dSeODAARQVFSEtLQ3JycnYsmULtm3bhm3btqFt27aQSCS4cOECbG1t+bo7IiIiombsqXMKTUxM0KdPH0yfPh0AsGHDBmzZsgUDBw6EIAhYu3Yt+vfvj6FDh+Lvf/+71gMmIiIiooZX73cfS6VSeHl5oXv37oiIiEBoaCj09fWRmprKOYVEREREzZTGSaG+vj569+4NMzMzAIBEIgHwv4Un/fv3106ERERERKR1GieFJiYm2Llzp3j8ZJJIRERERM1XvYePqz2ZJBIRERFR8/XUhSZERERE9PJjUkhERERETAqJiIiIiEkhEREREYFJIRERERGhjqQwNDQUGRkZ4vGdO3dQWlraKEERERERUeOqMyn8/fffxePBgwfj+PHjjRIUERERETWuWpNCU1NTFBQUiMeCIDRKQDVJSUmBQqGo8Z/MzEyVuufPn8ekSZPg7OyMPn364F//+hdKSkrU2iwvL8eXX36Jvn37okePHhg/fjySkpIa65GIiIiIXii1bl5tb2+Pbdu2obKyUnxrydmzZ/Hw4cM6Gxw9enTDRviYKVOmwMHBQaWsTZs24n+np6dj6tSp6Nq1K4KDg3H37l188803uHXrFjZu3KhyXXBwMGJjYxEYGAhra2tER0dj1qxZ2LlzJ1xdXbX2DEREREQvolqTwkWLFmHevHlYvnw5gEfvOo6MjERkZGStjUkkEq0mhe7u7vDx8an1/OrVq9GqVSvs3LkTxsbGAIAOHTpg8eLFSEpKgpeXFwDg4sWLOHToEBYtWoSpU6cCeJTMjhw5EiEhIQgPD9faM7wszM0MoGcgbeow6qWyvAy5+eVNHQYREdELqdak0M7ODseOHUNWVhaUSiUCAgIwZ84ceHt7N2Z8aoqKimBoaAg9PT218l9++QUzZswQE0IAGDVqFD7//HMcOXJETAqPHj0KfX19vPXWW2I9qVSKcePG4auvvkJ2djZat27dOA/UTOkZSHFu5cymDqNeei3YCoBJIRERUU3qfPexrq4ubGxsYGNjg969e8PDwwPu7u6NFZua+fPno7i4GHp6evDw8MDChQuhUCgAAL///jsqKyvh6Oioco2BgQHs7e2Rnp4ulqWnp6Nz584qySMA9OjRA4IgID09nUkhERERvVLqTAoft3PnTm3GUSd9fX0MGzYM/fr1g7m5OX7//Xd888038Pf3xw8//IDOnTtDqVQCAGQymdr1MpkMv/76q3isVCpV5iI+Xg8AsrOz6x2jpaVJva95EclkLZs6BK16mZ/vZX42gM9HRKRtGieFAFBVVYXo6GgcP34ct27dAvBozt7QoUMxevRo6OhoZy/snj17omfPnuLx4MGDMWjQIIwdOxahoaFYtWqVuIeigYGB2vVSqVRlj8XS0lLo6+vXWA8AysrK6h3j/ftFqKr63wrt5vo/eKWyUKN6fL4Xj6bPBrzcz9ccnw2o3/dHRPSsdHQktXZkaZwUlpaWYtasWTh79iwkEonYq3bq1CkkJCRg//792LJli5hYaZudnR28vLyQnJwMADA0NATwaKuZJ5WVlYnnq+tWVFTUWA9Aoz0DERER0YtC4669DRs24MyZM5g2bRqSkpKQkJCAhIQEJCcnY/r06UhNTcWGDRu0Gauadu3aIT8/H8D/hn6rh5Efp1QqVeYIymSyGoeIq6/lfEIiIiJ61WicFB4+fBi+vr5YsGCBuG8h8GiT6/nz58PX1xeHDh3SSpC1ycrKgrm5OQBALpdDT08Ply5dUqlTXl6O9PR02Nvbi2V2dna4fv06Hjx4oFI3LS1NPE9ERET0KtE4Kbx7926dK4979+6Nu3fvNkhQT8rJyVErO3v2LFJSUtC3b18AQMuWLeHl5YWYmBiVZC8mJgbFxcUYPny4WDZ8+HBUVFTg+++/F8vKy8uxb98+9OzZs8ZFKEREREQvM43nFJqamuLmzZu1nr958yZMTU0bJKgnffjhh2jRogVcXV1hbm6Oq1evIjIyEubm5njvvffEeh999BEmTpyIgIAAvPXWW7h79y62b9+Ofv36qeyv6OzsjOHDhyMkJARKpRKdOnVCdHQ07ty5I27WTURERPQq0Tgp9Pb2Rnh4OLy9vfG3v/1N5dzp06exe/duld64huTj44ODBw9i+/btKCoqgoWFBUaOHIn33nsP7du3F+s5ODhg+/btCAkJwfLly2FiYoLx48fj448/Vmtz5cqV+PrrrxETE4P8/HwoFAps3rwZvXr10sozEBEREb3INE4KP/zwQ5w+fRqzZ8+Gvb09unXrBgC4evUq0tPTYW5ujvfff18rQQYGBiIwMFCjum5ubtizZ89T60mlUixcuBALFy583vCIiIiImj2Nk0IrKyvs3bsXq1atwk8//YQrV64AAIyNjfH666/j448/Vum1IyIiIqLmo16bV7dv3x6rVq2CIAji4g8LCwtIJBKtBEdEREREjaNeSWE1iUQCS0vLho6FiIiIiJqIdt5LR0RERETNCpNCIiIiImJSSERERERMComIiIgITAqJiIiICEwKiYiIiAj1SAqLiooQGBgoblpNRERERC8PjZPCiooKpKamIj8/HwBQXFyMRYsWITMzU2vBEREREVHjqDMpfP/99/Htt98iLS0N5eXlKufKysqwf/9+ZGdnazVAIiIiItK+Ot9oUlJSgrCwMBQWFkJPTw8SiQRHjhyBkZEROnToAEEQGitOIiIiItKiOpPCLVu2QBAE/P7770hMTMSXX36JgwcPIioqCkZGRpBIJDh58iTMzMxgb2/PdyATERERNVNPnVMokUhgZ2cHPz8/AMD69esRExODWbNmQRAEhIeHY+zYsXB3d8c777yj9YCJiIiIqOHV2VM4Y8YM9OrVC7169ULHjh0BPEoSFQoFZDIZ1qxZg02bNsHU1BRnzpzB2bNnGyVoIiIiImpYdSaFBgYG2LlzJ9auXQtdXV1IJBJER0cDALp06QIA0NXVhZOTE5ycnDB9+nTtR0xEREREDa7OpHDDhg0AgBs3biAxMRHLli3DTz/9hJiYGEilUkgkEsTGxsLQ0BCOjo7Q06uzOSIiIiJ6QWm0T6GNjQ1GjBgBAFizZg2OHDmCoKAgCIKA6OhoTJw4Eb1798bUqVO1GSsRERERackzveauc+fOeOuttwA8Wnhy6NAh2tPQiQAAHqtJREFUzJ8/HxYWFg0aHBERERE1Do3He6VSKcaMGYPWrVurnbO1tYWtrS38/f0bNDgiIlJlZmoAA6m0qcOol/KyMuQXlD+9IhE1KY2TQiMjIyxfvlw8ritJJCIi7TCQSrF6UfPa/uvj5ZsAMCkketE988qQJ5NEIiIiImq+nmlOIRERERG9XJgUEhERERGTQiIiIiJiUkhEREREeI6FJo3p4sWLiI6ORkpKCu7cuYNWrVrB1dUVH374IaytrcV6AQEBSE1NVbt+xIgR+Oqrr1TKysvLsWbNGsTExKCgoAB2dnb46KOP4OXlpfXnISIiInrRNIukcOvWrTh//jyGDx8OhUIBpVKJ8PBwjB49Gj/88ANsbW3Fuu3bt8eHH36ocr2VlZVam8HBwYiNjUVgYCCsra0RHR2NWbNmYefOnXB1ddX6MxERERG9SJpFUjh16lSEhITAwMBALBsxYgTeeOMNbNmyBV988YVYbmpqiv/f3r1HRVXv7wN/BAZQRC4KlgiJBoOGgFBclHU8KhqZJqYEIhQJmGYm1knpVMeiztEUL4Ro5OUsPSl5AwEvZKLmUcG8gzimEhiEyCBxU4fhsn9/+GO+TYMdjIHtwPNai7XkvffseT4za5i3+7MvU6ZM+cPt5ebmYv/+/Xj//fdVt+YLCAjApEmTEBcXh23btnXIOIiIiIgeVzpxTKG7u7taQwg8uB+zg4MDCgoKNNZvbGzE3bt3H7q9zMxMSCQS1a36gAcX454+fTrOnTuH8vJy7YUnIiIi0gE60RS2RhAEVFRUwMLCQq1eUFAANzc3uLu7w9fXF19++SWam5vV1pHJZLC3t4eJiYla3cXFBYIgQCaTdXh+IiIioseJTkwftyY9PR23b9/GwoULVTVbW1t4eXlBKpWirq4O+/btw+rVq1FaWorY2FjVenK5HP3799fYppWVFQD8qT2Fffv2/hOjePxYWZmKHaFDdeXxdeWxARyfruvq4yPqCnSyKSwoKEBsbCw8PDzUjh/817/+pbbe1KlTsWDBAuzcuRPh4eEYPHgwAEChUEAikWhs1+j/32S+vr7+kTPduVOH5mZB9buu/gGUy2vbtB7H9/hp69iArj0+XRwbwPERUefQ0+vx0B1ZOjd9LJfL8cYbb8DMzAzx8fHQ0/vjIcyaNQuCIOD06dOqmrGxMRoaGjTWbWkGW5pDIiIiou5Cp/YU1tbWIioqCrW1tUhOTlZN9/6RJ554AgBQXV2tqllZWbU6RSyXywEA1tbWWkpMREREpBt0Zk9hfX095syZg6KiIiQlJammgv+X4uJiAIClpaWq5uTkhMLCQo0zlC9duqRaTkRERNSd6ERT2NTUhOjoaFy8eBHx8fFwc3PTWKeurg5KpVLjcUlJSdDT01O7U4m/vz8aGhqwa9cuVU2pVCIlJQXu7u6tnoRCRERE1JXpxPTxsmXLcOTIEYwZMwZVVVVIS0tTLTMxMYGfnx/y8/Px7rvvYtKkSbCzs8O9e/dw8OBBXL58GVFRUbC1tVU9xtXVFf7+/oiLi4NcLoednR1SU1NRWlqKpUuXijFEIiIiIlHpRFN49epVAMDRo0dx9OhRtWU2Njbw8/PDgAED4O7ujkOHDqGiogJ6enpwcHDAsmXLMHXqVI1tLl++HGvWrEFaWhqqq6shlUrx1VdfwcPDo1PGRERERPQ40Ymm8D//+c//XMfW1hZffPFFm7dpZGSExYsXY/Hixe2JRkRERNQl6MQxhURERETUsdgUEhERERGbQiIiIiJiU0hEREREYFNIRERERGBTSERERERgU0hERERE0JHrFBIRUfdgYdYTBoa69dXUqGzEr9X3xY5B1G669ckjIqIuzcDQAJfWHRM7xiNxffOvYkcg0gpOHxMRERERm0IiIiIiYlNIRERERGBTSERERERgU0hEREREYFNIRERERGBTSERERERgU0hEREREYFNIRERERGBTSERERERgU0hEREREYFNIRERERGBTSERERERgU0hEREREYFNIRERERAAMxA5ARETUXZiZGcPQUCJ2jDZTKhtQXa0QOwZ1EjaFREREncTQUIKVK1eKHaPN3n33XQBsCrsLTh8TERERUfduCpVKJVasWAFfX1+4uLjglVdeQXZ2ttixiIiIiDpdt24KY2JisGXLFrz00kv44IMPoKenh6ioKFy4cEHsaERERESdqts2hbm5udi/fz/+9re/YdGiRQgKCsKWLVvw5JNPIi4uTux4RERERJ2q255okpmZCYlEgsDAQFXNyMgI06dPx+rVq1FeXg5ra2sRExIREekWC3MjGEgMxY7RZo0NSvxaVS92jMdGt20KZTIZ7O3tYWJiolZ3cXGBIAiQyWSP1BTq6fXQqPWzMGllzcdba+N4GMM+fTswScd4lPH1623ZgUm071HGBgA9++nW+/co4zMz79WBSTrGo4yvj7luvXfAo41PYmrcgUk6xiO9f336dGAS7XuUsRlIDHE6a00HptEur3HR0NNTih2jU/3R+9lDEAShE7M8NiZNmoT+/ftj06ZNavUbN27gxRdfxGeffaa2F5GIiIioK+u2xxQqFApIJJoXEDUyMgIA1NdzdzIRERF1H922KTQ2NkZDQ4NGvaUZbGkOiYiIiLqDbtsUWllZoby8XKMul8sBgCeZEBERUbfSbZtCJycnFBYW4u7du2r1S5cuqZYTERERdRfdtin09/dHQ0MDdu3apaoplUqkpKTA3d0d/fv3FzEdERERUefqtpekcXV1hb+/P+Li4iCXy2FnZ4fU1FSUlpZi6dKlYscjIiIi6lTd9pI0wIOTStasWYOMjAxUV1dDKpXinXfewciRI8WORkRERNSpunVTSEREREQPdNtjComIiIjo/7ApJCIiIqLue6KJLikvL8fWrVtx6dIlXL58Gffu3cPWrVvh5eUldrR2y83NRWpqKk6fPo3S0lKYm5tjxIgRiI6OxlNPPSV2vHbLy8vDl19+iStXruDOnTswNTWFk5MT5s2bB3d3d7Hjad2GDRsQFxcHJycnpKWliR2n3U6fPo1XX3211WUHDhzAkCFDOjmR9uXm5mLt2rW4cOECGhsbYWtri/DwcLz88stiR2uXmJgYpKamPnT58ePHdf4qE0VFRVizZg3Onz+PmpoaDBgwAAEBAQgPD4ehoaHY8drt4sWLWL16NXJzc6GnpwcvLy/ExMTAzs5O7GiP5FG+w7OysrB27VrcuHEDffv2xfTp0zFnzhwYGHROu8amUAcUFhZiw4YNeOqppyCVSnHhwgWxI2nNxo0bcf78efj7+0MqlUIul2Pbtm0ICAjA7t27df5Lt7i4GE1NTQgMDISVlRVqa2uRkZGB0NBQbNiwAaNGjRI7otbI5XKsX78evXr1EjuK1r322mt45pln1Gq63lAAwPfff4958+bB09MTCxYsgIGBAYqKinDr1i2xo7VbUFAQfHx81GqCIODjjz+GjY2Nzr9/t2/fRmBgIExNTREaGgozMzOcPXsWK1euxPXr17FixQqxI7ZLbm4uQkNDYWNjg/nz56O5uRnbt29HSEgI9u7di379+okdsc3a+h3e8nn09vbGRx99hGvXriExMRG//vorPvroo84JK9Bjr7a2VqisrBQEQRC+++47wdHRUcjJyRE5lXacO3dOqK+vV6sVFhYKzs7OwuLFi0VK1bHu3bsnjBw5Upg9e7bYUbRq8eLFQlhYmBAaGiq89NJLYsfRipycHMHR0VH47rvvxI6idTU1NYKPj4/w6aefih2l05w5c0ZwdHQU1q9fL3aUdktKShIcHR2Fa9euqdXnz58vDBs2TFAqlSIl046IiAjB09NTqKqqUtVu374tuLm5CZ999pmIyR5dW7/DJ06cKEydOlVobGxU1VatWiU4OTkJhYWFnZKVxxTqgN69e8PCwkLsGB3C3d1dY5pj0KBBcHBwQEFBgUipOlbPnj1haWmJmpoasaNoTW5uLtLT0/H++++LHaXD1NXVobGxUewYWpORkYGamhosWLAAwIPxCV38YhT79u1Djx49MGnSJLGjtFvL3bj69u2rVu/Xrx8MDAygr68vRiytOX/+PHx9fWFmZqaqWVtbw9PTEwcPHhQx2aNry3f4jRs3cOPGDQQFBam9dyEhIWhubsahQ4c6OiYAnmhCjyFBEFBRUdGlGuG6ujpUVlbip59+wqpVq3Dt2jWNqS1dJQgCPv30UwQEBGDo0KFix+kQ7733Hjw8PODq6opZs2bhxx9/FDtSu2VnZ2Pw4MH4/vvvMXr0aHh4eMDT0xNxcXFoamoSO57WNTQ04ODBgxgxYgQGDhwodpx2e+655wAAH3zwAa5evYpbt24hPT0dqampiIqKgp6ebn+9K5VKGBkZadSNjY0hl8tRXl4uQqqOc+XKFQCAs7OzWr1///544oknVMs7Go8ppMdOeno6bt++jYULF4odRWv+/ve/49tvvwUASCQSBAcHY86cOSKn0o69e/fixo0bSExMFDuK1kkkEjz//PP4y1/+AgsLC/z444/YvHkzQkJCsHv3btjb24sd8U+7efMmysrKEBMTg8jISAwbNgxHjx7Fhg0bUF9fjw8++EDsiFp14sQJVFVVYfLkyWJH0QpfX18sWLAASUlJOHLkiKr+9ttvY968eSIm0w57e3tcvHgRzc3NqgZXqVQiNzcXwIOTN6ytrcWMqFVyuRwAYGVlpbHMysqq05pgNoX0WCkoKEBsbCw8PDwwZcoUseNozbx58xAUFISysjKkpaVBqVSioaFB588QrKurw8qVKzF79uwu9Qe6hbu7u9pZ4uPGjcPYsWMxbdo0rF27FitXrhQxXfvcu3cP1dXVePfddzF79mwAwIQJE3Dv3j0kJydj7ty5sLS0FDml9uzbtw8SiQQvvPCC2FG0ZuDAgfD09MT48eNhbm6OY8eOISEhAZaWlpgxY4bY8dolJCQEH3/8MT788EPMmjULzc3NWL9+vap5UigUIifUrpbxtPadYGRkhPv373dKDjaF9NiQy+V44403YGZmhvj4eJ2f/vgtqVQKqVQKAHjppZcwbdo0vP/++/jiiy9ETtY+69evh0Qiweuvvy52lE7j5OQEHx8f5OTkiB2lXYyNjQFA4/i6yZMnIzMzE3l5eRg9erQY0bTu7t27yMrKgq+vb5c5LGX//v1YsmQJMjMzVWdST5gwAYIgYPny5Zg4caLa8Xi6ZsaMGSgrK8OmTZuwZ88eAA+mViMiIvDll1/CxMRE5ITa1fJ5VCqVGsvq6+tVyzta1/nWJZ1WW1uLqKgo1NbWYuPGja3uQu8qJBIJxo0bh0OHDun0/3bLy8uxZcsWhISEoKKiAiUlJSgpKUF9fT0aGhpQUlKC6upqsWN2iCeffFLnx9byGfv9pT1aftf18f3W4cOHcf/+/S4zdQwA27dvxzPPPKNxaZ2xY8fi3r17uHr1qkjJtGfhwoU4efIktm3bhvT0dOzZsweCIKBHjx6wtbUVO55WtXweW/aE/pZcLu+0mRg2hSS6+vp6zJkzB0VFRUhKSsLgwYPFjtThFAoFBEFQnUGoi+7cuYOGhgbExcVh3Lhxqp9Lly6hoKAA48aNw4YNG8SO2SGKi4t1fo9Ty3UXb9++rVYvKysDgC41dZyRkYFevXph7NixYkfRmoqKilZPCGpoaACALnOykJmZGZ599lnVTMupU6fg4uKC3r17i5xMu1pO0rt8+bJa/fbt2ygrK+u0k/jYFJKompqaEB0djYsXLyI+Ph5ubm5iR9KqyspKjVpdXR2+/fZbPPnkkxqXk9AlAwcORGJiosaPg4MDbGxskJiYiICAALFjtktr79/Zs2dx+vRp+Pr6ipBIe/z9/QEAu3fvVtUEQcCuXbvQq1evLvNZrKysRHZ2NsaPH4+ePXuKHUdr7O3tcfnyZfz8889q9f3790NfX1/VRHUlBw4cQF5eHl577TWxo2idg4MDBg8ejB07dqg19MnJydDT08OECRM6JQePKdQR69atAwDVtfvS0tJw7tw59OnTB6GhoWJGa5dly5bhyJEjGDNmDKqqqtRujWZiYgI/Pz8R07VfdHQ0jIyMMGLECFhZWeHWrVtISUlBWVkZVq1aJXa8djE1NW31/dmyZQv09fV1/r0DHrx/PXv2xIgRI2BhYYHr169jx44dsLCwwPz588WO1y7Ozs4ICAhAUlIS7ty5g2HDhuH777/HiRMn8N5773WZPTEHDhxAY2Njl5o6BoCIiAgcP34cM2bMwMyZM2FmZoZjx47h+PHjCA4O1un/cAIPLpmUlJSEUaNGwdzcHBcvXkRqaiomT56MF198Uex4j6wt3+GLFi3C3LlzERERgYkTJ+LatWvYtm0bgoKCOu1KBz2Ern610i7iYf/rs7GxUbscga4JCwvDDz/80OoyXR8b8GAvTFpaGm7cuIGamhqYmprCzc0Ns2bNgqenp9jxOkRYWBhqamq6xL2Pt27dioyMDPz888+oq6uDpaUlfH19MX/+fAwYMEDseO2mVCqxbt067N27FxUVFRg4cCDCw8MRHBwsdjStCQoKQnFxMf773//q/AWdfy83NxcJCQmQyWSoqqqCjY0Npk2bhoiICJ0fa1FREWJjY3HlyhXcvXsXgwYNQmBgIEJDQ3XyJMS2focfPnwYa9euRUFBASwtLTFt2jS8+eabnXbvYzaFRERERMRjComIiIiITSERERERgU0hEREREYFNIRERERGBTSERERERgU0hEREREYFNIRERERGBTSERdVExMTFd5lZfYWFhXeq+vW3Rld4/Il3B29wRkaiKi4vx1Vdf4cyZM7h16xYMDQ3Rr18/uLi4YOrUqfD29hY7ok6RSqX461//iqSkJLGj/E+HDx+GTCbT+VsGEnUVbAqJSDR5eXkICwuDgYEBAgIC8PTTT0OhUODmzZs4efIkTExM2BR2YYcPH0ZqaiqbQqLHBJtCIhJNYmIi7t+/j7S0NDg5OWksl8vlIqTqXE1NTVAqlejZs6fYUYiom+MxhUQkmqKiIpibm7faEAKAlZWV6t8lJSWQSqVISEjQWC8hIQFSqRQlJSUayyorK7Fo0SJ4eXnBzc0Nr732GvLz89XWGTt2LEJDQ9VqSUlJkEqlmDt3rlp9xYoVkEqlqKioUHuOTz75BKNHj4azszNGjx6NTz75BL/++qvaY1NSUiCVSnHq1CkkJibCz88PLi4uOHjwIACguroaH374oSprWFgYLl++3Opr0161tbVYsWIFxo8fD2dnZ3h7e+Odd95BcXFxq5mzs7OxadMm+Pn5wdnZGc8//zxSU1M1ttvU1ITExESMGTMGw4cPx+TJk3HgwAGN9ygsLEz1eKlUqvpJSUnRyLlkyRL4+Phg+PDhCA4OxqVLlzrkNSHq7rinkIhEY2dnh8LCQhw6dAgTJkzokOeIjIyEmZkZ3nrrLVRUVODrr79GaGgoduzYAUdHRwCAt7c30tPToVAoYGxsDADIzs6Gnp4ezpw5g6amJujr6wMAcnJy8PTTT6Nfv34AHjQtM2bMwM2bNzFt2jQMGzYMMpkMycnJyMnJwa5du9C7d2+1TJ9//jkaGxvxyiuvwMTEBPb29mhoaEBERATy8vIwZcoUuLq64urVq3j99ddhbm6u1dektrYWwcHBKC0txbRp0+Dg4AC5XI7t27cjMDAQe/bsgY2NjdpjVq9eDYVCgaCgIBgaGiI5ORkxMTGws7ODh4eHar3Y2Fh888038PLywqxZs1QN8++3N2fOHDQ3N+Ps2bNYvny5qu7u7q62XkREBCwtLTFv3jxUVVXh3//+N2bPno2srCyN15WI2odNIRGJZu7cuTh16hTmz5+PQYMGwd3dHcOHD4eXlxeGDBmilecYMGAAEhIS0KNHDwDA+PHjMX36dHz++efYtGkTgAdN4Z49e3Du3DmMGjUKSqUSFy5cwKRJk5Ceno78/Hy4uLigtrYWMpkMM2bMUG1/48aNKCoqwj/+8Q/MnDlTVR86dChiY2OxceNGREdHq2VSKBTYu3ev2pTxjh07kJeXh3nz5uHtt99W1YcMGYKlS5dqNFXtER8fj+LiYuzcuVNtL+3UqVMxefJkJCQkYNmyZWqPUSqV2L17NwwNDQEA/v7+GDduHLZt26ZqCq9fv45vvvkGvr6+2LBhA/T09FTrBgQEqG1v1KhRyMjIwNmzZzFlypSHZh02bBg+/vhj1e9DhgxBdHQ09u3bh+Dg4Ha9DkSkjtPHRCSaESNGYM+ePZg6dSpqa2uRkpKCTz75BBMnTsTMmTM1pjL/jMjISFVDCADOzs4YNWoUsrOzcffuXQBQncySk5MDALhw4QIUCgUiIyNhamqK7OxsAMAPP/yApqYmtZNfvvvuO1haWiIoKEjteYOCgmBpaYnDhw9rZJoxY4bGMYSHDx+Gvr4+Zs2apVYPCQnR6h4xQRCQkZGB5557DtbW1qisrFT99OzZE25ubjhx4oTG40JCQlQNIQD0798f9vb2KCoqUtWOHj0KAHj11VdVDSHwYHrY19f3T+UNDw9X+73ltb958+af2h4RPRz3FBKRqKRSqWqv1C+//IIzZ85g165dOHv2LN58803s2bNHrRl5VK3tcRwyZAhOnDiB0tJSODg4wNraGoMHD1Y1hTk5ObCysoJUKsVzzz2HnJwcvPHGG8jJyYGenh48PT1V2yopKYGzszMMDNT/nBoYGGDQoEG4cuWKxvPb29tr1IqLi2FlZaXRABoaGsLW1hY1NTV/avy/V1lZiaqqKpw4cQI+Pj6trvPbhq6Fra2tRs3c3By//PKL6veW4wUHDx6ssa69vT2OHz/+yHl//7wWFhYAgKqqqkfeFhH9MTaFRPTYsLGxgY2NDaZMmYKQkBCcP38eubm5ePbZZ9X29v1eY2Nju5/b29sbO3bsQG1tLXJycuDl5aWqr1q1CkqlEjk5ORg6dCjMzMza9Vwtxy2KQRAEAMDIkSMRFRXV5se11ih2hpZjOX+vZRxEpD2cPiaix06PHj3g6uoKACgvLwcAVSNWXV2tsX5rZx23KCgoaLWmr6+PAQMGqGre3t5oamrCsWPHkJeXp5qm9PHxgUKhQFZWFq5fv65qFlvY2tqisLBQozFtbGxEUVFRq3vYWmNrawu5XI66ujq1ulKp1Mo0egtLS0v06dMHdXV1GDly5EN//oyBAwcCAH766SeNZYWFhRq1P2r0iajzsSkkItGcPHmy1b18CoUCJ0+eBPB/07+9e/eGlZUVcnJy1PYSFRcXt3rcXouNGzeqrZ+fn49Tp07Bx8cHJiYmqrqXlxd69OiB9evXo6GhQdUUOjo6om/fvli7di0EQdC4mLafnx8qKyuxa9cutfrOnTtRWVkJPz+/Nr0W48aNQ1NTEzZv3qxW3759u0aj2B56enqYPHkycnNzkZmZ2eo6d+7c+VPbHjNmDABg69ataG5uVtV//PHHVo9T7NWrFwBOBRM9Ljh9TESiWbp0KaqqqjB27Fg4OjrC2NgYZWVlyMjIQFFREQICAtTufztz5kysWbMGkZGR8PPzQ3l5Ob755hs4ODggLy+v1ecoLS1FREQExo4dC7lcjq+//hrGxsZ477331NZruV6iTCaDjY2N2h4+Ly8vHDhwABKJBM8++6za4yIjI5GZmYnY2FhcuXIFQ4cOhUwmw+7du2Fvb4/IyMg2vRYvv/wydu7cicTERJSUlMDNzQ0ymQyZmZmws7NDU1NTW19W3Lx5E+vWrWt1WXh4OBYuXIjz588jOjoaL7zwAlxdXSGRSFBaWorjx4/jmWee0Tj7uC0cHBwQFBSEHTt2IDw8HOPHj0dlZSW2b9+OoUOHIj8/X23voKurK77++mvVNR4lEglcXFzavHeViLSLTSERiSYmJgZZWVk4d+4cvv32W9TW1sLU1BSOjo6IiorCyy+/rLZ+VFQUamtrkZ6ejh9++AFPP/00/vnPfyI/P/+hTeHGjRuxdOlSJCQkQKFQwNXVFYsWLWr1gtne3t6QyWQaewN9fHxw4MABODs7q+1dBABTU1MkJyfjiy++wJEjR5CSkoK+ffsiODgY8+fPb/OZw4aGhti8eTOWL1+OrKwsHDp0CMOHD1fVfntCx/9SWFiI+Pj4VpcFBgbCysoKycnJ2Lx5MzIzM5GVlQV9fX088cQT8PDwQGBgYJuf6/eWLFkCa2tr7N69G59//jns7e2xZMkS5OXlIT8/X+14ykmTJkEmk2H//v3IzMxEc3Mzli5dyqaQSCQ9BB6tS0REHWzOnDnIycnBuXPnHnryCBGJi8cUEhGR1igUCo3a1atXcfz4cXh7e7MhJHqMcfqYiIi0JjU1FWlpaRg9ejQsLS3x008/YefOnZBIJGp3aiGixw+nj4mISGtyc3MRHx8PmUyG6upqmJiYwN3dHW+99RacnZ3FjkdEf4BNIRERERHxmEIiIiIiYlNIRERERGBTSERERERgU0hEREREYFNIRERERGBTSEREREQA/h9EYsc2Fq12ZQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 720x360 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gCDYsO8qyepC"
      },
      "source": [
        "### Misspellings"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p-n_ObF14sh3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "799d0000-2272-40e8-ce20-e229aa2724ac"
      },
      "source": [
        "'misspelled' in tokenizer.vocab # Right"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FnUu6C_VjTVQ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "622c983f-86d1-4f29-ad35-767db69d5686"
      },
      "source": [
        "'mispelled' in tokenizer.vocab # Wrong"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GrKICg1y45Rz",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9187b839-88e7-4757-ad69-fd87b27f5b86"
      },
      "source": [
        "'government' in tokenizer.vocab # Right"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fO72IlrU5I8t",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7541884a-bdbf-4494-c4c7-54f1751fe430"
      },
      "source": [
        "'goverment' in tokenizer.vocab # Wrong"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FfqU2qbpjubX",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3a049764-a229-460b-f7c7-f52294b384db"
      },
      "source": [
        "'beginning' in tokenizer.vocab # Right"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Cvzy-sMsjs5P",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "690f3d56-8240-44e4-ea73-7a8dadabd6b4"
      },
      "source": [
        "'begining' in tokenizer.vocab # Wrong"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WkqxG-675A1c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fdb39a91-4246-436c-d199-1db45af88ac3"
      },
      "source": [
        "'separate' in tokenizer.vocab # Right"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ziFBjur15GHg",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1eb10edb-ab02-402b-fae0-e8b17dc1e162"
      },
      "source": [
        "'seperate' in tokenizer.vocab # Wrong"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-KZHPgVsSgvz"
      },
      "source": [
        "What about contractions?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VvUru5j3jxm2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b768e9c6-0f8d-4d92-ff8d-2e8c959ef12e"
      },
      "source": [
        "\"can't\" in tokenizer.vocab"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wl_l2xKwj0_l",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "36fe1aa8-2db0-4642-be2e-0ae7e5c56ebb"
      },
      "source": [
        "\"cant\" in tokenizer.vocab"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RE4TqKOlykJD"
      },
      "source": [
        "### Start vs. Mid Subwords\n",
        "\n",
        "For single characters, there are both the individual character and the '##' version for every character. Is the same true of subwords?"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_Xf9FH7HWZya",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "22c1d119-343f-4cd8-99f7-650895d117d5"
      },
      "source": [
        "# For each token in the vocabulary...\n",
        "for token in tokenizer.vocab.keys():\n",
        "    \n",
        "    # If it's a subword...\n",
        "    if len(token) >= 2 and token[0:2] == '##':\n",
        "        if not token[2:] in tokenizer.vocab:\n",
        "            print('Did not find a token for', token[2:])\n",
        "            break"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Did not find a token for ly\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0rPGb9z8SmYn",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "507523b9-f516-474f-91cb-a29e7d012d32"
      },
      "source": [
        "'##ly' in tokenizer.vocab"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wUmQwKqfWx9K",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "83ea815e-ada1-478e-9971-0864631b9802"
      },
      "source": [
        "'ly' in tokenizer.vocab"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "False"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PRaaspCfyqsY"
      },
      "source": [
        "### Names\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s7OLeBs_zBiR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b14d5696-c528-4a6c-e213-3fa9b784a98a"
      },
      "source": [
        "!pip install wget"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Collecting wget\n",
            "  Downloading https://files.pythonhosted.org/packages/47/6a/62e288da7bcda82b935ff0c6cfe542970f04e29c756b0e147251b2fb251f/wget-3.2.zip\n",
            "Building wheels for collected packages: wget\n",
            "  Building wheel for wget (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for wget: filename=wget-3.2-cp36-none-any.whl size=9682 sha256=32c7010387cfb7331c7cbda9dd73edc7beb48843b540b2e7380ab1b4a83330a6\n",
            "  Stored in directory: /root/.cache/pip/wheels/40/15/30/7d8f7cea2902b4db79e3fea550d7d7b85ecb27ef992b618f3f\n",
            "Successfully built wget\n",
            "Installing collected packages: wget\n",
            "Successfully installed wget-3.2\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9Mac6NiYys4g",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 56
        },
        "outputId": "0966adca-8869-4579-83af-bc73d00508ee"
      },
      "source": [
        "import wget\n",
        "import random \n",
        "\n",
        "print('Beginning file download with wget module')\n",
        "\n",
        "url = 'http://www.gutenberg.org/files/3201/files/NAMES.TXT'\n",
        "wget.download(url, 'first-names.txt')\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Beginning file download with wget module\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'first-names.txt'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FHF1JgrqY7Zi",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "987de51e-dd7c-47e1-b614-bc35f3c0deb9"
      },
      "source": [
        "# Read them in.\n",
        "with open('first-names.txt', 'rb') as f:\n",
        "    names_encoded = f.readlines()\n",
        "\n",
        "names = []\n",
        "\n",
        "# Decode the names, convert to lowercase, and strip newlines.\n",
        "for name in names_encoded:\n",
        "    try:\n",
        "        names.append(name.rstrip().lower().decode('utf-8'))\n",
        "    except:\n",
        "        continue\n",
        "\n",
        "print('Number of names: {:,}'.format(len(names)))\n",
        "print('Example:', random.choice(names))\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of names: 21,985\n",
            "Example: gerdy\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6heL05KQzmCa",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ee5c06f3-3440-479a-ef89-c5f74faef7a4"
      },
      "source": [
        "num_names = 0\n",
        "\n",
        "# For each name in our list...\n",
        "for name in names:\n",
        "\n",
        "    # If it's in the vocab...\n",
        "    if name in tokenizer.vocab:\n",
        "        # Tally it.\n",
        "        num_names += 1\n",
        "\n",
        "print('{:,} names in the vocabulary'.format(num_names))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "3,869 names in the vocabulary\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BM99iB36RAxX"
      },
      "source": [
        "**Further Research**\n",
        "* Add more modern names\n",
        "    * This repo / file contains some more modern names. The file download isn't working, though.\n",
        "    * `https://raw.githubusercontent.com/arineng/arincli/master/lib/male-first-names.txt`\n",
        "* Add common names from other languages.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3HGvwsy0ynkq"
      },
      "source": [
        "### Numbers"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h5T_-gfQZKm6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ac5032d0-884c-4def-8832-1ef9736c53ec"
      },
      "source": [
        "# Count how many numbers are in the vocabulary.\n",
        "count = 0\n",
        "\n",
        "# For each token in the vocabulary...\n",
        "for token in tokenizer.vocab.keys():\n",
        "\n",
        "    # Tally if it's a number.\n",
        "    if token.isdigit():\n",
        "        count += 1\n",
        "        \n",
        "        # Any numbers >= 10,000?\n",
        "        if len(token) > 4:\n",
        "            print(token)\n",
        "\n",
        "print('Vocab includes {:,} numbers.'.format(count))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Vocab includes 881 numbers.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-M1biDEVYjaL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "2204922a-a841-4cf1-82a7-449a6c14a61d"
      },
      "source": [
        "# Count how many dates between 1600 and 2021 are included.\n",
        "count = 0 \n",
        "for i in range(1600, 2021):\n",
        "    if str(i) in tokenizer.vocab:\n",
        "        count += 1\n",
        "\n",
        "print('Vocab includes {:,} of 421 dates from 1600 - 2021'.format(count))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Vocab includes 384 of 421 dates from 1600 - 2021\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}