CJCU_talk.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/howard-haowen/NLP-demos/blob/main/CJCU_talk.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QEuBPlA2bXZn"
      },
      "outputs": [],
      "source": [
        "from traitlets.config.manager import BaseJSONConfigManager\n",
        "from pathlib import Path"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MJFMGDwcba-I",
        "outputId": "26117279-9541-4302-8d2a-72ae49f3f0b5"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'autolaunch': False,\n",
              " 'enable_chalkboard': True,\n",
              " 'scroll': True,\n",
              " 'slideNumber': True,\n",
              " 'controls': True,\n",
              " 'progress': True,\n",
              " 'history': True,\n",
              " 'center': True,\n",
              " 'width': '100%',\n",
              " 'height': '100%',\n",
              " 'theme': 'beige',\n",
              " 'transition': 'concave',\n",
              " 'start_slideshow_at': 'selected'}"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "path = Path.home() / \".jupyter\" / \"nbconfig\"\n",
        "cm = BaseJSONConfigManager(config_dir=str(path))\n",
        "cm.update(\n",
        "    \"rise\",\n",
        "    {\n",
        "        \"autolaunch\": False,\n",
        "        \"enable_chalkboard\": True,\n",
        "        \"scroll\": True,\n",
        "        \"slideNumber\": True,\n",
        "        \"controls\": True,\n",
        "        \"progress\": True,\n",
        "        \"history\": True,\n",
        "        \"center\": True,\n",
        "        \"width\": \"100%\",\n",
        "        \"height\": \"100%\",\n",
        "        \"theme\": \"beige\",\n",
        "        \"transition\": \"concave\",\n",
        "        \"start_slideshow_at\": \"selected\"\n",
        "     }\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0s3kuqUKPZta"
      },
      "source": [
        "# 自我介紹"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lm5dm2XyaZ9c"
      },
      "source": [
        "## 教育背景\n",
        "![](https://raw.githubusercontent.com/howard-haowen/Chinese-NLP/main/img/education.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KxN1aUc4kr18"
      },
      "source": [
        "## 現職\n",
        "\n",
        "- 香港商慧科訊業台灣分公司AI工程師\n",
        "\n",
        "![](https://raw.githubusercontent.com/howard-haowen/Chinese-NLP/main/wisers_ai_lab.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hb4xGyyNslMd"
      },
      "source": [
        "# 機器學習基本流程\n",
        "\n",
        "![](https://2s7gjr373w3x22jf92z99mgm5w-wpengine.netdna-ssl.com/wp-content/uploads/2018/09/WD_3.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dieJ3Nq4eiSb"
      },
      "source": [
        "# 大綱\n",
        "- 基本NLP介紹\n",
        " * 商業應用\n",
        " * 開發工具\n",
        " * 基本語言分析\n",
        "- 文本表徵\n",
        " * 象徵式表徵(symoblic representations)\n",
        " * 分佈式表徵(distribut**ional** representations)\n",
        " * 分散式表徵(distribut**ed** representations)\n",
        "\n",
        "-  遷移學習\n",
        " * 領域遷移(domain adaptation)\n",
        " * 語言遷移(cross-lingual learning) \n",
        " * 任務遷移(multi-task learning)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3a3ip6WWSHsA"
      },
      "source": [
        "## 參考書籍\n",
        "\n",
        "<figure>\n",
        " <img src=\"https://i.gr-assets.com/images/S/compressed.photo.goodreads.com/books/1630086235l/58870327._SX318_.jpg\" width:50%>\n",
        "  <img src=\"https://i.gr-assets.com/images/S/compressed.photo.goodreads.com/books/1628096299l/55711023._SX318_.jpg\" width:50%>\n",
        "</figure>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YgmpViOuK4Z4"
      },
      "source": [
        "# 常見NLP商業應用\n",
        "\n",
        "![](https://camo.githubusercontent.com/7eb96ba311f989fbac5257f412a7f454f695015e7ce52d29ac24e10c8f34ef9e/68747470733a2f2f7777772e63796269616e742e636f6d2f77702d636f6e74656e742f75706c6f6164732f323032302f30312f434b432d4e61747572616c2d4c616e67756167652d50726f63657373696e672e706e67)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a73sgPLZhg7i"
      },
      "source": [
        "## 全球NLP營收預估\n",
        "\n",
        "![](https://github.com/howard-haowen/Chinese-NLP/raw/main/img/nlp_revenues.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aBB0Kc10LFxq"
      },
      "source": [
        "## 模型能力展示：[Ｗisers AI Lab](https://www.wisers.ai/zh-hk/browse/single-article-analysis/)\n",
        "\n",
        "- 命名實體識別\n",
        "- 情感分析\n",
        "- 話題分類\n",
        "- 關係抽取\n",
        "- 評論歸納\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U2q0HQipNwyy"
      },
      "source": [
        "# NLP開發工具"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Us8FwcHEwJuX"
      },
      "source": [
        "## 基本套件\n",
        "![](https://lh4.googleusercontent.com/5WioE0xqmDCRydcfrYXaTDUOewQzQXbFmehGVTk5IeLiN3uTeDuCtPrPhedE-3X8ghElL0Z2McxganYCvGWhufZR9xEgKkAQL1pyC1d5E3CnY7rH_c7iA2XKBzpkagnKK5TBf54c)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_GaxajzGlass"
      },
      "source": [
        "## 進階套件\n",
        "\n",
        " * [HuggingFace](https://huggingface.co/models) transformers\n",
        " * PyTorch\n",
        " * TensorFlow\n",
        "\n",
        "![](https://venturebeat.com/wp-content/uploads/2019/09/hugging-face.png?w=1200&strip=all)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U8ud-0_KxBWz"
      },
      "source": [
        "# 基本語言分析"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y9YaJKatJ12e"
      },
      "source": [
        "## 斷詞"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "6KLrYo3k8bH-",
        "outputId": "9c3ca37b-c227-4044-bc56-be85134d27ee"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: pip in /usr/local/lib/python3.7/dist-packages (21.3.1)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (59.5.0)\n",
            "Requirement already satisfied: wheel in /usr/local/lib/python3.7/dist-packages (0.37.0)\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
            "Requirement already satisfied: spacy in /usr/local/lib/python3.7/dist-packages (2.3.7)\n",
            "Collecting spacy\n",
            "  Using cached spacy-3.2.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.0 MB)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy) (59.5.0)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (4.62.3)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (2.23.0)\n",
            "Collecting catalogue<2.1.0,>=2.0.6\n",
            "  Using cached catalogue-2.0.6-py3-none-any.whl (17 kB)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy) (2.0.6)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (1.19.5)\n",
            "Requirement already satisfied: typer<0.5.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (0.4.0)\n",
            "Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy) (3.10.0.2)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (1.0.6)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /usr/local/lib/python3.7/dist-packages (from spacy) (3.0.8)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (0.4.1)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy) (3.0.6)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (3.3.0)\n",
            "Collecting thinc<8.1.0,>=8.0.12\n",
            "  Using cached thinc-8.0.13-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (628 kB)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (1.0.1)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy) (1.8.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy) (2.11.3)\n",
            "Collecting srsly<3.0.0,>=2.4.1\n",
            "  Using cached srsly-2.4.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (451 kB)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (21.3)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from spacy) (0.8.2)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy) (0.6.1)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy) (3.6.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->spacy) (3.0.6)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy) (5.2.1)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy) (2021.10.8)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy) (1.24.3)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy) (3.0.4)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer<0.5.0,>=0.3.0->spacy) (7.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy) (2.0.1)\n",
            "Installing collected packages: catalogue, srsly, thinc, spacy\n",
            "  Attempting uninstall: catalogue\n",
            "    Found existing installation: catalogue 1.0.0\n",
            "    Uninstalling catalogue-1.0.0:\n",
            "      Successfully uninstalled catalogue-1.0.0\n",
            "  Attempting uninstall: srsly\n",
            "    Found existing installation: srsly 1.0.5\n",
            "    Uninstalling srsly-1.0.5:\n",
            "      Successfully uninstalled srsly-1.0.5\n",
            "  Attempting uninstall: thinc\n",
            "    Found existing installation: thinc 7.4.5\n",
            "    Uninstalling thinc-7.4.5:\n",
            "      Successfully uninstalled thinc-7.4.5\n",
            "  Attempting uninstall: spacy\n",
            "    Found existing installation: spacy 2.3.7\n",
            "    Uninstalling spacy-2.3.7:\n",
            "      Successfully uninstalled spacy-2.3.7\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "texthero 1.1.0 requires spacy<3.0.0, but you have spacy 3.2.0 which is incompatible.\u001b[0m\n",
            "Successfully installed catalogue-2.0.6 spacy-3.2.0 srsly-2.4.2 thinc-8.0.13\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n"
          ]
        },
        {
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "catalogue",
                  "spacy",
                  "srsly",
                  "thinc"
                ]
              }
            }
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting zh-core-web-md==3.2.0\n",
            "  Downloading https://github.com/explosion/spacy-models/releases/download/zh_core_web_md-3.2.0/zh_core_web_md-3.2.0-py3-none-any.whl (79.0 MB)\n",
            "     |████████████████████████████████| 79.0 MB 53 kB/s              \n",
            "\u001b[?25hRequirement already satisfied: spacy<3.3.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from zh-core-web-md==3.2.0) (3.2.0)\n",
            "Requirement already satisfied: spacy-pkuseg<0.1.0,>=0.0.27 in /usr/local/lib/python3.7/dist-packages (from zh-core-web-md==3.2.0) (0.0.28)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.3.0)\n",
            "Requirement already satisfied: srsly<3.0.0,>=2.4.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.4.2)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.8.2)\n",
            "Requirement already satisfied: typer<0.5.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (0.4.0)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.0.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (21.3)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.0.8)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (0.6.1)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.23.0)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (4.62.3)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (59.5.0)\n",
            "Requirement already satisfied: thinc<8.1.0,>=8.0.12 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (8.0.13)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.0.6)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.0.6)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (0.4.1)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (0.8.2)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.19.5)\n",
            "Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.10.0.2)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.0.6)\n",
            "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.0.6)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.11.3)\n",
            "Requirement already satisfied: cython>=0.25 in /usr/local/lib/python3.7/dist-packages (from spacy-pkuseg<0.1.0,>=0.0.27->zh-core-web-md==3.2.0) (0.29.24)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.6.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.0.6)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (5.2.1)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.24.3)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.0.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2021.10.8)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.10)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer<0.5.0,>=0.3.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (7.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.0.1)\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
            "\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
            "You can now load the package via spacy.load('zh_core_web_md')\n"
          ]
        }
      ],
      "source": [
        "!pip install -U pip setuptools wheel\n",
        "!pip install -U spacy\n",
        "!python -m spacy download zh_core_web_md"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PuXjrsVniTAf",
        "outputId": "5f8317d2-d196-47cd-e43d-812d25e747c0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Name: spacy\n",
            "Version: 3.2.0\n",
            "Summary: Industrial-strength Natural Language Processing (NLP) in Python\n",
            "Home-page: https://spacy.io\n",
            "Author: Explosion\n",
            "Author-email: contact@explosion.ai\n",
            "License: MIT\n",
            "Location: /usr/local/lib/python3.7/dist-packages\n",
            "Requires: blis, catalogue, cymem, jinja2, langcodes, murmurhash, numpy, packaging, pathy, preshed, pydantic, requests, setuptools, spacy-legacy, spacy-loggers, srsly, thinc, tqdm, typer, typing-extensions, wasabi\n",
            "Required-by: en-core-web-sm, fastai, texthero, zh-core-web-md\n"
          ]
        }
      ],
      "source": [
        "!pip show spacy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ztHzKCBAfyru",
        "outputId": "ce35d535-1d76-4c7e-d557-2fcb03ac3f26"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['tok2vec', 'tagger', 'parser', 'attribute_ruler', 'ner']"
            ]
          },
          "execution_count": 143,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import spacy\n",
        "\n",
        "nlp = spacy.load(\"zh_core_web_md\")\n",
        "nlp.pipe_names"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aLxNcrXGO7SP"
      },
      "source": [
        "### spaCy的模組化設計\n",
        "![](https://spacy.io/pipeline-fde48da9b43661abcdf62ab70a546d71.svg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MzOcpL4yPvF6"
      },
      "source": [
        "### spaCy的內建斷詞模型"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "G-Z040z2J9-R",
        "outputId": "4059e3db-0928-47a2-d9df-fbbb1536105b"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'送人 | 的 | ， | 不 | 知道 | 好不 | 好 | 。 | 聽說 | 紅心 | 火龍 | 果還 | 是 | 比 | 較甜 | 的 | 。'"
            ]
          },
          "execution_count": 144,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "text = \"送人的，不知道好不好。聽說紅心火龍果還是比較甜的。\"\n",
        "doc = nlp(text)\n",
        "tokens = [tok.text for tok in doc]\n",
        "\" | \".join(tokens)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "JU7p1GVvxSFB",
        "outputId": "29cc4996-af45-4f4d-9303-303f9d2ec1b9"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'送人 | 的 | ， | 不 | 知道 | 好不 | 好 | 。 | 听说 | 红心火 | 龙果 | 还是 | 比较 | 甜 | 的 | 。'"
            ]
          },
          "execution_count": 145,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "text = \"送人的，不知道好不好。听说红心火龙果还是比较甜的。\"\n",
        "doc = nlp(text)\n",
        "tokens = [tok.text for tok in doc]\n",
        "\" | \".join(tokens)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xvUPwUo8Ptad"
      },
      "source": [
        "### 繁體中文版jieba"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZaY5RPFpT8Fq",
        "outputId": "c518e2ba-f156-44ef-a441-c7e04160313d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Cloning into 'jieba_tw'...\n",
            "warning: --local is ignored\n",
            "remote: Enumerating objects: 2320, done.\u001b[K\n",
            "remote: Total 2320 (delta 0), reused 0 (delta 0), pack-reused 2320\u001b[K\n",
            "Receiving objects: 100% (2320/2320), 36.07 MiB | 12.46 MiB/s, done.\n",
            "Resolving deltas: 100% (1498/1498), done.\n",
            "/content/jieba_tw/jieba_tw/jieba_tw\n",
            "Changelog  jieba  LICENSE  MANIFEST.in\tREADME.md\n"
          ]
        }
      ],
      "source": [
        "!git clone -l -s https://github.com/L706077/jieba-zh_TW.git jieba_tw\n",
        "%cd jieba_tw\n",
        "!ls"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ACh_KTpsUmhC"
      },
      "outputs": [],
      "source": [
        "import jieba"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "UVAhkqRyVGSr",
        "outputId": "697a6f1e-e50f-4884-f388-2ad1bfb9467b"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'送人 | 的 | ， | 不 | 知道 | 好不好 | 。 | 聽說 | 紅心 | 火龍果 | 還是 | 比較 | 甜 | 的 | 。'"
            ]
          },
          "execution_count": 148,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "text = \"送人的，不知道好不好。聽說紅心火龍果還是比較甜的。\"\n",
        "tokens = jieba.cut(text) \n",
        "\" | \".join(tokens)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "VnQCbHsxKini",
        "outputId": "c7f98f1b-194e-4a5f-ad02-b42c159737e0"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'送人 | 的 | ， | 不 | 知道 | 好不好 | 。 | 听 | 说 | 红 | 心火 | 龙 | 果 | 还 | 是 | 比 | 较 | 甜 | 的 | 。'"
            ]
          },
          "execution_count": 149,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "text = \"送人的，不知道好不好。听说红心火龙果还是比较甜的。\"\n",
        "tokens = jieba.cut(text) \n",
        "\" | \".join(tokens)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-W4sPfcZYMCq"
      },
      "source": [
        "## 命名實體"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BREfnaQcaQ3W"
      },
      "outputs": [],
      "source": [
        "from spacy import displacy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 157
        },
        "id": "KozpzTVNYLnb",
        "outputId": "491d9b9c-c2a5-45dd-c07e-b35c38fe3894"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<span class=\"tex2jax_ignore\"><div class=\"entities\" style=\"line-height: 2.5; direction: ltr\"></br>\n",
              "<mark class=\"entity\" style=\"background: #7aecec; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    長榮大學\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; vertical-align: middle; margin-left: 0.5rem\">ORG</span>\n",
              "</mark>\n",
              "\n",
              "<mark class=\"entity\" style=\"background: #bfe1d9; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    今天\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; vertical-align: middle; margin-left: 0.5rem\">DATE</span>\n",
              "</mark>\n",
              "與勞動部\n",
              "<mark class=\"entity\" style=\"background: #feca74; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    台南\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; vertical-align: middle; margin-left: 0.5rem\">GPE</span>\n",
              "</mark>\n",
              "就業中心舉辦「勇往職前･迎薪未來」校園徵才；</br>包括電腦資訊、餐飲旅店、\n",
              "<mark class=\"entity\" style=\"background: #feca74; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    文創\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; vertical-align: middle; margin-left: 0.5rem\">GPE</span>\n",
              "</mark>\n",
              "、休閒運動、零件製造、醫療器材、</br>服飾用品等共\n",
              "<mark class=\"entity\" style=\"background: #e4e7d2; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    34家\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; vertical-align: middle; margin-left: 0.5rem\">CARDINAL</span>\n",
              "</mark>\n",
              "在地廠商釋出含工讀與全職共\n",
              "<mark class=\"entity\" style=\"background: #e4e7d2; padding: 0.45em 0.6em; margin: 0 0.25em; line-height: 1; border-radius: 0.35em;\">\n",
              "    923\n",
              "    <span style=\"font-size: 0.8em; font-weight: bold; line-height: 1; border-radius: 0.35em; vertical-align: middle; margin-left: 0.5rem\">CARDINAL</span>\n",
              "</mark>\n",
              "職缺</br></div></span>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "text = \"\"\"\n",
        "長榮大學今天與勞動部台南就業中心舉辦「勇往職前･迎薪未來」校園徵才；\n",
        "包括電腦資訊、餐飲旅店、文創、休閒運動、零件製造、醫療器材、\n",
        "服飾用品等共34家在地廠商釋出含工讀與全職共923職缺\n",
        "\"\"\"\n",
        "doc = nlp(text)\n",
        "displacy.render(doc, style='ent',jupyter=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "veULoMBuyvmw"
      },
      "source": [
        "## 依存句法"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 373
        },
        "id": "U32pNPApXs4C",
        "outputId": "c8badc9f-3f8c-4f0e-835f-eb381f083102"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<span class=\"tex2jax_ignore\"><svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xml:lang=\"zh\" id=\"1c767119b84043e8b0cba97989606d2a-0\" class=\"displacy\" width=\"960\" height=\"332.0\" direction=\"ltr\" style=\"max-width: none; height: 332.0px; color: #000000; background: #ffffff; font-family: Arial; direction: ltr\">\n",
              "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"242.0\">\n",
              "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"50\">我</tspan>\n",
              "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"50\">PRON</tspan>\n",
              "</text>\n",
              "\n",
              "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"242.0\">\n",
              "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"180\">想要</tspan>\n",
              "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"180\">VERB</tspan>\n",
              "</text>\n",
              "\n",
              "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"242.0\">\n",
              "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"310\">一</tspan>\n",
              "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"310\">NUM</tspan>\n",
              "</text>\n",
              "\n",
              "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"242.0\">\n",
              "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"440\">份</tspan>\n",
              "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"440\">NUM</tspan>\n",
              "</text>\n",
              "\n",
              "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"242.0\">\n",
              "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"570\">2</tspan>\n",
              "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"570\">NUM</tspan>\n",
              "</text>\n",
              "\n",
              "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"242.0\">\n",
              "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"700\">號</tspan>\n",
              "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"700\">NUM</tspan>\n",
              "</text>\n",
              "\n",
              "<text class=\"displacy-token\" fill=\"currentColor\" text-anchor=\"middle\" y=\"242.0\">\n",
              "    <tspan class=\"displacy-word\" fill=\"currentColor\" x=\"830\">餐</tspan>\n",
              "    <tspan class=\"displacy-tag\" dy=\"2em\" fill=\"currentColor\" x=\"830\">NOUN</tspan>\n",
              "</text>\n",
              "\n",
              "<g class=\"displacy-arrow\">\n",
              "    <path class=\"displacy-arc\" id=\"arrow-1c767119b84043e8b0cba97989606d2a-0-0\" stroke-width=\"2px\" d=\"M70,197.0 C70,132.0 170.0,132.0 170.0,197.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
              "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
              "        <textPath xlink:href=\"#arrow-1c767119b84043e8b0cba97989606d2a-0-0\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">nsubj</textPath>\n",
              "    </text>\n",
              "    <path class=\"displacy-arrowhead\" d=\"M70,199.0 L62,187.0 78,187.0\" fill=\"currentColor\"/>\n",
              "</g>\n",
              "\n",
              "<g class=\"displacy-arrow\">\n",
              "    <path class=\"displacy-arc\" id=\"arrow-1c767119b84043e8b0cba97989606d2a-0-1\" stroke-width=\"2px\" d=\"M330,197.0 C330,67.0 825.0,67.0 825.0,197.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
              "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
              "        <textPath xlink:href=\"#arrow-1c767119b84043e8b0cba97989606d2a-0-1\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">nummod</textPath>\n",
              "    </text>\n",
              "    <path class=\"displacy-arrowhead\" d=\"M330,199.0 L322,187.0 338,187.0\" fill=\"currentColor\"/>\n",
              "</g>\n",
              "\n",
              "<g class=\"displacy-arrow\">\n",
              "    <path class=\"displacy-arc\" id=\"arrow-1c767119b84043e8b0cba97989606d2a-0-2\" stroke-width=\"2px\" d=\"M330,197.0 C330,132.0 430.0,132.0 430.0,197.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
              "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
              "        <textPath xlink:href=\"#arrow-1c767119b84043e8b0cba97989606d2a-0-2\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">mark:clf</textPath>\n",
              "    </text>\n",
              "    <path class=\"displacy-arrowhead\" d=\"M430.0,199.0 L438.0,187.0 422.0,187.0\" fill=\"currentColor\"/>\n",
              "</g>\n",
              "\n",
              "<g class=\"displacy-arrow\">\n",
              "    <path class=\"displacy-arc\" id=\"arrow-1c767119b84043e8b0cba97989606d2a-0-3\" stroke-width=\"2px\" d=\"M590,197.0 C590,132.0 690.0,132.0 690.0,197.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
              "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
              "        <textPath xlink:href=\"#arrow-1c767119b84043e8b0cba97989606d2a-0-3\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">dep</textPath>\n",
              "    </text>\n",
              "    <path class=\"displacy-arrowhead\" d=\"M590,199.0 L582,187.0 598,187.0\" fill=\"currentColor\"/>\n",
              "</g>\n",
              "\n",
              "<g class=\"displacy-arrow\">\n",
              "    <path class=\"displacy-arc\" id=\"arrow-1c767119b84043e8b0cba97989606d2a-0-4\" stroke-width=\"2px\" d=\"M720,197.0 C720,132.0 820.0,132.0 820.0,197.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
              "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
              "        <textPath xlink:href=\"#arrow-1c767119b84043e8b0cba97989606d2a-0-4\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">compound:nn</textPath>\n",
              "    </text>\n",
              "    <path class=\"displacy-arrowhead\" d=\"M720,199.0 L712,187.0 728,187.0\" fill=\"currentColor\"/>\n",
              "</g>\n",
              "\n",
              "<g class=\"displacy-arrow\">\n",
              "    <path class=\"displacy-arc\" id=\"arrow-1c767119b84043e8b0cba97989606d2a-0-5\" stroke-width=\"2px\" d=\"M200,197.0 C200,2.0 830.0,2.0 830.0,197.0\" fill=\"none\" stroke=\"currentColor\"/>\n",
              "    <text dy=\"1.25em\" style=\"font-size: 0.8em; letter-spacing: 1px\">\n",
              "        <textPath xlink:href=\"#arrow-1c767119b84043e8b0cba97989606d2a-0-5\" class=\"displacy-label\" startOffset=\"50%\" side=\"left\" fill=\"currentColor\" text-anchor=\"middle\">dobj</textPath>\n",
              "    </text>\n",
              "    <path class=\"displacy-arrowhead\" d=\"M830.0,199.0 L838.0,187.0 822.0,187.0\" fill=\"currentColor\"/>\n",
              "</g>\n",
              "</svg></span>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "text = \"我想要一份2號餐\"\n",
        "doc = nlp(text)\n",
        "displacy.render(doc, style='dep',jupyter=True, options={'distance':130})"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eqLB6wQZJNdj"
      },
      "source": [
        "# 商品評論資料集 "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uJ6IRZUcJx5H"
      },
      "source": [
        "## 載入資料集"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NVU87c0tKxPf"
      },
      "outputs": [],
      "source": [
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "54llyAjuJbi3",
        "outputId": "ced4346b-db28-4cee-adc6-6d01612e5655"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>cat</th>\n",
              "      <th>label</th>\n",
              "      <th>review</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>47428</th>\n",
              "      <td>酒店</td>\n",
              "      <td>0</td>\n",
              "      <td>前臺的服務員好冷淡！對客人也是愛理不理的！衛生不錯，房間隔音不好</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23339</th>\n",
              "      <td>洗髮水</td>\n",
              "      <td>1</td>\n",
              "      <td>還送了一罐發膜，廣告不錯，試試效果怎樣再說</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13019</th>\n",
              "      <td>水果</td>\n",
              "      <td>1</td>\n",
              "      <td>好吃，買了好多次了，又翠又甜，贊一個！希望一直保持下去！</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44464</th>\n",
              "      <td>酒店</td>\n",
              "      <td>1</td>\n",
              "      <td>好於預期。如果服務更好一店，價格更加合適一點的話，會更加有競爭力。</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7912</th>\n",
              "      <td>平板</td>\n",
              "      <td>0</td>\n",
              "      <td>SIM卡 插入沒反應 建議別買上當受騙</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       cat  label                             review\n",
              "47428   酒店      0   前臺的服務員好冷淡！對客人也是愛理不理的！衛生不錯，房間隔音不好\n",
              "23339  洗髮水      1              還送了一罐發膜，廣告不錯，試試效果怎樣再說\n",
              "13019   水果      1       好吃，買了好多次了，又翠又甜，贊一個！希望一直保持下去！\n",
              "44464   酒店      1  好於預期。如果服務更好一店，價格更加合適一點的話，會更加有競爭力。\n",
              "7912    平板      0                SIM卡 插入沒反應 建議別買上當受騙"
            ]
          },
          "execution_count": 154,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "data_path = \"https://raw.githubusercontent.com/howard-haowen/Chinese-NLP/main/online_shopping_5_cats_tra.csv\"\n",
        "df = pd.read_csv(data_path)\n",
        "df.sample(5, random_state=900)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nnfwJ0aPJipM",
        "outputId": "6a573693-b16b-4c97-9511-a8dc717918e9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "水果     10000\n",
              "衣服     10000\n",
              "洗髮水    10000\n",
              "酒店     10000\n",
              "平板     10000\n",
              "Name: cat, dtype: int64"
            ]
          },
          "execution_count": 155,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df['cat'].value_counts()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HOxJ2lgHJjqq",
        "outputId": "3d5f4b80-212c-4356-e6b9-7206ddb9b0f9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "1    25000\n",
              "0    25000\n",
              "Name: label, dtype: int64"
            ]
          },
          "execution_count": 156,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "df['label'].value_counts()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G8DufoY5QH3J"
      },
      "source": [
        "## 文本預處理"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "l7SoykhOUotP",
        "outputId": "10635afe-a957-43b8-ef15-c127e3168440"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'褲子質量挺不錯的，料子很舒服，穿起來棒棒噠！顯得腿型特別好，也不褪色，不錯哦(-ω-`)建議大家購買'"
            ]
          },
          "execution_count": 157,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "text = df.at[31902, 'review']\n",
        "text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "fckruMO7Q1Ld",
        "outputId": "11e74c95-b201-4a51-dbbe-9805946763b7"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>cat</th>\n",
              "      <th>label</th>\n",
              "      <th>review</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>9701</th>\n",
              "      <td>平板</td>\n",
              "      <td>0</td>\n",
              "      <td>不好，什麼都沒有，裸機，而且總出故障！差差差！</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1304</th>\n",
              "      <td>平板</td>\n",
              "      <td>1</td>\n",
              "      <td>很喜歡，主要是不卡，拿出門也很方便</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49735</th>\n",
              "      <td>酒店</td>\n",
              "      <td>0</td>\n",
              "      <td>標準間的衛生間居然熱水都上不來，後來叫來了服務生和經理折騰了半個小時才弄好。。。 第一次住這...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>33756</th>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "      <td>寶貝收到了，質量很好，摸起來很舒服，很不錯的一條褲子哦，值得購買!</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31171</th>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "      <td>褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      cat  label                                             review\n",
              "9701   平板      0                            不好，什麼都沒有，裸機，而且總出故障！差差差！\n",
              "1304   平板      1                                  很喜歡，主要是不卡，拿出門也很方便\n",
              "49735  酒店      0  標準間的衛生間居然熱水都上不來，後來叫來了服務生和經理折騰了半個小時才弄好。。。 第一次住這...\n",
              "33756  衣服      1                  寶貝收到了，質量很好，摸起來很舒服，很不錯的一條褲子哦，值得購買!\n",
              "31171  衣服      1                             褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡"
            ]
          },
          "execution_count": 158,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "downsized_df = df.sample(5000, random_state=100)\n",
        "downsized_df.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k6NcmmDMRMRv",
        "outputId": "a787752e-efb5-4b35-eae9-ca90e4bf856b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "水果     1020\n",
              "平板     1015\n",
              "衣服     1000\n",
              "酒店      995\n",
              "洗髮水     970\n",
              "Name: cat, dtype: int64"
            ]
          },
          "execution_count": 159,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "downsized_df['cat'].value_counts()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "owdSAvEaRQGz",
        "outputId": "001f8307-aaf8-4090-f37a-4a786cf36c4e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0    2521\n",
              "1    2479\n",
              "Name: label, dtype: int64"
            ]
          },
          "execution_count": 160,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "downsized_df['label'].value_counts()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "JMQIRyBcRy76",
        "outputId": "940d7553-18c4-40b9-bb2b-059abeeea1e3"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>cat</th>\n",
              "      <th>label</th>\n",
              "      <th>review</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>4995</th>\n",
              "      <td>平板</td>\n",
              "      <td>0</td>\n",
              "      <td>不好，什麼都沒有，裸機，而且總出故障！差差差！</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4996</th>\n",
              "      <td>平板</td>\n",
              "      <td>1</td>\n",
              "      <td>很喜歡，主要是不卡，拿出門也很方便</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4997</th>\n",
              "      <td>酒店</td>\n",
              "      <td>0</td>\n",
              "      <td>標準間的衛生間居然熱水都上不來，後來叫來了服務生和經理折騰了半個小時才弄好。。。 第一次住這...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4998</th>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "      <td>寶貝收到了，質量很好，摸起來很舒服，很不錯的一條褲子哦，值得購買!</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4999</th>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "      <td>褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     cat  label                                             review\n",
              "4995  平板      0                            不好，什麼都沒有，裸機，而且總出故障！差差差！\n",
              "4996  平板      1                                  很喜歡，主要是不卡，拿出門也很方便\n",
              "4997  酒店      0  標準間的衛生間居然熱水都上不來，後來叫來了服務生和經理折騰了半個小時才弄好。。。 第一次住這...\n",
              "4998  衣服      1                  寶貝收到了，質量很好，摸起來很舒服，很不錯的一條褲子哦，值得購買!\n",
              "4999  衣服      1                             褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡"
            ]
          },
          "execution_count": 161,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "downsized_df.reset_index(drop=True, inplace=True)\n",
        "downsized_df.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ql3deDTDMDE1"
      },
      "outputs": [],
      "source": [
        "from spacy.tokens import Doc\n",
        "\n",
        "class WhitespaceTokenizer:\n",
        "    def __init__(self, vocab):\n",
        "        self.vocab = vocab\n",
        "\n",
        "    def __call__(self, text):\n",
        "        words = text.strip().split()\n",
        "        return Doc(self.vocab, words=words)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "M7DHrR3jSu3J"
      },
      "outputs": [],
      "source": [
        "nlp = spacy.load(\"zh_core_web_md\", disable=[\"tagger\", \"parser\", \"ner\"])\n",
        "nlp.tokenizer = WhitespaceTokenizer(nlp.vocab)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NltTjnFAMekf"
      },
      "outputs": [],
      "source": [
        "def whitespace_cut(raw_text: str) -> str:\n",
        "    tokens = jieba.cut(raw_text)\n",
        "    results = \" \".join(tokens)\n",
        "    return results"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QhLoytocMUc_"
      },
      "outputs": [],
      "source": [
        "def preprocess_text(raw_text: str) -> str:\n",
        "    raw_text_with_space = whitespace_cut(raw_text)\n",
        "    doc = nlp.make_doc(raw_text_with_space)\n",
        "    res = [tok for tok in doc if not tok.is_punct]\n",
        "    res = [tok for tok in res if not tok.is_stop]\n",
        "    res = [tok for tok in res if not tok.like_email]\n",
        "    res = [tok for tok in res if not tok.like_url]\n",
        "    res = [tok for tok in res if not tok.like_num]\n",
        "    res = [tok for tok in res if not tok.is_ascii]\n",
        "    res = [tok.text for tok in res if not tok.is_space]\n",
        "    res = \" \".join(res)\n",
        "    return res"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9fiaLikpWIix"
      },
      "outputs": [],
      "source": [
        "import time"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xgrN-GXsTAhS",
        "outputId": "f3f4f762-fc71-4b11-ff45-9209eea69091"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時2.6363587379455566秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "downsized_df['tokens'] = downsized_df['review'].apply(preprocess_text)\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "id": "RaOtxEOATZ3t",
        "outputId": "73be1547-572a-48b4-f663-52afa5e0b3c1"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>cat</th>\n",
              "      <th>label</th>\n",
              "      <th>review</th>\n",
              "      <th>tokens</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>4995</th>\n",
              "      <td>平板</td>\n",
              "      <td>0</td>\n",
              "      <td>不好，什麼都沒有，裸機，而且總出故障！差差差！</td>\n",
              "      <td>什麼 沒有 裸機 總出 故障 差差 差</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4996</th>\n",
              "      <td>平板</td>\n",
              "      <td>1</td>\n",
              "      <td>很喜歡，主要是不卡，拿出門也很方便</td>\n",
              "      <td>喜歡 卡 出門</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4997</th>\n",
              "      <td>酒店</td>\n",
              "      <td>0</td>\n",
              "      <td>標準間的衛生間居然熱水都上不來，後來叫來了服務生和經理折騰了半個小時才弄好。。。 第一次住這...</td>\n",
              "      <td>標準 間 衛生 間 熱水 來 後來 叫來 服務生 經理 折騰 個 小時 弄好 次 住 這麼 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4998</th>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "      <td>寶貝收到了，質量很好，摸起來很舒服，很不錯的一條褲子哦，值得購買!</td>\n",
              "      <td>寶貝 收到 質量 摸起來 舒服 不錯 條 褲子 值得 購買</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4999</th>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "      <td>褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡</td>\n",
              "      <td>褲子 顏色 圖片 一樣 質量 尺碼 準 喜歡</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     cat  ...                                             tokens\n",
              "4995  平板  ...                                什麼 沒有 裸機 總出 故障 差差 差\n",
              "4996  平板  ...                                            喜歡 卡 出門\n",
              "4997  酒店  ...  標準 間 衛生 間 熱水 來 後來 叫來 服務生 經理 折騰 個 小時 弄好 次 住 這麼 ...\n",
              "4998  衣服  ...                      寶貝 收到 質量 摸起來 舒服 不錯 條 褲子 值得 購買\n",
              "4999  衣服  ...                             褲子 顏色 圖片 一樣 質量 尺碼 準 喜歡\n",
              "\n",
              "[5 rows x 4 columns]"
            ]
          },
          "execution_count": 168,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "downsized_df.tail()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UWyZMhzFUOrE"
      },
      "source": [
        "## 切割資料集"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8dbgL2UJUWm1"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "def train_validate_test_split(df, train_percent=.6, validate_percent=.2, seed=None):\n",
        "    np.random.seed(seed)\n",
        "    perm = np.random.permutation(df.index)\n",
        "    m = len(df.index)\n",
        "    train_end = int(train_percent * m)\n",
        "    validate_end = int(validate_percent * m) + train_end\n",
        "    train = df.iloc[perm[:train_end]]\n",
        "    validate = df.iloc[perm[train_end:validate_end]]\n",
        "    test = df.iloc[perm[validate_end:]]\n",
        "    return train, validate, test"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-B1AjhmpUZsx"
      },
      "outputs": [],
      "source": [
        "# train, validate, test = train_validate_test_split(downsized_df, seed=100)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XlpenGs1d19N"
      },
      "outputs": [],
      "source": [
        "from sklearn import feature_extraction"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ezJTwMaudzid"
      },
      "outputs": [],
      "source": [
        "train, test = model_selection.train_test_split(downsized_df, test_size=0.3, random_state=100)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DUYDJev6UdTB",
        "outputId": "5266ca0f-8e70-4819-eefd-01b1b041bd78"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "平板     712\n",
            "水果     712\n",
            "酒店     704\n",
            "衣服     695\n",
            "洗髮水    677\n",
            "Name: cat, dtype: int64\n",
            "0    1760\n",
            "1    1740\n",
            "Name: label, dtype: int64\n",
            "水果     308\n",
            "衣服     305\n",
            "平板     303\n",
            "洗髮水    293\n",
            "酒店     291\n",
            "Name: cat, dtype: int64\n",
            "0    761\n",
            "1    739\n",
            "Name: label, dtype: int64\n"
          ]
        }
      ],
      "source": [
        "print(train['cat'].value_counts())\n",
        "print(train['label'].value_counts())\n",
        "print(test['cat'].value_counts())\n",
        "print(test['label'].value_counts())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "w7MHZyDCJyWM"
      },
      "source": [
        "# 文本表徵"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jqbOq1ptgKmq"
      },
      "source": [
        " 詞頻: TF-IDF\n",
        " * 淺層神經網路下的靜態詞向量: fasttext\n",
        " * 深層神經網路下的動態向量: BERT (CNN + transformer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_j3urOZK-eDh"
      },
      "source": [
        "## 圖片向量化\n",
        "\n",
        "![](data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wCEAAkGBxMTEhUTExMWFhUXGBkZGBcYGBodGhoYIB8ZGhoXGh0bHSggHRolHxcfITEhJSkrLy4uGx80OTQsOCgtLysBCgoKBQUFDgUFDisZExkrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrK//AABEIAJ8BPAMBIgACEQEDEQH/xAAcAAAABwEBAAAAAAAAAAAAAAAAAwQFBgcIAgH/xABNEAABAwIDBQQGAwwHCAMBAAABAgMREiEABDEFEyJBUQYyYXEHFCNCgZEIUpIzRFNigoShscHD0/AVFyRyk9HSFjRDVGNzouElsvGj/8QAFAEBAAAAAAAAAAAAAAAAAAAAAP/EABQRAQAAAAAAAAAAAAAAAAAAAAD/2gAMAwEAAhEDEQA/ALxwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwMDAwAwU6uEkwTAJganwE88G4GAij3brKpSpQDiglKF2SIUlbLmZCkkqCY3bS5M6iMOea7R5ZtsuKdFIDhsFEndhZWkCLrAaWadeBVrHCcdkcrTRQqN28331dx0ysa8tE/VBIFseZrsjlXHC4tKiTX76o4w8lUdJGYWLRqPqpgFr+1gFNoQ066pYCoQEihFhWveKTAk90SrW1jHDnaPKgVHMNgdSbRBVV/cpSVVaQlRmAcHObLSVNrqWFtpoqCyCpPCSFjRV0gyRIvESZb/8AZLK0KbpVSposd42ZpWhLQPJKUuKjmKjgFae0OVJjfonp04lov04m1JvzTGuOV9ocuHN1vOKlSjAJACQySCQICiMw2QNTWOowlV2VaL7jqioocS3LUkCpDj71RM34n5AtBQNZt4ex+VMkhaiZkqWSTIy6bza3qjJHijxIILl7cY3briVFYZQVrCAVKACa6Y+sR7uuC8v2iYKEKWsNFaSsJWts8HGQoqbWpEKS2pQhVwlX1TB+T2U20lxLcp3hBUQYMhCWwUwITZA0EThu/wBjMpxSgmtK0LJUZUFqdWozqlVT6zKadR0EA55PajLqqW3AowZAm0KUghVuFQUhQpMGUqtY4cMNOT2G004lxINSUqQNBwqNREJA53jTnE4dsAMDAwMBnP8Ar62j+Ayn2Hf4uDWvTptAj7hlPsO/xcVIBjtKVcpwFpq9PG0QT7DKfYd/i4A9PO0fwGU+w7/FxX+Q2LmFqbKWHFBSkxCSapIjznFkdmOxr4f9rkVU0K7zNpkRqNcAT/XvtD8Dk/su/wAXHJ9PO0fwGU+w7/FxaPZ3sswEK3uSZBqtWwjSOUpw/t9mMhAnJ5Wf+w3/AKcBR39fW0fwGU+w7/FwP6+to/gMp9h3+Li5Ntdl8mWVbvJZeqRFLDc6idE4r/tL2QUS3usjyVNDA8ImE4CPJ9O+0fwGU+w7/FwFenfaP4DKfYd/i4S5vsZm6jTkHY5QyenliObR7KZxDSlKybyQIuWyALgax44CVf19bR/AZT7Dv8XA/r62j+Ayn2Hf4uKzzORcbitCkzMSImIn9eDGynd3iYPSeeA2k2qQD1AxXXpe7eZnZfq3q6GV77e1b1KzFG7imlafrmZnlixGe6PIYpH6TP3h+cfuMA3n0z7V3u59XydfSlyNKtd9GmAPTPtQtF4MZMoSYJodmeHlvp98X8cVglhG/p3xo/Cwfq+fW2uOcmwpaFpClVCKGwCa5PFA8AJ+HhgLMc9N21EoQssZSldVJodvTAP/ABupwoT6YtrVlv1fJ1imRS57xQlN9/Grif5GIb2Z7MOuwCy6qVQsblat2m0K0g1X0+qMWFszssttpIGR3hg8amSknvEC6ZgEJA/9YBrR6aNqKbU6GMmUJ1NDn4vLfT7w/Tgtfpv2mEJcLGTpUSAaHeWtt9i0W9hZcLCf6Ly5RxX3TXIGJSUe8Up+Ywd/QOWpV/8AF5SR3Rum4Pekzu5Fkp5c/jgKsV6aNqhxLXq+TrVEClzncX30Y4a9Ne1FVxl8n7MEq4HbAGD/AMa+J1tbs42XKm9mMi1gllECmYulMSQB8+l8QrNdi17t0jJrCpISkZc3EiIIT/MD4AX/AF27UoDm4ydBUUTQ73gAYjfToceq9NG1AptBy+TlwJKeFy4V3f8AjW+OGbafYvMJS1TlXjUeIDLrFEhM8r8x+SPhHtsdnXW8yluh4DglZaUmAeYGth5aYCco9NW1CXEjL5OWwpSxQ5YJMH/jX15YmXoo9IOd2k+4h9phLSW1EKbSsKrCmxBqcVaFnl8cUAwwkLcCnKQkKgkHjIUAEkHSdb6Ri2/QKy0M44UOAqOWVKABaVtSZB8h/wDmAvfBCcwkrLc8QAJEcjMX+GD8J0qVvFCiE0iFzqb2jw/bgCnNoNpQVkmkFSe6dU1VCIv3Tg13MpTRJ75CU85JBI08tcE1qLayWhIK4RI4omDpaf24NccV7P2cyeK44BSb+N4TbrgPTmkhRQTxBNcQe7JE/o0xwc+3u95PBe8HlPKJ5YTHNPSv+z6WSahxCqBoDAg1fPHDuZeCRGWBkE01AQZAANiJgk/DxwC1WcQAhRNllISYNyq48vjjr1lO83c8dNUQe7MTOmuEgzTtYG44ZF6hawkxHImLHkfAHzL5t495inu3qnUpB5TYEn8k6WJBUjOIKVqBs2VBVjYpufP4YC86gJQsnhWUhJg3Ku75TPPBC33Q3VuQVkgUBXIxUZjkZ8wPhjrfrlA3VikEmRwKlIpjqASZ8MAo9YTWG54ikqAg90EAmdNSPnjlGbQQsg2QSFGDYgSfPHDbyy4pJbhI0XOvd5R1J58vG3qHFQ4d3BBNIkcYAEGeU6YD1WbQEoXNllITY3Ku75a4VYbnMw4A1DM1U1ie5dA6XiSeXdw44DEDIviU9m+yD2bbU42ttISsoIVVMwlU2SbcWGrs3kUPOlK5gIJsYvKR+3F8+iXswz6q7df3c8x9RvwwA7L9gsw23llFxohAaJgq0TSTHD4YspDdJk4SsLKKWx3UwkTrAt88L1iRgOFCrTA3Rx2hMYac7tJaCsCm0xIP+eAcQiL47TfFeZjttmQmYa5e6r/VhMjt7mujX2Ff68BZ08sR3tBsJx/LraSpAUqmCZiykq5DwwNjbZddZQ4oJqVMwDFiR18MP7ioGAoH0g+j/MI3EuNX3mhX/wBP8XFXbXyKmHlNKIJTEkTFwDz88a52rsRvN070qFExSQO9EzIP1RjO3pP2K01tHNISVQgNxJH4JtXTqcBqFjujyGKR+kz94fnH7jF3M90eQxSP0mfvD84/cYCpmX2A4pamTu4EIk2PDJJnwPzGLI7I+j1xLrYWWVLqJrrWITSLRTewV88MexNiOuS7JD5HFBRAgppICgR7qfK+mLrRsnJNrKgtdaA4CDMQQ6F+6JsVm3TAd7F7NPMJWErbSpVMqSSZhKuSkwOIzh09VzISE+sCQDJoTeSsi0cgUi31TrhAwrJqS4DISCKgErsfa8o/v6C0Dpg3N5fJqACyoppXF16e2quPNYg6x5YBerJ5isqD6QmVQigdHAni5XUkmx7mtzjk5LMwoesJkxCqBw9/lodU9JpOmEmcOUStZk73j+sJMO1ERZUJKxN4sOYmPbZ2q003woTSq6wpK4gFwC1ovX8h8QlTmVzEADMAEVSaRBkqi0cgQPhz5nHLZiqrfimTw0DSDz84McoNzpiulbbW5ZLTahxATWDKlGr37yVHW3PlOF+z9ruuOpUtpsAklShVOhST3rkVEXnWOdwlz+RzCkxvwTaJAtczonnMaaAaXJj20uxrzrhWtbS5AAKlKBABV9VMHUfpw4ZP1UhUggFKCRCzaohNiDzBsBa2gwfnPVEpBUTTCiDCjaaVGY5m3j4zgKB2v2VLPranA2opU8UwpfDBPhciP04kP0c1o9deFPHuFmufdrY4Y874nnbLZ2VGTzZS85WWnTBAglSSuJKNOLrzA53i/oJ2UprNLVBoLCykkj3ls/roHywF44SOtLqWQuAUQkQOFV+Kfl8sK8Nz6WqnSrvboV2P3Pjj9SreGA8dYe3ZAeAWSohUCwIVSIi8Eg/DHq2XiqzoAkWgXEEEaWkmfhhuW9kyiaiUHeXhZuVErOmtUkDrpjqnKh0XNdUCKomkxeIPCTfz8cAoTlczf+0C4SBwpsR3jMXny546OUzMAb8A8V6AdSCLW0EjXQ8zfCJpOUCTBVEIkQue9w2idb20mfekhbWTCUSSEQspMr0kBRq1AJA587a4Bf6u/WDvhSDdNIuISDflcEx46nTBbeVzMGXwTa4SkR3J5Ge6q/4+gjBbbGXLgIqrqEd7UITF4vwkfMjmcE5NnKEcBJAoHvcy0pI01mieknS+AV+qZmAPWBN5NAuJRb5BQn8bwx63lcxUkl4FIpkUi8BFXKRJCjrzHxQhrJ0pAJKQVADiMGWgoaWuUecnqcGBjKbxCjVWCkJJr1hkpB8bN69Y5xgFLWXzAF3wTw8k9UT7vOFD8r44NaYeDZBeBWSkhUCw4JEReYV9rDdl2MpSSmqmw0VzLVoi8kI858ceNvZMoTClFIIAsrUbmxtI0bvgHlTTlCAF8QKKlQOICKh4ThXhpL2XQhhqqEkN7oXMhKmwi/mpAv1w7YDMnoVyqF59YWhKh6uswpIImtq8HzxoTZGWS2ghCAgFUkJAAmBe3O2M8ehvaSWc+tagoj1dYtE99o8z4Y0LsHaSHm1KSFABRF41hJ5HxwCraRCWHFmAQ2s1aRCSZnlHXFK9tO06/VvZZ1YVWnuPqCovPdVMYujbmXLuVfbTAK2XEidJUkgT4XxnLtf2IfyuX3ri2imtKYSVTJnqkdMBNvRP2sSGH/W89xb0U7/MXppT3a1TEzpiI9sO06zm8zus4soK1U0PmmI92FRHliu8w0ZGCaMAsO18z+He/wARf+ePUbUf5vu/4iv88JK8Omx9hOZoKKFJFJANU856A9MAryO189wUZjNUyIpddpib6GOuNZtk130k4o7sz6Ns0rLtLDjMXNyubKV+L4YvNscXzwB8gdBjNfpeSo7UzsAkeziP+w1i/O0O10ZeisKNVUUxyp1kjrjPfpH2+2vP5ghK7huJA/BIHXAabZ7o8himPpFNhS9nJJgE5gEnkJy979MXOyeEeQxTH0i1pSvZxUJTOYkdROXkajALvR9kkf0exDKHiQs71QupVbppJg90pCZnmMWEcw5UoerSnjvbije8o50p8954HFZ9h9tpTs9lLRW0kByEwlUEuPEmTc3I58ji0Nw/Uoh4QaqRAt36eWglJ/JwEW7ZbXUyhslAYJK7VhIXAPgJ+Xv4qnbnaR5eacHrzjKKUxDpIBsDEKj3lKt44s70gdmMxmjl5cbcS2pRKVcBgm4qSkySABoIpt1xS3aLZictnXG3WwoBLXChRIkhom5g34h+VgLY7Vds2zk3ktvtVFIFaH01a6wLkmOR97xxUvaHbTtDRRnVrJkqAcJpsk3v1UofA4jbNIQsKTKiBQqe6ZE/MfzfHi1I3aQEwsFVSp1FoHw/nWwOX9JPb1KPXHQk0VLC1QCQCrRUWJifDCjKbSfKSpOddCklQSkLVKoopgTPFJ5Wjzhm3qN4g0cIolP1oCavtGT8cS/sz2MeW8kHdErijiUAkmDJ4enn+3ATb0Q5zMlbxcdcfJbRwLKjTJSSrncFShpyxaTeZcoBOVkmuQdRBsO7z18uumI12O7Jv5UrJLaVKSkVJUTIBBIIUmBMHTErGWzFCYeEgKkkCCeVotH68Ay9vM0s5DOpDEDcOioESPZm8RqCYseR8AYP6D2AM0tW8knLqlEHh4279MPXbTb/APZ82mtUBDqSKU34aYm3ME/EdILF6D9qsu5xaW21JUMsoqUdDC2gYv1M4C78NjuacBVDE8pk3FSx9XSAD+V83PDa4y/KodAmKRAtdZvbmCkfk/MC8w8saZcK7/MaSQJtzF4F749RmFlyNxaYq+Bkm3W1vHwB8VlsxFnhPFcpF5Jpm3IRcfLBqGXt5JeFE90pBJEEQCIi9+emA5yr6lVVZeiEyJi5vbT+b+EknOOgAjK8UExOh4bTTzk/ZwrQ07Usl0FJHCAkCnWDN5/bj1ppzdqBcBWaoVAgfV5csAWp1QWgBnvQSq3CeYMDUD+Rg8Djp3YppBqjnNk6con5YC21wgBYBBTWYHEAOIRynBsGuarRFPjOuAROOKoUdwCUrpSm3EmQKhaw/wAseuOKG7IZuqkq/EugG8cgZ/IwoKFUr47maTA4bQPODe/XBS2XTu4dApiuw4+7Plz+eARt5tyP91iyYHmW7d20SfsYHrTlKT6pedJFvud9PHw7hwY3l34u+Cbck/8ATk6eCo/v+GB6tmKQN+J5kAfieHgr7QwBruZWN1DBIVFVx7O6fnEnTphxw3LYeO6IcAAA3lu8ZQSR0kBQ/Kw44DIPYfNKRmFERO7UL+aP8saH9FuYLmWdKokPEW/uNn9uMt5V9SDKVFJiJBjE97EdpVNMrSc2puXCYLpTPCgTE+H6MBplYkEcojEX7WdlmM0xunCsJrSrhIBkTGoPXCvs1txheWy85htS1No1cSVFRA8ZJJw+OoBEQDgM4dv+xmXyjjSWy4QpBJqUDcGLQkYrvOilakjQGMbGzOyMu4QXWGlkWBWhJIHhIxENpdkmC6spyTREmCGUx/8AXAZ07NZBD76W1zSQo2MGwJxdfo07DZZSH+J2ykaKHRX4uJNsLsuwl5Kjk202VfcpHI+GJhlMo21NCEInWlIE/LAFbN2UhlpLSCqlMxJE3JPTxwRt3NqYYW6iCpNMTpdQB0jkcH5jPtpJBdSCORUBFsVh6Qu1bC9nupZzralktQEPAqjeNkxBnSfhgGX0n9uMyPV4S1fe+6r/AKf42Kk2tn1PuqdWAFKiaQQLAJESTyGFG2c8t2ipxS4qiVExMfLT9GESYpvEwfPAbWY7o8hikvpMH/cPzn9xi7We6nyGKR+kz94fnH7jAMGx1vraFbaa0zVcACSpKTYx/wAT9OLsU1lK1mpVftArvdHSrlFpX8hrjMOXdIWpr1tSUADjBsZKZtVyqJ593FydmNvpOYQV5tLnCslsuJNZ3ZVSb9VHl7p64CZZjL5JTa+NQQCarGxh3kU2sVHTkMRbafZXJKzKsyMw8CsRYJp4Savc6tK16HE2y+0wsKKGEKUk6JUkz90vIFtOY9/HuZcVYjJ1SDPdt90jleYGn18BmjObHU3l3SiS2hShJI1qan/6J5dfgwvFzctyBu6llJtMmmrxi368af2vsTLlDqBs5ogzcNpMkEmSKeZQOuo8MRb/AGabpV/8WjhJpTuRxd644IE0p+fhgK77O9mzmEN5hSlBxISAAURACQjW90kc/li7Nl7FyrZQN84VAAQQPq1H3PxeWkDHOyMglrLinZySeM0hKUxBISLp5hI00kWjD45mSkkergJE8fhCteG3Txq+YImcvkgFEKVHBVNQgAkJ5CIPMRFjzkx7bm2mG1UIpU2AriVVMkwsfo+PjJw67T7QJQmVJbbMpiXEio30Mctet+WKl7Y7a3ue/wB4DCN2m4WFCZVfhIudfhgGDbPaJ59eaSGkEDe1GTIRNJVc3OnXriW/RzLnrTwj2W5Xe011M266Yq9CQpx0l2IrIPJwz3fytcWt9H9lAzjikuAqOXVLY5cbV7GOUfLAX3hpzrbBU6VkghtJcgmyE1kERzurS9/HDthE84anBu5AQCD9c8XBp/nrp1BpzKMoUitSqSFweLQlRVeL3mx1tqcGKby29upQcnodaSOl7GPjHO6lx9Uf7vPfAHgJp933v2nXmBmFVx6vae98DJ7vW3xPgCCbKO5UVBCzdCUkQZCVEgRw2Mm/6cGA5ZLEVQ2VkflXkAR58r/HHScysi+WvwWte8EAkche8c9LEg5tygf2XWolM2BBEe7zn9B1tIKsyppSWio2K0FBvdUGnTrj15TaXK1EhQSE6HRSgBoL8UeWCU5twuBG4NNuMmwsk6RqJI/J1vgpvOOLSSrLEHgEEzIJTPLlJN/q8sAQ8nKmCVK9oVEWVepTSlai1wj545Q3ld6g1KrBRTNQ91qmTF7BNjzPU4PTmVlKZykXIIsYHBeyb6n7GDBmF7xI9WEWlcixhv8AFm0qv/0/GwIMsnJElKVkmEyOLQlgCZHOGx89L4UZDK5VTVTaiW0wZ6Wac5johBjzwfkMwpaoVlt2AmQoxE8MIiJnn+Tg/JZlam1KUyUKHuSL8IOv6PhgOVBndM8kS1u4nW1A/wD3DlhuczCwGgGZqCSoT9zugdOQUTy7mHHAYcx0k45x7gLC7NdsWGfVgpLhLZamAPdKZji8MXj2Z7e5bOv7hpDoVQpcrSkCAUg6KN+IYoPs32TaeVlipbg3impim1SkgxI8cXz2W9HrGRf37bry1UKRC6IhRSZ4Ugzw9cBLHETpj1JgRgwDBShfAdJTGPHUTpjvHLi4wFadq88lGZeSQZEaf3Enr44zmtXD8BjWO1OxrOYcW8pxwKXqE0wIATaU+GKS7XdgMvlcm4+h11SkUQFFEXWlJmEg6KwFaoVGPCcdLTGOYwG3mO6PIYpH6TP3h+cfuMXcx3R5DFI/SZ+8Pzj9xgKcacZDoUUEtc0yQdPOdb64WdmNoMsPBx1ClQRBTy1ChEiZBx4HMx63NI3/ANW0d3zju+OOtgZdThW0J3agneKESIMpgnmVcvPpgLR7J+kHLNh0N71CiEcVAOiVjQkjvGfI4s/Y+0nc6wjMZd2hCqwK0iSQpxNxB0hPPkfjWHYjsSyQovuutoZKVopCDJWKlVQgkiG04svL5HKNNISXlkAKCVRFvag91Ec1D8kfEHX1fMBSlbxNMqpTGllxeOpR5UnWccDLZohQ3yZkUkpFhx6iL6p6afMlxrLb1XGoOcciD0cq92DAKuug+KZtvJUuALURascSou7Fikxeuw0jlzByey2ZgBLwBhV4FySqLU8gR8ufNBt9p5LTy1OJ3YCjSBeIIF46wY8Dc46fYytIqWuPaQeI6lwruUkmSVedowXncrlXFKbU4pK1EixMzSo2tfhqvfQ+MhR/pBzUts1mpIdmIi0KkcuVtRiBuutF1JCIbFNSeagIqJvqb6Hpi/8Atf2Yybzbf9odhtYXYARYgaonl+kdRioe1Gyg1nkpyxLhCG1iqBcGkaxbhGnjgI2HUSvgsQQgEnhkiDPMgTi0fo5rR668KTvPV1mqbU1s2jrN5xWYS4VPimTCi4NIAUCoi/JQFr4tL6Ohc9aeEez3K7/j1MSPlGA0BgkBVZM8MJgdDKpOnMEfLB2ESA3vlEH2lCZF4pkx+v8AmbgC07u1isVmuhUd2Zp5crcvnz5LT1U1iJ0t9U27tuIzzsMF+xDLnF7P2hWTNtSvUafr8Zv084zU2gqNSYWgCrSFJkxqIJ18PDAEpy+ZpILqauGFQLQTV7t5Fvh8/fV8zA9qmeKTSLyQRFrQJE310OuEZTlFbxdajVSVWVaVEiBTbiP6R1uHUZUpTLiiAFgG55iu9N7kec85wC8sZjeVbxNE92BpAtNPWT8vLBbeXzNJl5JVwxCQAO5V7pk2V9rlGE9OW3qfaK3lQpuq5pQByggiL88FNM5SgwtZTCZ72lSIEUwCTTykzgFqcvmIT7ZPiaRcSiwt0ChP4w1wZuX60neJptIgfiTHDzhX2tejapnKUplxUSYJnX2STJjwR8CT1ODgnLFxHGoqBQUzNzDdN6byAnU88A6ttrrJKuG8Jt0RBNpkEK5+9jphCxXUqZUSmLQm0DzwkyOYaWtRbUSo3IgiJS31AvFJjxOO8i61Q4tCpSVFSiZ1IBOo0iMAYWnKEAL4gUVqIHEBFXKAT/MYWYbXQ1umqlGgKaoJ1KpARIjmSOXyw5YDE2SZK1QElVpgAnpe2Li9EvZll3KuqfyiFqD5ALjQJCaGzAqGkk/M4gvot2WvMZtaEFIIZUrimIqQOQPXGiuwuzF5ZhaFlJJcKhSSRFKBzA6YBVs/s9lEIbpyzKVJCSIbSCCIIi1iDh1W6E3UoJHUmB+nHGZXQlSzokFRjWBf52xAe3Pb3LJy0lDsVp91Pj+NgJpm9ssIICsw0meriB+s4qXtR2scGYzAbzpCQpVNL1oi0QcV32y7Qs5tbam0rAQkg1AC5M2gnETePEYwE+7PdtM6X0hzPvFMKmp0xoY54t/0f9pG1pe9YzaFEFNO8dT0VMScZfThblMwlMyDfwwGymH0rAU2oKQdFJIKTyMEW1w27f2Sw5l1oLLawaeEoSQYUDpHhiufR76Rsrl9n5dhbbxUmsEpSmLuLIiV+OLWOVPhgM8+mDYjTPqu5y6W6t9VQ2EzG6iYF4k/M4rN1JBgiPDGivS/slbnqtJTbfak8914eGKF7RZYt5hxBiRTppdKT+3AbLY7o8hikfpM/eH5x+4xdzHdHkMUj9Jn7w/OP3GArPKbIK1qIdVCQCl4DvaAwahpJFie7i9dndnssh6f6LQAAvi3UTAVA7nvFIjXUeExLsT2eL+SbdaSA2qukKUZAC4M36pX8x8LgWzmZVDiaTVSI071N6eXD10OuAbmEJbSot5EBWlKR3oDgFymIgAX+vHmY9tNKYCsumYVIJTI+6QIpvMf/wBOc3I2nn3csn2jt1k0lKBYCswRH4yB+STaYxWXbHty0nMrSXFBYCZO6EaEi09Cn5fMJx2i7SMbp5KHGkGYDocbm6lBRjXQXmx3mpvise0vaV8bkNbRutcLKVpNIsJVfQVHpz8YgD+YZLbgoJcUolKpgAEpIkTcwF8uY+CFakbtICSFgqqVNiLR/P65sFubF7QLLUL2iFGe8VJFlQSIKvdqUPGD1xauV2+ypSW0hpZNqkuIJUaVEmmJJJEWmZ8gcqbxneINCt2KKkzc6VQfH+YmBKew23cqxmmXHEOFSFylQ920XE31PL/0GiVO1A15IWpgKAgkkzok6a/G4GI3tvZTKnQ4NnJKoiyRMA2vAF5mJ6664V9ne0/ryltsukqQEqNbYTaog8jJMgacvm9HIZqAN6iq8mBGoi1HS3x5xcMtZ/Ke3zWrQSXiBBgwuzVv5tiwvo5tj1x5Vd9wsbvwrZNevw0w3do9kFKc4p0BS6niFJUQNSZjzm3jhw+jmpHrrwpO89XWaptRWzaOs4DQuETrpCnIbJpQCFDVR4uCwn9eumkrcIX23SpVKwAQKRAsbydD4fLlzBIvPuJTw5YmayUiReT1THFE/Hnjs5tRXBYMaVGdIJPu/Dpf595hnMkcDiUni1AOpVT7vIR8ufPotZiqd4mme7F4jrHX5dTgEgzSoP8AZCLIgWuSbiyTEa3/AEY6W+YSRlZkKkR3eIC/DzF7TpodcGoYzNJBdSTCYNIsZ4jp0/Z4z2lrMbuC4neVE1BNgnkIi/6PPAFqfO8pGWlNQ44gXCZV3eUx+TgbyFKT6vwgpAVAg8SRpTNpJ0jh16LHEOQiFAEFNZjUe8B0nHdK95NQopimL1TrPSMA1nNqEU5U8+UQPZ/i2uf/AAx0MwreJHqpg0yu3DZvwvE8vwfwwqcbfpUA4kKK5SY0RI4SIMmJE4KbYzNSSXUFPDUKdbJqi1pNR15jAEZTNq1GVosm46EoEdwGwUT+Qfgq2e6S2SWN0eSDF+FJ5fZ/J+GCm2Mzep1JsnkPxKvc8F/aHTHbbOY3RBdSXahCqbU8MiI8/ny5AaHju2zujxUSj6kxMwPd/ZywuwjKHKUCoVAorMWIEVxa0/zGFmAy36GHyjPOEAfcFi/99vGi+zj5cbUTAhZFvJP+eMs9hs3uswpVdHsyJqjmm0/DGg/RnthtWWcKn0E74i7idKG/HASzbhjLPno04f8AxOM59t82VZWCB30afHGkFQsRZSFCOqVJP6CCMNe0OyuRUiDk2FCRYtJPXwwGR20TghwcRGL77ddj2g436vkUxQat0xaZ50p1xX+d7KP1LjIvReIy7n6IRgIKtAAx42icTPs52VzJfSHcjmKIVNeXcpmDEymMWn2S7G5cpc32QRMpitjzmJTgIp2J7JMvZNh1S3ApVRIBTFlqHNPhjQC1Yadl7IYaaShDDaEpmEhAAEknSLXM/HDm86lIKlkJSNSogDpqfHAVv6Ytoqa9VpCTVvtZ5brx8cUB2lzBczLiyACadNLJSP2Yur0559pRydDrZ+7zStJ/A9DijNqKBdUQZFr/AAGA2mx3R5DFI/SZ+8Pzj9xi7mO6PIYpH6TP3h+cfuMAf2Wzf9hbGYFK4XVTNhW8RcAxaTryGLLLWWrcG8UFGuof4tUCm8VK690a86S7PPpTlkpVm0mKhJKSSKpm6pgVn5Hxi7PXWyVpDCCeOIg1/dCbBMmqD1+6c5uEO7csNUthglcqXVJin3REgQOMj9PXFL9oEOf0hDafaS2Up/GpCuv7caWeQlaal5KVJJhJF4lZmQmL7sH8oc8M+f2DlC4HP6MQpdr0cQiUi9J0CAbGYItpIZvQ256uspTLYkFU3BJaJtV+InlzPwKQhxbbSKRSVkINpKiQIJJtfnbncxays92YO4dpYWACYaDS+LiRcACOn2B0tHc/2XfTuKG3RUoHhac9kTuyVG1iJ6jufII2p13ftmkbz2dAsAZAo52kEcxHhAhz7I5FbuZCTZLhKXFCJSAUKVA8ykaGxMaWmfZ3skVNpU60ouSONbS6h3SNb8Ex+T8rI2T2fy7ak05BCaT3gggHhJKop5lAHObeGASdjsjl0bxJdUpKUJSeHoQBMJ14cSXMM5YBNTigIWUm95VxRw3vy5i95OPMsEpqpyASYR7qeK5tZJ7ut+vI2wa9n0J1y6QSFcJpBiZEgp0Ub/DngKU7S7bd3WbQltJaBebrkkxUY1NzBB054V/R0Uv1p4Ujd7lfFzrqZtrpHh+rEa264FHOkPhPtX/Z1CTxGBAVfWNDp8pH9HQf2141/wDAX7O/12OPpfT4YDQeGnOssqWqtRmE1ACRAqp5Hr+r613bDbnHIUr2NcJSZiarqgd06fok6WkG15jK08TiwDvTJBm5UVi6epNvK17n7rLl6a1bwqmLwTBj3YNv1xzv68+Yk5SokLJESRdVjw+8b2nXnzO3ntCPV7VH2ka8JM2TMnT42J0wA2ShkFW6UVGEhRPLkOQ6X/8AePXiwW1yrh3igogSa5Mg2PWPLBWWzJAkZYoMJFhFiYIsnRIv8/j05mCWazliTV9zgTfVVx4/r1wCnMltQaKlGK0FBHvKg0zbQ49QWy8SCd4EUkXikEHpE8Q+YxwvMGptO5JBpMxZB06aj/PHJeKXFQx09oB3pKAfdmwM8xwnAeoDSUugKMKWQuxJC1QIAjxEWi+E05ZS2uMylKQgCYKZbKTp1Ui9tcdJzRUIOWMFUkEc5bAXFP4xN78BxyXiHEj1TmmVgCEmG792bSRI/B/AARl8tlqSQ4sp4QZnq3FqeoT9o6YDGWyxZBS4S3UkTqSVFopF0zelHz6aGNP2/wB0iwgU+LUDucpnw3fKMKG35aKvVyOJMtkXPd4oi8T093AGuIb3TQKjTU1QeZIIKJtzgT8cOOEKj7Ns7mbt8FuCSm9/qa2HLC7AYgZUAb4nHYvtIxl2VocC5LhUKUg2pQOvhiB4UMOwPjgNQ9nO3OVUzl0AOypLaRwjUwB73jiXtZpKzSJnW+Mp7M7auM7sBpB3dMSTemNflia7K9NGYLlsqyLH3l+HjgL/AE2x7WMVr2e9Iz2YSpSmW00qixV0nrib7PzxcbQsgAqAMDAOhGOQYx3iP9ptuKy5bCUpVUFG88o6eeAdnRcnEB7Z9vMqrJOCHb7v3R9dH42I/tv0u5hl5xoZZkhMCSpcmUg/txVG0+1a3mi0W0gGm4JmxB/ZgD+2W128zut1VwVzUI1oiLn6pxFza2O994YLWZOA26x3R5DFI/SZ+8Pzj9xi7mO6PIYpH6TP3h+cfuMBUrOYyocUVNrLUcKQeIHhm9X97rqPhaHZvtoyM03Dh0cgbu80O0mY0unlyOvOsA6/61VQnffU5d3z6X1wq2Ltp5hNaG0qQ0QSTVaopieLqgaDr1OA0fsrb63w4G3pUmCCUAQDvIkUmTNPy+077rMrAKXUCypNIJJ9oEwCm0SjrNJtzOeMt6R3kNwcu2UqsCFGZCaTrPJQPL9eJNle3uZWkIOUTUkGwdIIClKi8dXY1tbxwFzrbzNZIcTRxQnn79N6OpR10OvPkMZqFDepn3TAt39eC+qR8D8YfkNtIdzCahSpdUwTzS6TApgmKo8ucmX5lrKkOe0XEJquowJdiJSYupWnQDSxBwey+bKeF1AVCrlMiSVU2ptAp66G3PHbjOZqMOJovwxfRQF6epB56c+aF7L5YpALy4hyDN9V13pvBJ+Q63hm3e1Ia3roRK2ypQTUoAmFJuIiIJvF4EzGAlm0toO5VFTzo4iEpKUaESTIp6D/ANCL1d247YtHMXcJVQL7vxUUjTl/M4aNudvnwltz1ZKQe7LqlAgA6CZHe+FhyxFtp7cfczLbpaQlam20oTqkpngNzBM9el8A0uOILjylgqqrKItCiZSoidNbXxZv0dFo9eeEHeerrM8qa2IHnM/PFcMvPQ8hKBosuaykSAqLxGg5/qxZH0dHleuOogUlhxQPOoLYBHlBH8jAaEwkdQ5UuFAJKAEDmF3vp5dfLqrwjQlG+UQTXSkEcgJVB87/AA8JuCVxnM02cbCpXeJEEmgaagQCecaXx0WcxXO8TRPd8I0mnmflbWLlu5dlTK5WaEqcUo2seKrVOgmdL2N5MmqyKJSalcRFOhuEqj3ekm/7YwHDbOagy4gnhgx0MqkUiZFot8OYVl8zSmHU1QajAiTERwaC/wA8FK2cympBWsQhJIkWQkkg93rrz+eCV5TL0pl1ccYB5mSCr3b8vOfHALQzmawd4imRIjlABA4esn5Y5ZYzQHE6gnh5dCkq90aiR8tMJUsZetPtV1VAi5gmlsDlBEFF+dQnBbDOVpMOrKYRPgKk02ptePtHScAs9XzUD2qJvJp17nKnwV8x5473OarSd4ii1SYubImDFrhf2hfohVlsuUpBdXFwkyZJJZ6JvegxzqJ8jBkGAppVbkwkoubgbpIkRe6USTe/TAKWWMyO84g93p1RVejmAsacxpyPybbwbUHFpUv3VAWHCNbCeKT5YKymRbbdELUV7uKTEUcCZgACeAfpwMvlGg2pCVqhKk1KJBIUgI6iIhAm0a6YBSUO0IAUKwUVnkQIri3O/IfDCvDchDSmmoUd2N0UHrpRNtDb/wBYccBjT/ZXPf8AJZr/AAHP9OOk9l89/wAlmv8AAc/042TgYDGx7LZ7/ksz/gOf6cetdm9oJMpyeaB8GHP9ONkYGAyNlcntdsENs51INzDTuv2cOLW0O0KQAn+kQBoA27b/AMcaowMBltO1+0U3VtH/AA3f9OCM7mduuRWNoKiYlt23/jjVeBgMfZnYu01qKl5bNqUdSWXZPL6vQYI/2Wzv/JZn/Ad/042RgYDGyuy2e5ZLM/4Dn+nHP+y2e/5LNf4Dn+nGy8DAFsjhHkMU19IvIOu+o7ppxyPWJoQpUTuImBbQ/LF04GAyCOzmb9Yppf8A+9u3Pq9f/HXCdjs9my24dy+mAng3TnHfyvGuNjYGAxw72ezW7QQy+ZKpRunOA2vp7wj5eGHBvY2dS8Gx6wEkAF0NuRFIMT4aXPIactb4GAyhlHNqgLWN+lSBwgsGVTwkCUdFGfPpJCr+mdtBAUF5iVEgp3FxGhPBzqPzOt41LgYDMzW0dsqWEF58JhJqLHCKimRdHKtSutjIBJhiWdqOpcrTme7JBZVxkqSkju6won4HGtsDAY8e2HnFNJUtrMKIUUhBbcJSIFxaw5fAa8uz2cze8aFD1w37TduezmLG0ijw6Y2BgYDHjPZ3NlTvsn0wlRndue0uBTperXniyvo/7Jeazbq3UOIBZWkIWhSb1smoTblHwxfOBgBhIlXtVCiOFPHGt1cMxy6TzOlpV4GAb6/ZOHdabz2cd/XlF6vIzPPUmvKjd+zmVAadzhVxaW+ry72FeBgEaiC6UlHuCVxYiTwEx8YnmdOZCUo3azuBwFYSmkcUGZSKfeI6a9cOeBgG5SEANqDAlRSCKRKJGptygDHpYQFlAZTFAVVSKZBsnTUQD8BhwwMA2ZZKFIKtwEUzCSkA+6q1rSQPkOYjBrqwA2vdSSUp0ugG5JtIAjC7AwCUr9rFB7hO8i2o4J/TGOWXLOndEQpVoErgDiHWdPhhZgYBAV+zb9lqW+CO5cXiLU+QiOXJfgYGA//Z)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ePaa3CuJhERO"
      },
      "source": [
        "## 文本向量化"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wMdcDcD3eNCP"
      },
      "source": [
        " * 象徵式表徵: TF-IDF (依據詞頻)\n",
        " * 分佈式表徵: fasttext (靜態詞向量)\n",
        " * 分散式表徵: transformer (動態字向量)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OkXUDKlaJ7Z_"
      },
      "source": [
        "### TF-IDF"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l2u951-CQKxA"
      },
      "source": [
        "![](https://miro.medium.com/max/1000/1*vWWmJlDykVRkjg9c38VbxQ.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "4RzA97fLWdOn",
        "outputId": "aebc48d6-2a07-4d04-de98-75781b6b5b82"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: texthero in /usr/local/lib/python3.7/dist-packages (1.1.0)\n",
            "Requirement already satisfied: scikit-learn>=0.22 in /usr/local/lib/python3.7/dist-packages (from texthero) (1.0.1)\n",
            "Collecting spacy<3.0.0\n",
            "  Using cached spacy-2.3.7-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.4 MB)\n",
            "Requirement already satisfied: nltk>=3.3 in /usr/local/lib/python3.7/dist-packages (from texthero) (3.6.5)\n",
            "Requirement already satisfied: gensim<4.0,>=3.6.0 in /usr/local/lib/python3.7/dist-packages (from texthero) (3.8.3)\n",
            "Requirement already satisfied: pandas>=1.0.2 in /usr/local/lib/python3.7/dist-packages (from texthero) (1.1.5)\n",
            "Requirement already satisfied: tqdm>=4.3 in /usr/local/lib/python3.7/dist-packages (from texthero) (4.62.3)\n",
            "Requirement already satisfied: unidecode>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from texthero) (1.3.2)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from texthero) (1.19.5)\n",
            "Requirement already satisfied: plotly>=4.2.0 in /usr/local/lib/python3.7/dist-packages (from texthero) (4.4.1)\n",
            "Requirement already satisfied: wordcloud>=1.5.0 in /usr/local/lib/python3.7/dist-packages (from texthero) (1.5.0)\n",
            "Requirement already satisfied: matplotlib>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from texthero) (3.2.2)\n",
            "Requirement already satisfied: smart-open>=1.8.1 in /usr/local/lib/python3.7/dist-packages (from gensim<4.0,>=3.6.0->texthero) (5.2.1)\n",
            "Requirement already satisfied: six>=1.5.0 in /usr/local/lib/python3.7/dist-packages (from gensim<4.0,>=3.6.0->texthero) (1.15.0)\n",
            "Requirement already satisfied: scipy>=0.18.1 in /usr/local/lib/python3.7/dist-packages (from gensim<4.0,>=3.6.0->texthero) (1.4.1)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.1.0->texthero) (3.0.6)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.1.0->texthero) (1.3.2)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.1.0->texthero) (2.8.2)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.1.0->texthero) (0.11.0)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from nltk>=3.3->texthero) (7.1.2)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from nltk>=3.3->texthero) (1.1.0)\n",
            "Requirement already satisfied: regex>=2021.8.3 in /usr/local/lib/python3.7/dist-packages (from nltk>=3.3->texthero) (2021.11.10)\n",
            "Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.2->texthero) (2018.9)\n",
            "Requirement already satisfied: retrying>=1.3.3 in /usr/local/lib/python3.7/dist-packages (from plotly>=4.2.0->texthero) (1.3.3)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.22->texthero) (3.0.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<3.0.0->texthero) (59.5.0)\n",
            "Collecting catalogue<1.1.0,>=0.0.7\n",
            "  Using cached catalogue-1.0.0-py2.py3-none-any.whl (7.7 kB)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.0.0->texthero) (1.0.6)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.0.0->texthero) (0.8.2)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.0.0->texthero) (2.23.0)\n",
            "Collecting srsly<1.1.0,>=1.0.2\n",
            "  Using cached srsly-1.0.5-cp37-cp37m-manylinux2014_x86_64.whl (184 kB)\n",
            "Requirement already satisfied: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.7/dist-packages (from spacy<3.0.0->texthero) (1.1.3)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.0.0->texthero) (2.0.6)\n",
            "Collecting thinc<7.5.0,>=7.4.1\n",
            "  Using cached thinc-7.4.5-cp37-cp37m-manylinux2014_x86_64.whl (1.0 MB)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.0.0->texthero) (3.0.6)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.0.0->texthero) (0.4.1)\n",
            "Requirement already satisfied: pillow in /usr/local/lib/python3.7/dist-packages (from wordcloud>=1.5.0->texthero) (7.1.2)\n",
            "Requirement already satisfied: importlib-metadata>=0.20 in /usr/local/lib/python3.7/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy<3.0.0->texthero) (4.8.2)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.0.0->texthero) (2.10)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.0.0->texthero) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.0.0->texthero) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.0.0->texthero) (2021.10.8)\n",
            "Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20->catalogue<1.1.0,>=0.0.7->spacy<3.0.0->texthero) (3.10.0.2)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20->catalogue<1.1.0,>=0.0.7->spacy<3.0.0->texthero) (3.6.0)\n",
            "Installing collected packages: srsly, catalogue, thinc, spacy\n",
            "  Attempting uninstall: srsly\n",
            "    Found existing installation: srsly 2.4.2\n",
            "    Uninstalling srsly-2.4.2:\n",
            "      Successfully uninstalled srsly-2.4.2\n",
            "  Attempting uninstall: catalogue\n",
            "    Found existing installation: catalogue 2.0.6\n",
            "    Uninstalling catalogue-2.0.6:\n",
            "      Successfully uninstalled catalogue-2.0.6\n",
            "  Attempting uninstall: thinc\n",
            "    Found existing installation: thinc 8.0.13\n",
            "    Uninstalling thinc-8.0.13:\n",
            "      Successfully uninstalled thinc-8.0.13\n",
            "  Attempting uninstall: spacy\n",
            "    Found existing installation: spacy 3.2.0\n",
            "    Uninstalling spacy-3.2.0:\n",
            "      Successfully uninstalled spacy-3.2.0\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "zh-core-web-md 3.2.0 requires spacy<3.3.0,>=3.2.0, but you have spacy 2.3.7 which is incompatible.\n",
            "spacy-pkuseg 0.0.28 requires srsly<3.0.0,>=2.3.0, but you have srsly 1.0.5 which is incompatible.\n",
            "en-core-web-sm 3.2.0 requires spacy<3.3.0,>=3.2.0, but you have spacy 2.3.7 which is incompatible.\u001b[0m\n",
            "Successfully installed catalogue-1.0.0 spacy-2.3.7 srsly-1.0.5 thinc-7.4.5\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n"
          ]
        },
        {
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "catalogue",
                  "spacy",
                  "srsly",
                  "thinc"
                ]
              }
            }
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "!pip install -U texthero"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sIi-oXx2j5po",
        "outputId": "6adc52c3-9206-4b67-8e44-f01cd899fb01"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Name: spacy\n",
            "Version: 2.3.7\n",
            "Summary: Industrial-strength Natural Language Processing (NLP) in Python\n",
            "Home-page: https://spacy.io\n",
            "Author: Explosion\n",
            "Author-email: contact@explosion.ai\n",
            "License: MIT\n",
            "Location: /usr/local/lib/python3.7/dist-packages\n",
            "Requires: blis, catalogue, cymem, murmurhash, numpy, plac, preshed, requests, setuptools, srsly, thinc, tqdm, wasabi\n",
            "Required-by: en-core-web-sm, fastai, texthero, zh-core-web-md\n"
          ]
        }
      ],
      "source": [
        "!pip show spacy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_FsMZZsOmC5h",
        "outputId": "920ea99e-5291-4a48-bc33-b768becce6cf"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "spacy                         2.3.7\n",
            "spacy-legacy                  3.0.8\n",
            "spacy-loggers                 1.0.1\n",
            "spacy-pkuseg                  0.0.28\n"
          ]
        }
      ],
      "source": [
        "!pip list | grep spacy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "L54vkQgVKDrX"
      },
      "outputs": [],
      "source": [
        "import texthero as hero"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MiWOGGwoKIAV",
        "outputId": "5bdc697d-a842-49b5-af0f-e8e6ed023143"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時2.5875747203826904秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "train['tfidf'] = hero.tfidf(train['tokens'])\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "id": "8fd8mlwIWvJz",
        "outputId": "8dd996ac-47d4-45b1-88a5-4be1aa7a2035"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>label</th>\n",
              "      <th>cat</th>\n",
              "      <th>tokens</th>\n",
              "      <th>tfidf</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>4149</th>\n",
              "      <td>1</td>\n",
              "      <td>水果</td>\n",
              "      <td>新鮮 感情 京東 小哥 幫 搬上 樓 度 高溫 辛苦</td>\n",
              "      <td>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1890</th>\n",
              "      <td>0</td>\n",
              "      <td>酒店</td>\n",
              "      <td>老氣橫秋 不過 價格 那樣</td>\n",
              "      <td>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>350</th>\n",
              "      <td>0</td>\n",
              "      <td>酒店</td>\n",
              "      <td>飛機 取消 臨時 住宿 怎麼 不會 去住</td>\n",
              "      <td>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>79</th>\n",
              "      <td>0</td>\n",
              "      <td>平板</td>\n",
              "      <td>系統 接完 電話 掛掉 宕機 無 響應</td>\n",
              "      <td>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3927</th>\n",
              "      <td>1</td>\n",
              "      <td>水果</td>\n",
              "      <td>挺 還 沒償 著 覺得 應該 味道 還 不錯 快遞員 特別 給力 給十 個 贊</td>\n",
              "      <td>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      label  ...                                              tfidf\n",
              "4149      1  ...  [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...\n",
              "1890      0  ...  [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...\n",
              "350       0  ...  [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...\n",
              "79        0  ...  [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...\n",
              "3927      1  ...  [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...\n",
              "\n",
              "[5 rows x 4 columns]"
            ]
          },
          "execution_count": 179,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train[['label', 'cat', 'tokens', 'tfidf']].tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BMxKRQ8HslkD",
        "outputId": "1268ec38-ff1d-4e22-98fa-6a7ca8017044"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "9615"
            ]
          },
          "execution_count": 180,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "len(train.iloc[0]['tfidf'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pEMk2GHyLIAO"
      },
      "source": [
        "### PCA"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FS_zJujkRv8z"
      },
      "source": [
        "![](https://3.bp.blogspot.com/-aGt6JrHLt_E/WjXd4Ge7B-I/AAAAAAAAAgY/-gF8sXb4qqAwObOmP9hTiJa0ot5wR82ZACLcBGAs/s1600/Principal-Component-Analysis-And-Dimensionality-Reduction.jpg)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WCoeoCZsW1O9",
        "outputId": "9b35fa66-6aad-460e-d538-f42412ecc77b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時6.714311838150024秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "train['pca_from_tfidf'] = hero.pca(train['tfidf'])\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "id": "5_5we3k3LIeH",
        "outputId": "44794ef2-4f1d-4a43-bd3f-6df6c045b70f"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>label</th>\n",
              "      <th>cat</th>\n",
              "      <th>tokens</th>\n",
              "      <th>pca_from_tfidf</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>4149</th>\n",
              "      <td>1</td>\n",
              "      <td>水果</td>\n",
              "      <td>新鮮 感情 京東 小哥 幫 搬上 樓 度 高溫 辛苦</td>\n",
              "      <td>[-0.033385881479503686, -0.018786756386627083]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1890</th>\n",
              "      <td>0</td>\n",
              "      <td>酒店</td>\n",
              "      <td>老氣橫秋 不過 價格 那樣</td>\n",
              "      <td>[0.0010343640774986157, 0.04453904148327922]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>350</th>\n",
              "      <td>0</td>\n",
              "      <td>酒店</td>\n",
              "      <td>飛機 取消 臨時 住宿 怎麼 不會 去住</td>\n",
              "      <td>[-0.032349283013356794, 0.04388112093477036]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>79</th>\n",
              "      <td>0</td>\n",
              "      <td>平板</td>\n",
              "      <td>系統 接完 電話 掛掉 宕機 無 響應</td>\n",
              "      <td>[-0.021481646422848312, 0.0428861094195463]</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3927</th>\n",
              "      <td>1</td>\n",
              "      <td>水果</td>\n",
              "      <td>挺 還 沒償 著 覺得 應該 味道 還 不錯 快遞員 特別 給力 給十 個 贊</td>\n",
              "      <td>[0.09462444035495124, -0.06286792656536076]</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "      label  ...                                  pca_from_tfidf\n",
              "4149      1  ...  [-0.033385881479503686, -0.018786756386627083]\n",
              "1890      0  ...    [0.0010343640774986157, 0.04453904148327922]\n",
              "350       0  ...    [-0.032349283013356794, 0.04388112093477036]\n",
              "79        0  ...     [-0.021481646422848312, 0.0428861094195463]\n",
              "3927      1  ...     [0.09462444035495124, -0.06286792656536076]\n",
              "\n",
              "[5 rows x 4 columns]"
            ]
          },
          "execution_count": 182,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train[['label', 'cat', 'tokens', 'pca_from_tfidf']].tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "gIvsvZilMY-R",
        "outputId": "86226df5-c484-446e-a8e4-f1bb688ae5d0"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>\n",
              "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
              "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
              "            <div id=\"4199c138-a69f-4c05-81e8-45d42eb4af38\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
              "            <script type=\"text/javascript\">\n",
              "                \n",
              "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
              "                    \n",
              "                if (document.getElementById(\"4199c138-a69f-4c05-81e8-45d42eb4af38\")) {\n",
              "                    Plotly.newPlot(\n",
              "                        '4199c138-a69f-4c05-81e8-45d42eb4af38',\n",
              "                        [{\"hoverlabel\": {\"namelength\": 0}, \"hovertemplate\": \"cat=\\u6c34\\u679c<br>x=%{x}<br>y=%{y}\", \"legendgroup\": \"cat=\\u6c34\\u679c\", \"marker\": {\"color\": \"#636efa\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"cat=\\u6c34\\u679c\", \"showlegend\": true, \"type\": \"scattergl\", \"x\": [0.006716820914296273, 0.1342925286456653, -0.02470403113615121, -0.12971551125226266, 0.021263492672449112, -0.07108270367453373, -0.09628215100944543, -0.058460303816676024, -0.11115584711058563, -0.05237959568291481, 0.016065475522662808, -0.19304831304769848, -0.02783155860697278, -0.09835299152264719, -0.08007847559858738, -0.08624225859299957, -0.06611902950127899, -0.03175424175347286, -0.14157987246830242, 0.03381253563152528, 0.11019199072484867, -0.007782898284959778, -0.07902459940817816, -0.08537944755752302, 0.011561911907421153, -0.0603327388838532, -0.09130694692652423, -0.10170539217503638, -0.052006710082322966, -0.1227911313258988, -0.014248154484171369, -0.044616034675703777, -0.06394120750447792, -0.024717834442748228, -0.07316295262084895, -0.04246935873029277, -0.10613139258039664, 0.03667218221942502, 0.12354416456241667, -0.14185720726104645, 0.1448354190695758, -0.1108399809981128, -0.015239095531174045, -0.03440454507926396, -0.08380796229117034, 0.06952111362420133, -0.06440677143361515, -0.09406748440092497, -0.0013725661003196705, -0.13333709499104054, -0.0625204599829412, -0.05925825762398807, -0.07685147499220786, -0.18632027999038794, -0.07422451431990111, -0.033096745192185424, -0.057532484412711485, -0.08387383375968911, 0.04962785189409256, -0.10722804302167276, -0.06404036735862305, -0.04128110705146857, -0.08269103165506383, 0.11142466100738119, 8.231585614611378e-05, -0.11511844844019514, 0.011928741811211861, -0.1600719509235999, -0.10330771385703606, -0.03785157040407431, -0.09662638290407016, -0.04444519349302274, 0.07013433695712831, -0.07822492904274168, -0.06177274912582219, -0.02839827456109694, -0.11345465803838162, -0.09981961996392232, -0.11277908631270922, -0.02619637853719201, -0.17239807245140376, 0.12124359305647549, -0.10515612605060011, 0.034546971929295206, -0.1087648246138575, 0.03961564769342998, -0.03765875266007768, -0.0408920941467643, -0.046349793932091425, -0.026020285138811175, 0.04420764227641567, -0.05420454563808286, -0.035162495033693175, -0.037107033539463395, -0.0032134286874236076, -0.055689438360508596, -0.05184930297953428, -0.08162631149824034, -0.07599081599642153, -0.08130959854616808, -0.015155078540079948, -0.06349237210672963, 0.04372831845469649, -0.07520762724997157, 0.11104136853617604, 0.05370408741338455, -0.10470845755965953, -0.015632704527651642, -0.08597459306995885, -0.03528585491783801, -0.09888421121922286, -0.038263448361881086, -0.03431314748581591, 0.08438730343089244, -0.03779750123016249, 0.0019776724915377243, -0.07350386475342555, -0.04166966773963971, -0.017369799143146097, -0.0982806861970633, 0.08876427813070041, -0.07006302765697828, -0.12955154488200496, 0.07690615694234647, -0.03250389457523184, 0.1507322769409681, 0.047064744758237714, 0.021220354425182848, 0.038988855721138936, 0.1133049414264158, -0.09269676121439786, -0.08930138583949061, 0.013800908717165546, -0.0024826849512074632, -0.027697148961887644, -0.1432245356965889, 0.18297262260337005, -0.05621574580150369, 0.010182259896954341, 0.025469158312844284, -0.17860444630042924, -0.030119769556111828, -0.08041130137023887, -0.12768081784689145, -0.018691436827992007, -0.052539350007542604, 0.050252559119822585, -0.049205391462956376, -0.06354108241512808, -0.06830672680838687, -0.01251732128182942, -0.17039297786677582, -0.11835544216077762, -0.022661967026531536, -0.042467526660369376, -0.12242158296768699, -0.06559147624650383, -0.1239965289349194, -0.07263263437871965, -0.00029210863581662696, 0.29806679618390536, -0.07429448423530899, -0.019918890922353407, 0.07717216991980122, -0.007488676892215015, 0.021623413272395597, -0.08321181645067975, -0.002203060735046089, 0.06285652527954827, -0.12146198638046647, 0.03615866129295931, -0.010664387265308554, -0.04433256059623096, -0.040182711400272436, 0.03221824152626607, -0.03284420645822316, -0.06160584165235196, -0.03303184224551585, -0.1083077735373772, 0.09446870228206869, -0.06900602918181171, 0.11793327048861614, -0.1461878412694662, -0.15359328341071177, -0.02196408650908488, -0.0948215990940116, -0.16903789546406495, 0.15409753557249742, -0.13082130684569712, -0.0937812094224528, -0.059580255510043846, -0.07774953036187247, -0.04134068695399718, 0.0007617431216477616, -0.0922083433439702, -0.013266092008129195, 0.08956546387300615, -0.018709740457406122, -0.04406063945120746, -0.12238687046544608, 0.007786820965092408, 0.17345250884014227, -0.06629132163732855, -0.1118770224713136, -0.02712768412908098, -0.16119869605587933, -0.08317525446375115, -0.07482007038597098, -0.053447432890649786, -0.07604289775537558, -0.006534556225702221, 0.24265259525247423, -0.12496713563972686, -0.15147437070083808, -0.010433289115964485, 0.13367908519431737, -0.07860968691294666, 0.05615134814360592, -0.0632285214562837, -0.1043577237500677, -0.12653612075492754, -0.057447962920728575, -0.05316621232002623, -0.10303458268168099, -0.10233904038147634, -0.06781381181192939, -0.07649591568254456, -0.1046132614105131, 0.020406684286440324, -0.03354594538342447, -0.04839899026113371, -0.09078171355186329, 0.0036793038283242222, -0.0056735676809343254, 0.0031543088057370407, -0.1022727573576898, -0.04955340284310686, 0.032566888139445804, -0.06489046719469295, -6.461394695823921e-05, 0.1674713939054074, -0.011563225851613467, -0.06023219711169039, -0.06167161960907697, -0.04849352456256114, -0.07594950242907296, -0.05065528265257278, -0.0679389212769898, -0.04915598895524259, -0.014672809113583134, -0.10210841305765553, 0.035223537203217976, -0.08302166324285783, -0.0009685683271931058, -0.07862066326636756, -0.1268252962471544, -0.12326019889089218, -0.0352751062990502, 0.08621543398998888, -0.05218865359380022, -0.002438701851108554, -0.03977622890797324, 0.026689287845030225, -0.06589614285960142, 0.11293051475495754, -0.07290207605780168, 0.015304967350199065, -0.08110367390528575, -0.15846031749591202, -0.08438732803599683, -0.05026421299680681, -0.06529474535969358, -0.020473354953311203, 0.17179116744802525, -0.05344469330776379, -0.02211336329119536, -0.12201795481411454, 0.008000657354545661, -0.10477819938620436, -0.042484287502487686, 0.0868012378188407, -0.1951423762946137, -0.04113698255719733, 0.08720431629622827, -0.09378425361677625, -0.02898066027536759, -0.10318095367132409, 0.010976700704608093, -0.015922407398853165, -0.05390703391838776, -0.014947886572177799, -0.025219895759732113, 0.043625511403735245, -0.054760730223885994, -0.0524765191033535, -0.050908103031331586, -0.06118569429790668, -0.047467149602385315, -0.07116196852130466, -0.04701265665242429, -0.10635060298638854, 0.055442949435720534, -0.12813367911520143, 0.12124165185230967, -0.02827420272342144, -0.08831729649909655, -0.026645198588121328, 0.011754296367973033, -0.12798280353630886, -0.016321215922745207, -0.03621184861109452, -0.03039916904894711, 0.06927160692527243, 0.07962204531067996, -0.09260802021385986, 0.19156324309544273, -0.05556185553435324, -0.1691964300436622, -0.030026186374425383, -0.050231287230954255, -0.0938366300081756, -0.12031795189730532, -0.052772051797492094, -0.07301074752173892, -0.04545433189740327, -0.09075233627571984, -0.08180183174220065, -0.07208908162885241, -0.04160071353130132, -0.1556414654570998, 0.13443759862024032, -0.13629296834348395, -0.061391322355444256, -0.010648981824824067, -0.08569774126676231, -0.09653367042474577, 0.06835768073924724, -0.0642154941339569, -0.10117205713657769, -0.09762369252519462, 0.15462862247996864, -0.043870851578572304, -0.1052773264589829, -0.13242596504508952, -0.0860398241159455, -0.05795464126775504, -0.16734972526165603, -0.011565097493261782, -0.06955396844102413, -0.03285770993090527, -0.06722649703676459, -0.008019570195333767, -0.018084635031321898, -0.09024021142844878, -0.07497683070543411, -0.13788199329130682, -0.009779226441051224, 0.015465540519404445, -0.15362478169378058, -0.06832105564872691, -0.08986014295440847, -0.06608537997591261, -0.0008809915776903465, 0.002949349665479442, -0.03233067101959407, -0.0249218714748775, -0.09867979023573545, -0.03332532460710183, -0.16780255103035294, -0.04969663799625574, -0.05687992888186097, -0.09829057400786333, 0.18675931393439402, -0.15683097202361018, -0.10243613491946098, 0.01738162883706886, -0.1007528471181667, -0.07248289041975801, -0.11519721143347586, -0.00110491372443901, -0.08863231038322918, 0.015580800055793402, -0.034061228360039976, 0.030600635857934023, -0.05876523384519566, -0.08958248315464566, -0.031300656377378194, -0.02855666779945155, -0.04282083744740798, 0.03267799922681067, -0.06009281612200393, -0.06231322371882318, -0.05186033293719671, -0.03284560737925786, 0.07240462415995312, -0.04803749820656433, -0.0748372092629015, -0.08781664052757032, -0.10579493419483349, -0.04935522279613288, 0.04157755659615456, -0.09621273759461016, -0.06744247805855483, -0.12635894496794067, -0.05919716309695937, -0.03968439629500191, -0.08653623423757359, -0.045288068190282804, -0.11743889636698361, -0.13286611141495278, -0.03325090288548212, -0.06841634288811362, -0.05137638689857272, -0.06960943502349552, -0.06785328002756286, 0.038975864962961196, 0.23727569200573945, 0.03735115401540964, -0.053789745627417354, -0.11905815430282747, 0.06750187837151515, -0.09865005673692849, -0.08636942039579885, -0.0642086632202324, -0.0815024794924351, 0.005163032993485084, 0.0660742206372989, -0.06385158841636109, 0.024529672541223943, -0.012513638620681097, -0.014172581066391344, -0.02755236198849565, -0.08497397715331426, -0.04638741133549772, -0.11448688689138428, -0.1316895760633947, -0.04880849852329912, -0.018344156901287437, -0.11348308008327558, -0.04304095528897512, -0.1665061751088814, -0.12937123896476221, -0.1331330430291034, -0.03878550520008994, -0.03724515812087039, -0.1000928174412921, -0.06254316520746056, -0.08876166051842563, -0.07460023802396018, -0.06304044079590514, 0.1263621113629622, -0.026603691240685158, -0.031023679635478016, -0.01965409544415952, -0.06389290474754157, -0.06981059403388977, -0.021078254918731092, -0.15656854831449793, 0.15679772423289276, 0.06704545443451805, -0.09335216591006067, 0.0019376040846570603, -0.0605869380309026, -0.02815832724944907, -0.07008212359443902, -0.07975281034535239, -0.028808988285706997, -0.059899893239229646, 0.028278736661555493, -0.05527980603750396, -0.09603107357285262, -0.05627515409763512, 0.10935912760153843, -0.046683830850612645, -0.012513638620681097, -0.05437926140053627, -0.10977270693825343, -0.07373475937113859, -0.08207937122081668, -0.1001705325171616, -0.06173800010217422, 0.08861739771914545, -0.11936516130979231, -0.12650554627965188, -0.113605052246179, -0.08491514633898457, -0.03344768190438679, 0.09198997927719009, -0.1735363526354762, -0.07840235126583711, -0.11771362559273837, 0.06976781965188779, -0.10142050574441056, -0.06900929566581782, -0.06469490903843159, -0.04285192348989644, -0.016204267391844735, -0.09267146782758288, -0.09194469476021239, 0.12978849969022743, -0.11407584732987723, -0.047906604375693274, -0.07280911294661463, -0.052855455478024525, -0.17326740715666225, -0.03542871863049157, -0.023804985496994016, 0.07140251065838694, -0.10398512998576387, -0.03648810935628538, 0.22915723972897828, -0.08091550100223122, -0.06596811647697123, -0.09201262315920244, -0.0737357943582764, -0.13944745790780394, 0.14153214051147503, -0.06094835942241012, -0.05857358405705903, 0.15595624029763022, -0.05777965913319507, 0.15450130599560138, -0.071688240205779, -0.08596772593405683, -0.038128781106029554, -0.07087895248509708, -0.09528867665114686, -0.05964320366957784, -0.07158474052541987, -0.06046216531975093, -0.0224993530644505, -0.05256733970207618, 0.1577194192710188, -0.11850880212533554, -0.038953913021707955, 0.003017261854448211, -0.09310570751004801, 0.06145582552151685, -0.016281988798412934, -0.0550671902572124, -0.04781157161177016, 0.2003699996856521, 0.02662007094933188, -0.08370782177785605, -0.0748706255024465, -0.0673183137392374, 0.16566774545102214, -0.11201471972620528, -0.041349701934900514, -0.1566614352092577, -0.11422034179040533, -0.027166874171674096, -0.10219882964696173, -0.14740964302949477, -0.027924474911485326, -0.05619114793817676, -0.04289976979159607, 0.14008820702502728, -0.0841156656626105, -0.0703568235503839, -0.03653123744336197, -0.08330771345846624, -0.1543826127021251, 0.022382329441433924, -0.06141199324100484, -0.02173837876103404, -0.09521794946928981, -0.09941003582793143, -0.12791847417767027, -0.03487885776808544, 0.029059148046224358, 0.5920951637461059, -0.014447628958620042, 0.13517405324644957, -0.07611641549201043, -0.05146704911554045, -0.01946348250926406, 0.10676809828849317, -0.1967943910017797, -0.07909273686851906, -0.10890099025851194, 0.010090885025912478, -0.10302170843290262, -0.06267605413587027, -0.09956967402064415, 0.01690999538230466, -0.07020856138298476, -0.08497007918067062, -0.05340690101949835, 0.07083156747290487, -0.041368171938447745, 0.17165416755364252, -0.012062806522033972, -0.020328598883056832, -0.07556050345797691, -0.04526873744778281, 0.018558430664512324, -0.061999819003860784, 0.02914512644325999, -0.045666544644539786, -0.043934553249828444, 0.08658061831159417, -0.12449046048081898, -0.10476636102326868, 0.006501583139418173, -0.13522929207976225, -0.11174431230637044, -0.07071639993192091, -0.0054986912086328055, -0.0490535983490695, -0.03390901750895921, -0.17552948168929836, -0.0665505001475061, -0.1340149829115096, -0.12933601213108364, -0.15941893508234775, -0.02915019548395862, -0.04155305539048789, 0.20852338743568322, -0.11702004682235648, -0.04626473418977322, -0.037010443431811625, 0.05160947320332042, -0.0899171000509959, -0.035066301803293586, 0.0500745009690797, 0.06826330990690789, 0.09675277554207408, -0.10237084282923858, -0.12457943327580479, -0.07388085806279245, 0.07400230530600754, -0.03787381965543503, -0.06461720825622291, -0.12425187375488664, -0.030256703850438445, -0.04263048677584027, 0.02836052889655556, -0.11522787440518677, -0.09752274392481125, -0.040719471424965015, 0.15522268722475388, -0.09686078112014046, -0.03317958280123942, -0.038903298704325746, -0.020115020119821143, -0.10479641381460249, -0.046722121044711504, -0.05889778185641523, 0.05524685141226298, 0.05024266792557882, -0.024925782466171317, -0.03823470317316581, -0.003562946563921247, -0.12439298740083098, -0.05278671020172816, -0.04085127111731736, -0.03660770955751842, -0.024321529045680593, -0.04472734246789088, -0.09203698953483858, -0.09536512737928805, -0.06353029740159699, 0.0173693966305595, -0.030010628146141853, -0.09330895642977985, -0.12998522245720373, 0.01966901737105013, -0.0053730091816996004, -0.0420701367929998, -0.04257817091397308, -0.04915125586659071, -0.04849224968667403, -0.1108112776734657, 0.08959090345436801, -0.07591773146586227, -0.09116014570865288, -0.035952668494336745, -0.1796261393773729, -0.08938338090206108, -0.09411608838540063, 0.020496335716464947, -0.00742742609596885, 0.06768211374470211, -0.05208245757989025, -0.11107043135825312, -0.08421914525946961, 0.01526599924349722, -0.05245003055113736, -0.03614110787903443, 0.09961828378805605, -0.029266275532151834, -0.025407046801106933, -0.04373060005026447, 0.009370764840449769, -0.020305975483177197, 0.004722601578069483, 0.018663678813562033, -0.02101303181381677, -0.016459614627990827, -0.03337599739004804, -0.08754225568040254, -0.11347617244020741, -0.07221619427869917, -0.1548072177522591, 0.05890737248703631, -0.04810224415502861, -0.13742402521571767, -0.08925241272027393, 0.03567021612071249, -0.033385881479503686, 0.09462444035495124], \"xaxis\": \"x\", \"y\": [-0.0560315979574663, -0.07852365418078935, 0.027672108330283635, -0.17371171773773852, -0.025178071136929426, -0.11050887038274901, -0.05074489211668846, 0.01243673697589805, -0.055018636772543025, -0.06993623157609735, -0.1254628276540634, -0.06690403231922677, 0.0077153585492032994, -0.3836170396672197, -0.06078454407542337, 0.009427539299013405, -0.1801901127570564, 0.0018769516905646482, -0.16948559771604255, -0.1001861576307076, 0.00171241956687162, -0.012592768176327696, -0.036336661036739125, -0.03594441675179799, -0.1261453693709886, -0.013625241277529258, -0.11715818881964099, -0.23701822186059165, 0.06591496151850458, -0.09289950026413166, -0.1128280486805457, 0.02380258852940384, -0.10593631807842668, 0.0013353704734431033, -0.15122147685921783, 0.021814924767502235, -0.1547396722444952, -0.25253557499810364, -0.1446179494919538, -0.13923181567488116, -0.04701691506271131, -0.052190252811803714, 0.008065703260079653, -0.04944301790643321, -0.10362660992450556, -0.03292092770878964, 0.041511349645571446, 0.03775256966901666, -0.02260855293273938, -0.3116616273790657, -0.03878697218504184, -0.16419150582541941, -0.06893794637584075, -0.2709808963378972, 0.122413019682667, -0.08293564838961523, -0.057512262332064995, -0.16768633254251447, -0.11233704681524295, -0.12843272524607233, 0.017228403144149414, 0.03415941855583329, -0.11208459935662607, -0.0314075127001857, 0.0003112763161696384, -0.0753174095096392, -0.04912937728243726, -0.2179683245446524, -0.04898952308617522, 0.031893118247938126, -0.0974350889689182, 0.009114896599129663, 0.02269725006371913, -0.08230297485023494, -0.007631523246291521, 0.012014244643697504, -0.10405164148379895, -0.14040392988325792, -0.09295242877500104, -0.06347544106185456, -0.21141305675269015, -0.1062271111349633, -0.07843320653624372, -0.07715145091330605, -0.0150999676503977, -0.1385223479042801, -0.01507989137161849, -0.062332980546860935, -0.14085420270897267, -0.0027717775699293906, -0.02255843199006862, -0.12112152213730731, -0.12922901482790652, -0.07045944218248938, -0.005657770986852453, -0.008629095375871502, -0.07716127816648248, -0.222375045395846, -0.04869586155070224, -0.17757490826259803, -0.050692981858418265, -0.08828802257251381, -0.09142771049484973, -0.05365597190290131, 0.002745272451115263, -0.1862225298442545, -0.04533581216408987, 0.03659319441100033, 0.11126962412353342, -0.04267141604929398, -0.10358488433499281, -0.06759714282687695, -0.030582474638288823, -0.0599679887377641, -0.06596757069033325, 0.00666588183428227, -0.15434361134474384, 0.004711779962745865, -0.15269886532404348, -0.15409480538494974, -0.00699153091525242, -0.050303761963245916, -0.12942239271482178, -0.006982228523986558, 0.0221873324950835, -0.2952944854294408, -0.04560204920385845, -0.06964959725598362, -0.036691417004523004, -0.07186071943065023, -0.027983830696742436, 0.011373885150768322, -0.07116455280180885, -0.04425063445866016, 0.023613282115845576, -0.12690003695552854, -0.036519254583744803, 0.009249987709413779, 0.021350443748178682, -0.07922842051952425, -0.08654009253364617, -0.11572029284297704, -0.2527499892646602, -0.05036622538421053, -0.021901903361060286, -0.12220477219046247, 0.05036270696100526, -0.17095779509386674, -0.07173379605287947, 0.03184025252712365, -0.07175222418569874, -0.07698796268195324, -0.18023927347402954, -0.025022209897681138, -0.08580531632518719, -0.19541155637310978, 0.057188326097754955, -0.08384748270408947, -0.06353637836058841, -0.05392751973172021, -0.011166056844819642, -0.12285982983531436, -0.04908540714506753, -0.04164851838184037, -0.2088381034243654, -0.11997394209948688, -0.11699892883034274, -0.00837904793648332, 0.008068097235733434, -0.2027082193599833, -0.09023539953651553, -0.03157303288517043, -0.03069206641170874, -0.020829161766834824, -0.07414174639835645, -0.04117908087885735, -0.11058287080372453, 0.03257042977099986, -0.0497992921879638, -0.11227748928307194, -0.025525828699897266, -0.03655589580659657, -0.1510827539144741, -0.19737149430107093, -0.08669641935428869, -0.06042827504809416, -0.2056353434432676, -0.08312827592451429, -0.06458416218107844, -0.058749466186367276, -0.07415849927763081, -0.08183737817138573, -0.00037092675973590564, -0.011230013339937768, -0.3682012209763449, -0.05547101369427276, -0.24499590635533428, 0.021725156209163567, -0.051943485920307016, -0.344729313800759, 0.0347349331905608, -0.05471959425196944, 0.031880279174594385, -0.12193998124683383, -0.08724791789793493, -0.2361665227206991, 0.006859283393130865, -0.024669351172872646, -0.276528602780762, -0.10774369450766873, -0.19765608992997172, -0.048748030931068845, -0.06673634041988406, -0.09509584575159725, -0.11119732357941002, -0.11684296087599287, -0.016289664415430494, -0.08786543417809065, -0.12888109155840258, -0.20204417193090268, -0.07455279963114611, 0.02225414598758372, -0.01949839050260649, -0.04460180239197035, -0.07693275843671968, -0.06826689198660695, -0.17441636657059614, -0.04365190825145517, 0.0576012319654172, -0.005967798734602499, -0.020228225641168277, -0.20236363198840754, -0.07188015908165156, -0.08855219233217787, -0.1231190772937874, -0.18512995238106347, 0.01617472988531693, -0.0027506297515660937, -0.0036222047605986704, -0.009128232919143288, -0.033406949303062046, 0.03530083952767648, -0.05868544261734489, -0.04074202895202882, -0.008358382031992673, -0.06512997338192987, -0.012833033185597805, 0.0192958007848213, -0.021228355115988078, 0.027080770199070205, -0.029451938488320874, -0.05271402174093455, -0.07420685030272263, -0.13567227765127723, -0.020535243285813985, -0.2670635098118872, -0.06489797498001004, -0.035941589467816705, -0.20071614797383078, -0.1286722150141037, -0.19160628108691002, 0.027467068684038535, -0.24738154307993862, 0.03772053626564646, 0.01906131084401295, -0.07282850300779321, -0.15977556065714824, -0.2419810522181841, -0.1379140379815789, 0.014191824606431444, -0.0012812565617066748, -0.2146529701418949, -0.09452117167551878, -0.13436041773295274, -0.022509461912967568, -0.04652656385987308, -0.07759658275719236, -0.006848059852196636, -0.1817118903166194, -0.16842017910310098, -0.0365627789801699, -0.3291812423271987, -0.1867997892350392, -0.03596196454883204, -0.09062805198002463, -0.0634360844645315, -0.20764553467062827, -0.0748878229602887, -0.1582049289015203, 0.017808949197354103, 0.03846152026708096, -0.13273528114036465, 0.009279933398008616, -0.0883340329761929, -0.01946216129374889, -0.006785833677404697, -0.09989936598916663, -0.04707690586907735, 0.07529999031071903, 0.005110262121709034, -0.165914666171892, -0.03610138940733078, -0.12185968904811455, -0.020867759452008944, 0.040801122091180315, -0.03345986124612378, -0.03777480699184537, -0.014616077092606143, -0.07831452978420511, 0.009494643065778963, -0.08375404474946604, -0.015146966514112306, -0.10480835829018323, 0.0080015658528166, -0.12002155135656074, -0.050916015200634344, -0.037887227024024, -0.2247837914275637, -0.12382382350468046, 0.016947315225102382, -0.1253752491740807, -0.21094668503322514, -0.06265205957947398, -0.019614082754116535, -0.0883572593291221, -0.0677376203289556, -0.14921673071553498, -0.0265371875577323, -0.020193032529363098, -0.08794997667158784, 0.01257314878317474, -0.09648099635027939, -0.023301679525417673, -0.004585718160972344, -0.033267207820001796, -0.02493125512033893, -0.05883591238903598, -0.09370398184548118, -0.1265235763976385, -0.022260832040264415, -0.0027837475851366276, 0.007299194070454611, -0.17008498652979054, -0.08039145343489265, -0.05685832028047046, 0.01217980579275933, -0.13772141915158814, 0.009192377264890056, -0.05126783983501303, 0.08607633336604653, -0.11996147609521286, -0.11944955725667453, 0.025376742865524948, -0.027618342352939754, -0.2261018492042249, -0.13248460363382283, -0.0010042300987942291, -0.0011201188873281353, -0.11948681911711291, -0.06868319396018019, -0.022835735983299632, -0.02554830761299371, 0.07584355408282721, -0.18493107154099792, 0.03187111432914156, 0.006030915310321881, -0.06702812727237624, -0.062151920238311335, -0.09857531794183726, 0.056436537322881127, -0.007870396632094602, -0.04326312777907261, -0.08589194119346479, -0.2926628482793179, -0.09859898294852344, -0.03764428631870476, -0.10744212633958514, -0.06416677327470716, -0.057874761847726504, -0.08301112317570067, -0.07218155506831521, -0.0494889204593074, -0.0011671123385148426, 0.029078067786903757, 0.003880560503213234, -0.09430420529231365, -0.042203702322648244, -0.0011121853951311447, -0.04989247739128623, 0.001517788797944971, -0.061745190754531123, -0.02538447447938384, -0.07439126860986706, 0.013533771664864818, 0.018022337039171547, 0.018662975856415243, -0.043443138125704606, -0.06667600056725534, -0.04429567145829598, 0.004123740303847573, -0.04982439304299314, -0.0731239812304056, -0.019410051662444593, -0.1991247916158023, -0.1364481809493271, 0.03682273797440711, -0.0222490470850891, -0.09355405978609346, -0.08377266392586394, -0.1129119186566958, -0.03182879348944365, -0.14351149737107138, -0.08003715178648355, -0.03097743988687772, -0.09274318840147279, -0.2539901056711881, -0.054027714048739406, -0.12879045324876165, -0.14427103953146816, -0.22055574939831837, -0.139717803136682, -0.0463606373830928, -0.15713607380276048, -0.14620959481474363, -0.10697943635574934, 0.07648581887233337, -0.015118336120944972, -0.08570752374431019, 0.008789320419224651, 0.03881251485170444, -0.05129419354512205, -0.03253206793897377, -0.09939485605767977, -0.21506001317434917, -0.0651957500429665, -0.1877955720205469, -0.008958129708762107, -0.07003050039261687, -0.13701135785348023, 0.0377928629870319, -0.11564471859486876, -0.17304911794469324, -0.10777351736341559, 0.10077517781567644, -0.037449123007427025, -0.1059841166362018, -0.07384271529786988, -0.1571090616563235, 0.028085518500718037, -0.10500308428907434, -0.16125703779366063, -0.05812000143352315, -0.029143644487543183, -0.08568662477827309, -0.08248912809251924, -0.06901941520855759, 0.0022773419701372965, -0.1947879806591209, -0.03505626852533662, -0.012485431250457202, -0.2882380411916062, -0.11378026265426669, -0.14485932386457429, -0.17250045519791005, -0.01815006349907869, -0.003545593833277653, 0.030673956137406713, -0.09288272409997873, -0.012885273670000901, -0.05409483581677306, -0.14748585301585293, -0.001780757648849538, -0.0994246918669878, -0.0386820138082457, 0.03881251485170444, 0.007094695634184038, -0.09526543877785476, -0.06126732648171641, 0.07337407876693919, -0.07797791823800854, -0.05638521336702402, -0.0744603485740359, -0.2164055382135406, -0.10496144798502298, -0.17428936832555164, -0.06652499675993409, 0.030847278531052867, -0.078676255281776, -0.14167365562456422, 0.012433146427706054, -0.11963425007210635, 0.0037133889110121093, -0.06449113117711831, -0.021355445197316743, -0.05793072938815026, 0.007189502478872706, -0.01606645153726641, -0.114148015004747, -0.09488034689916988, -0.06174173346025478, -0.16248986337862054, -0.07482741338202206, -0.10446435927847028, -0.06276997580268272, -0.23498789545019053, -0.22376324287178823, 0.03252958699584069, -0.06511850636668094, -0.14454106712480697, 0.018096142617179293, 0.01067653146226061, -0.09692718502847014, -0.047942405268594925, -0.09837405639166352, -0.08533815141036682, -0.2361906643408316, -0.030835223263392306, -0.011406214256684869, -0.08809435736151604, 0.027255257098284663, 0.028214209636092613, -0.13858460870375067, -0.05375256120876702, -0.12698255395126434, -0.048708555972048324, -0.028994947466976815, -0.07820514115492778, -0.025202090302389468, -0.07591805742956317, -0.007402514318995991, -0.01730816011659245, 0.0198259982593005, -0.029233017189426808, -0.25227356223904657, -0.13917310096176594, -0.008266586782087268, -0.10860616369643694, -0.1020375294703301, -0.017832645738940226, -0.008376025404360481, -0.018172955248697113, -0.10827658913604282, -0.05068721594024576, -0.1245000049270717, -0.03801143605511674, 0.09172469089085723, -0.09563129086710205, -0.07529487557525238, -0.06254973483958218, -0.18100143960141404, -0.06760525349649761, 0.0025401168136436007, -0.2026891963791708, -0.1955626821890915, -0.05777721551268336, 0.02114497123491257, -0.045701478531434185, -0.02158405454553765, -0.05541043684147965, -0.043112158928743774, 0.027127685215207864, -0.05647614905816557, -0.21767081172554756, -0.012034185459042572, -0.0051165277494048065, 0.02244152817825631, -0.08046885079700491, -0.296029667925713, -0.26666101226013744, -0.026680075003653977, -0.01806606638097275, -0.024979268951373996, -0.05708423700499897, -0.04288977575972457, -0.027196996959682845, -0.10403931308423704, -0.30186203832224145, -0.04542061587874696, -0.19815705766523764, -0.14212954581150578, -0.18109549164370048, -0.030168396068595248, -0.14457542479340085, -0.06417424191867872, -0.1044842282195864, -0.18830593461167341, -0.1767249443132375, -0.062277599937482414, 0.011671694642670403, -0.10804600772629569, -0.024664823656102074, -0.019325558442307794, 0.00786117893876163, -0.0032680047004895124, -0.1745777214391043, -0.01888799708568301, -0.1675771165929485, -0.009903969773904104, -0.11570853674643927, 0.018059020151462536, -0.03742844609898743, -0.06010185759970297, -0.11177203552155988, -0.16776254499806662, -0.12588057195847388, -0.17511669117729925, -0.3455433882056439, -0.018112600507214017, 0.061993556125453, -0.006198377392476996, -0.09014641922540888, -0.2298592201706897, 0.007073481151145442, -0.11643682423400942, -0.16471077276639873, -0.1432566910132217, -0.24385057672045923, -0.07351175396346585, -0.03437202065777709, -0.042742364459853535, -0.11433352303273016, -0.16750846506774, -0.005344686392154049, -0.17094993802865976, 0.007519715447475951, -0.03442167315830652, -0.025407860057062314, -0.10558182325341978, -0.061154151581893246, -0.15768673042733558, -0.025729005076505388, -0.029649441605550372, 0.035729099511696866, -0.07542188530643758, -0.13330335557103573, -0.05037433741039347, -0.07841376428753756, -0.03243356262416488, -0.19085958676659542, -0.021054736907939645, -0.06204340707836662, -0.2420654324266105, -0.060942645601705446, -0.026703866895472595, -0.011708505613155193, -0.02617213908991918, -0.25160129573285334, -0.09640993873120773, -0.08296074676982702, -0.16298637265849725, 0.008579056671952063, -0.14245267054606356, 0.022615639062357057, 0.009386933023747194, -0.09750598886282204, 0.0062770702650800266, 0.0047698723542307095, -0.02568351384820229, -0.07190798756437483, -0.05320513382886358, 0.013330120259597756, -0.14318731644462268, -0.06547634149773003, 0.029677627867807088, -0.0627344123474153, -0.1853316113989809, -0.17660451809015384, -0.08292615099779062, -0.07952378650671796, -0.03148741092964721, 0.01096099834830665, -0.024059944451297464, 0.021591179318900464, -0.06870314076678306, -0.05312229930629163, -0.08806946237185118, -0.04967137411680025, -0.26602976897636504, -0.27503580295682245, 0.019657381210261622, -0.07161734253422446, 0.030458080574691244, 0.049285311813085694, -0.03952203394035628, 0.011091908091683136, -0.09053287633403041, -0.12109656439206119, 0.014463857596576684, 0.03474654975033881, 0.06491629123839851, -0.0335562894098584, 0.011476574544202365, 0.01249993451380248, -0.03094463281345774, -0.07295564053072232, 0.011688387127955568, -0.0899457872127109, -0.0766899028529632, 0.04619432750374564, -0.104890488005848, -0.09090111120914876, -0.10715874085830737, -0.01836689240292592, -0.05701885132633687, -0.09074857000087079, -0.01152747866127027, -0.050847652931993184, -0.1977351046409984, -0.0033816650756694636, -0.16957014112816637, -0.018786756386627083, -0.06286792656536076], \"yaxis\": \"y\"}, {\"hoverlabel\": {\"namelength\": 0}, \"hovertemplate\": \"cat=\\u6d17\\u9aee\\u6c34<br>x=%{x}<br>y=%{y}\", \"legendgroup\": \"cat=\\u6d17\\u9aee\\u6c34\", \"marker\": {\"color\": \"#EF553B\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"cat=\\u6d17\\u9aee\\u6c34\", \"showlegend\": true, \"type\": \"scattergl\", \"x\": [0.02415329977205472, -0.10677944213637985, 0.003888065427254825, -0.030876236188153484, -0.07510853749125106, -0.008922272115162916, -0.052634260484167, -0.020261489333105866, -0.10746837614609514, -0.04847514201118354, -0.08893277430076016, -0.12402050394808499, -0.03485281867220487, -0.03355128241722638, -0.03759143251458817, 0.021292351314530063, -0.052908154561807655, -0.037894175455055044, -0.05773925590178017, 0.0017313929175520018, -0.0383281764349479, -0.05623889174637782, 0.0394806118432948, -0.058906364266006116, -0.10727076036661967, 0.2656633261950162, -0.05239285126494742, -0.051156019089722574, -0.017252648044387794, -0.011484290822299225, -0.022144944286730115, -0.021157966816563074, 0.02449054234753958, -0.06422510687513723, -0.07400030646129625, -0.03035091372229078, -0.06599412149054194, 0.00046393763197546484, -0.09555916750779828, -0.1180520411781864, -0.021465117301315135, -0.056093200083805826, -0.06383796986182916, -0.08316700475305777, -0.024875947655974427, 0.010379677643497178, 0.05330354685327187, -0.12582195352124345, -0.07719452735194399, -0.04728965230209502, -0.042901801587027985, -0.021144721258360435, -0.030856698892314207, -0.07213191133803973, -0.09262443228040738, -0.06676982348896092, 0.07373724876857608, 0.033880280391248664, -0.014427405723369905, -0.0510394514510572, -0.06306332372413082, -0.08198119712504161, -0.037542145597541134, -0.0552794114433355, 0.019265878414504025, -0.06021462841182305, -0.05323706805659475, 0.023402864259550522, 0.07924281987301644, -0.07415032209839097, -0.031574986768346615, -0.10989667483693447, -0.03390766156835956, -0.020468252090220337, 0.03342586587829781, -0.055460822813773396, -0.029857968576297588, 0.12164244211506228, -0.03066437474727548, 0.025198653606106937, -0.09558661812772651, -0.040547246009394924, -0.06606386209736612, -0.029221060324551296, -0.006029447439236462, -0.02481359431432341, 0.020616609401440125, 0.08354201638855692, -0.06081857355395206, 0.048834216118313255, -0.014496802830199019, -0.04549869206504621, -0.10054630820042146, -0.05005055993843267, -0.005991499230453761, -0.051928922859489905, -0.07527906726654345, -0.115016677393589, 0.12121769440897986, -0.01877485563855039, -0.08937473761625306, -0.04162715597905164, -0.07149114661696576, -0.06380274454846584, -0.1482168707852017, -0.014860401055775038, -0.02323669843007064, -0.01661388234356965, -0.04560919294300134, -0.0022134428315604884, -0.04793341986837338, -0.03896724456389239, -0.0394779505433253, -0.03756740495669209, -0.07292733312395966, -0.05591091328438309, -0.04510032503567081, -0.07044609249474992, -0.06773204664366372, -0.01054953804898221, -0.09347471092968428, 0.002459567203106742, -0.0771137570473524, -0.004806190391017812, 0.029482689333066022, -0.08940415161011324, -0.1477449835476396, -0.040834908603083975, -0.00437722074954283, -0.07023572350947568, -0.03816950233618851, -0.2000312122224593, 0.014100611145719205, 0.13283343495432964, -0.030023579476382057, -0.02800362216289958, -0.07754484963267927, -0.1116813836501302, -0.022275794269256778, -0.08081793491286805, -0.061481323022521614, 0.20161241573452193, -0.039285767134133, -0.03368353347733756, 0.09207831936955521, -0.0952250538937133, -0.06980686402210848, -0.07216547965044239, -0.06488196038227659, -0.061478137796699334, 0.010442537389135519, -0.1073813513048864, -0.05629269560932108, -0.12857442244192493, -0.06253460605548913, 0.025955993041708196, -0.038118975561341265, -0.024957116189310517, 0.08325973869019551, -0.06972672649232385, -0.035941493286426276, 0.07269314935988425, -0.04095916972852507, 0.0003068944324737506, -0.07967534170189815, -0.01843920974107349, -0.045876217222417796, -0.04489041239254691, -0.02340274762853165, -0.01986762138130456, -0.049343703540505796, -0.037656774253665835, -0.06563801012832055, -0.04909073469716441, -0.011804132487195132, -0.07357472322833206, -0.09981035266309839, -0.011562019372941495, 0.1900455348017372, -0.016497318714967162, 0.00498878483399512, -0.05335981873910533, -0.07305530693944588, -0.021421143600734376, -0.059722327302532206, -0.033022826678829754, -0.04172488935355256, -0.00506998767493999, -0.07781476114273646, -0.07397873780564389, -0.1474395079882587, -0.03548873494670194, -0.08440635748210279, -0.06523208861884297, 0.0034376557591527675, 0.09789225021102749, -0.05369536165184451, -0.016005160331377576, -0.01549483356757144, -0.01850308574085689, -0.09644253973111032, 0.18891313912306512, -0.02700337050331596, -0.003502163981536938, -0.027933216760702973, -0.08607604167260431, 0.055887163366514024, -0.05604669986066584, -0.028771973939384802, 0.032440361595456144, -0.09280530757962635, -0.04226083087513571, -0.050093993348544634, 0.04496560587080021, -0.054734416828074346, -0.004321329200740937, -0.051079710295426836, 0.12683998062650645, -0.10559852825998516, -0.010665623415839256, -0.022110175532604262, -0.028243555309152527, 0.046763146815181546, -0.020557810766467486, 0.013390138716182705, 0.14186366288257654, -0.09286506077635334, -0.024424552690185033, -0.07240659252560129, -0.08313718114013884, 0.09635137678139263, 0.12330987482822364, -0.050648713887753355, 0.2497471238894583, -0.035983927684890116, -0.06761976027553243, -0.05801203112135309, -0.026113798706761043, -0.056771623012782664, -0.023874303486364726, -0.02277898762136689, -0.041773725129252724, -0.030107200641594944, -0.07300840367273526, -0.06485341896335305, -0.01653916257178847, -0.027519436229396455, -0.027367722182746957, -0.030816281810345694, -0.04185346747377329, -0.08381774250426556, 0.11706613161655818, -0.005919679791951667, -0.02306529629129697, -0.0977879268560033, 0.020997016472902603, -0.1043846433732964, 0.005805583634009732, -0.03004251853966626, 0.11519588416609591, -0.08206811918567658, -0.060194983512658555, -0.08330224271072874, -0.07348327599467251, -0.01673468010262602, 0.09260756466120569, -0.09010905996572115, -0.12991356942915808, -0.03169564472863954, -0.03238227674869349, -0.01742183031785441, 0.1390361671802394, -0.05386656442520383, -0.009354559067371595, -0.09123802812702128, 0.0021357767399165337, 0.07289323894525734, -0.02540878226759495, -0.0034106542834367037, -0.08138080132920303, -0.04526139964945801, 0.05316431118550644, 0.09753643044853703, 0.001804489580669696, -0.07980193955497072, 0.009306487808944492, -0.055403829318394916, -0.040495777791011364, -0.15729835554633612, -0.017728624892516014, -0.03182769947938746, -0.07456992697623767, -0.03596157013914913, 0.08531878432659419, -0.08684019078357261, -0.06396537045270644, -0.08142093584504634, -0.11682202561729468, 0.005897156315580785, -0.020078813767163143, -0.058347557869624526, -0.02134690061231394, -0.09113085075432058, 0.04953341868130158, -0.03030985617206465, -0.04666724494851749, 9.168899259036177e-05, -0.03901979536259889, 0.000969649280785319, -0.002619757470826245, 0.07249697547336943, -0.019522978897612023, -0.030363791329381377, -0.018947274681977648, 0.001195173103845552, -0.01881821801288362, -0.04071687187042988, -0.05033829563854631, -0.05495604579395904, -0.09497612583318361, -0.010643762328039173, -0.027890647522477944, -0.005170785965502586, -0.017286403502466925, -0.014271544839661416, -0.034055216742838575, -0.028980242383990877, -0.046137190109942, -0.026817935654610457, -0.05046517599947705, -0.030019015890235742, 0.12492356327918995, -0.02502413870623432, -0.12050740199097447, -0.08939319143980419, -0.07112362776007894, 0.07527495216380024, -0.04310226745332892, 0.09846161941046147, -0.014231770094194133, -0.01816974995646757, -0.04217276390464083, 0.12496078185955645, -0.05779298650133669, 0.19147067893339637, 0.029983225581197388, -0.015440962208127006, -0.045093942905752644, -0.07319891668993318, -0.05093129587568894, -0.01518225674287973, -0.009984842146515023, -0.064428671856129, -0.03988204692273854, -0.08240342050885438, 0.09434770628362062, 0.026925844895082093, -0.01509478812158582, -0.15429578271345296, -0.04423257088967927, 0.09251201306722455, -0.04135890379612377, 0.12148869796038968, -0.017472185728388906, 0.0337917044162835, -0.09261367149610474, 0.01858839731437371, -0.11491495339857082, -0.06525596085716831, -0.06217007311810443, -0.06126211558834578, -0.07842367191380924, -0.020388489423243752, -0.02317690811532964, -0.0268564203361237, 0.021710223336243235, -0.07924645389420905, -0.056339679358975575, 0.015398906899347033, -0.02108594608906573, -0.02512993567154584, -0.088833897545268, -0.08955950203901297, -0.02190669556268454, -0.06668851809763841, -0.05364388067480157, -0.0563026861097374, 0.045480359207475465, 0.009240633570189463, -0.02269684747414754, -0.1179133900522281, 0.17624091196616065, 0.01130503599062234, -0.04928767302117602, -0.08509786250512014, -0.08342634878503415, 0.06308778373479827, 0.005423868502776387, -0.038304033514133806, 0.055873378003756254, -0.06810314395946288, 0.08176488385122083, -0.008120159264852923, 0.10249278111822192, -0.011683550675945953, -0.06012047295792063, 0.07239018284791496, -0.048656333331733595, -0.07293029535965469, -0.061167852291642656, -0.016878265549561462, -0.04802071569705854, -0.022152304830219866, 0.16504603570906737, -0.04496371111542667, -0.07262605588968296, 0.03065474437690901, -0.08655266830722254, 0.04655435002105945, -0.04831927735207882, -0.019223303180033464, -0.0364020625971142, -0.05942052257939232, 0.13642896366420973, -0.019596131395818835, -0.012589359111174299, -0.15036731392789904, -0.13363517240168446, -0.022980336537332804, -0.007006781694893177, -0.0884248920044392, 0.02672534990815651, -0.018511210096444436, -0.05907710183170294, -0.0532923824257885, -0.004418050642357223, 0.07298422950258604, -0.05479592498059548, -0.010336940345180203, -0.011971110114784257, -0.06966110363930739, 0.031864447432801146, -0.0072642237934443625, -0.10731521136365045, -0.04449683312292586, -0.0408938481722749, -0.09658062667462625, 0.15353693927738754, -0.14961254130725235, -0.00496442779873586, 0.06578638027784942, -0.11069290902734803, -0.03461646606647831, -0.028650513654291186, -0.06038221704036398, 0.13197051898650164, -0.047313817686503355, -0.06668663577901751, -0.017794721564655218, 0.05139374728249991, -0.07762368818317159, -0.001566975807222653, -0.04746277973987406, -0.07239091035600514, -0.011974008769060122, -0.06634938422129093, 0.0940681520646036, -0.0019597057554454123, 0.05024841710546535, -0.056001956424210214, -0.08416046483729953, 0.008278061250649909, 0.014347906442967406, -0.04283985803344131, -0.05215525392045059, -0.09692780844826467, 0.0006192213285246486, -0.021767230873882524, -0.03376925021492424, -0.11986429640605863, -0.054637545806774684, -0.019282215586797033, -0.02953051520342418, -0.05882814013365672, 0.058730723485333654, -0.03159869414291584, 0.013469516898991984, -0.014145883438539682, -0.03919458232275365, -0.0350393054215711, 0.0631892931263815, -0.05030205966344309, -0.05234775896404357, -0.057057675909379664, -0.07915586771261934, -0.057422342196273235, 0.032425875496993176, 0.12656704600386742, 0.04378253728431059, -0.04463523197156736, 0.2301453647282284, -0.05395769484837065, -0.12575766195963003, -0.1094990260381142, -0.06135169823593629, 0.09252396695141878, -0.016125611561259565, -0.04323275494953939, -0.05078741013059994, -0.06207950963731414, -0.0693135686430406, -0.03310125848831602, -0.05998369989836083, -0.04098460744433922, -0.1116850929780764, -0.035326749229199365, -0.03899290866366594, -0.04789336554907352, -0.05076741766938838, 0.07950454439115913, -0.05506005688695036, -0.04615619926822698, -0.07973324274187271, -0.04157933628860834, -0.011200505936034747, -0.03246091396261476, 0.06930205748999588, 0.028922242342134755, -0.02050697068692121, 0.010550347115307366, -0.03643426310138275, -0.03648897836838233, -0.015866101381355878, 0.009222592823642697, 0.23064708839133002, -0.06775967310319637, -0.01710227459981375, -0.09356805009579137, -0.00787861205006641, 0.04049644883010845, 0.19313415638733333, -0.07013733161734408, -0.09237676803865802, -0.031863294073162, -0.023732095482619318, -0.12615143497938772, -0.11146332842568875, -0.045573173225019796, -0.040997773600099124, -0.03785899817736198, 0.028506845975701647, -0.05233352758518622, -0.05993853938193171, -0.015889370129403176, 0.04939691113045645, 0.010931360580179583, -0.12439090247037694, -0.09856476084311899, -0.07042431755659329, -0.016188810909035733, -0.09332519111650407, -0.02787109213604012, -0.06232862375297552, -0.046521319265650576, 0.05262592012691085, -0.046176296428458476, -0.03853250813202597, -0.03734855270729889, -0.14624690897040138, 0.02232718039608877, -0.06793243234103505, -0.04567856578817038, -0.04550442509800637, -0.05031755610516636, -0.06885798051742303, -0.06294919012496614, -0.036535567388438085, 0.0600886053631801, -0.008294080368321795, -0.036252701360224585, -0.03434896997095022, -0.05316421478422375, -0.056657921871062875, -0.0713800338700689, -0.04517240695140008, -0.017485418171510468, -0.054324613979306706, -0.005745949255231788, -0.038710637777663495, -0.03771170915972763, -0.009518514689046362, -0.05164515937195789, 0.033622179146232146, -0.005574090115609488, 0.02433778595807677, -0.025207431564263232, -0.06479798056679707, -0.03617257561994495, -0.08473773587351517, 0.023070778388209704, -0.016564864577177024, -0.08192260281383336, -0.06806247945830322, -0.04427472523024787, -0.10869883034205907, 0.09261811624219526, -0.021278015990552648, -0.050065052999049946, -0.004866481252974527, -0.03774515845670896, -0.005750312254706303, -0.06480805324913304, -0.021125441431379854, -0.029342161536255938, -0.036089884993268956, -0.07463020022348758, -0.05117831589978076, -0.010825062459203758, -0.0017660599035080192, -0.02204679486183041, 0.0016698828415771289, -0.08922637389729692, -0.06159032725534795, -0.0977879268560033, -0.012701193271425255, -0.0016773776626785332, -0.118427183905515, 0.01676687404957156, 0.013957899535882898, -0.0379635125368252, -0.04650766120148757, -0.020346381732679757, -0.029478745640405236, -0.027227963808192826, -0.0260193018896233, -0.015185865031670054, 0.0437132862189398, -0.03279306513453775, -0.03789036652339161, 0.027956782579834256, -0.044877720325487626, -0.05237986440660097, 0.0017493925571166774, 0.16656934733947026, -0.06954305417840204, -0.012849037196812829, -0.08816974082873377, -0.027049800317856865, -0.06349837434490431, -0.16992344637593126, -0.09433162244770436, -0.12603344471887476, 0.016741536714080966, 0.1213438642305583, -0.05633674689655862, 0.0010722289437762077, -0.03085311537236602, 0.003298801458623643, -0.036762912365475194, -0.022705444975240553, -0.10086302516363695, -0.04955423475885177, -0.08865099412085077, -0.10695359058424819, -0.04462640508641994, -0.0672827126133222, -0.0725913792337562, -0.06899806398646133, -0.07548752771444123, -0.0552961097699059, -0.03675810294642958, -0.07065901936559207, 0.02055468140321909, 0.019076357235784765, 0.16722440897510701, -0.058113686047697635, -0.0934570569480075], \"xaxis\": \"x\", \"y\": [-0.05604308636447672, -0.16388046449138513, -0.007410750190371259, 0.03323264200879534, -0.01673345744287065, 0.032393450096533105, -0.05513910944086011, -0.06313788384777269, -0.061217942190902506, 0.04006571613484294, -0.10409258270090008, -0.15506620255908798, -0.004128619352515748, 0.020164813510524338, 0.030537235341105497, -0.029004738565916073, -0.014028820819480939, -0.06642098417974387, 0.03858463024068967, -0.0039603102317458914, 0.025456455550529934, 0.018188977839445383, -0.08559278840548495, -0.07006167249990555, -0.2915937929322987, -0.15132038125221126, 0.02169601632551241, -0.02651131487658306, -0.020350883192709727, 0.04028334336067519, -0.07190488233696056, 0.04055095876750447, -0.05563845386805795, 0.06035579037674816, 0.13521448676242376, 0.016313273364270128, -0.043981648398793725, 0.018836678078096972, -0.10784037724840612, -0.19642975562885737, -0.05720587637242023, -0.18312157324772982, -0.11614854586892133, -0.11920384451490414, 0.04669049529403823, -0.055519819234586584, -0.08124460404370742, -0.22003371167000696, -0.07572198211092349, 0.030463965396963843, 0.020921259332149462, 0.019998198565164105, 0.026094729461455527, -0.08521231325871927, -0.019133826022832912, 0.12070156191647534, -0.11631800778240349, -0.0770117120611917, -0.039576140755260586, 0.010369364106618175, -0.028561970718276163, 0.06895398439936884, 0.01981522201465915, 0.0276754110916636, -0.13578874395317705, -0.09279494706788231, 0.03077110121953172, -0.04334178733871585, 0.01071555945339797, -0.06619091817096359, 0.027984263781040252, -0.04082012203358786, 0.029356342225863467, 0.033014363127381534, -0.09129460281740871, -0.09637785806267557, 0.03332436547125435, -0.06268914464015443, 0.046224736139748715, -0.019162414888894783, -0.0777307597462695, 0.02516304542472891, -0.024498662609116775, 0.022426966917330034, -0.047862741521467383, -0.026142927509065324, 0.010098682365841919, 0.0017794001042443226, -0.013494310354189545, -0.07790403678916896, 0.03592015147871247, 0.00532082277746964, -0.08168765072769703, 0.02259914876997418, -0.025415888899522943, -0.04866331062959093, -0.11233146177190276, -0.12764575483625243, -0.023219420496272847, 0.023621775141742272, -0.027185731795534233, 0.00595926088264354, -0.1472298467080354, -0.0799682379637318, -0.11123572410430345, 0.026063163948013644, 0.00846840785959316, -0.05954122900037187, -0.10551536172905653, -0.07906826199265932, -0.09831093718836288, -0.026194681328836338, -0.01836397610651519, -0.025376914502411967, -0.03388159270030174, -0.06429453990031342, -0.045557642052898494, -0.07257786292600857, 0.08374029056532664, 0.011827132996561232, -0.05897352369967419, -0.03448255959781131, -0.07268061418102528, -0.06751464255752429, -0.1864637094243127, -0.049130524139446095, -0.038241739605014834, 0.02317687851050791, -0.12348986830872584, -0.048378729985547764, 0.035919073300431904, -0.3924789448338552, 0.003976363244891833, -0.031522115284446345, 0.03462162563694376, -0.07375379402619074, 0.04659460290929708, -0.1588976593741442, -0.06928775304710594, -0.08192555048980493, -0.06094558870627703, -0.1838457670557021, 0.020528344991844694, -0.004646899083181492, 0.00047189421763666086, -0.08018111410204272, -0.0007659947372024807, -0.22023437685567213, -0.08232437771251581, 0.025249401822833254, -0.08028128390663804, -0.07508487695808039, -0.05039350537355872, -0.2505587583708132, 0.023389214137578756, -0.043452331221029636, -0.0008564261475669161, -0.11498631028360182, 0.014278635963292978, -0.17098714588296077, -0.016854383882991656, -0.008453976169512774, 0.0244952111368982, -0.0017298775017124588, -0.12539929740923522, 0.04556815209612321, -0.0019406841074882818, 0.020123760514680917, 0.025825665451353983, 0.03948891901831529, -0.1361577592456659, 0.03100837956552516, -0.03848592604421279, 0.02391825768706449, 0.10164725101777705, -0.16540593436032688, -0.11848888682497094, -0.18840042295457093, 0.004722065550230131, -0.012514455764689096, -0.08782850307264987, -0.04842807697894935, -0.019955974573599097, 0.02699050815984983, -0.16273072314906215, 0.03558759655596704, 0.030671985334907812, -0.2591421159297279, 0.019684239371120852, -0.0583050362797372, -0.09189423866126573, 0.05673715538195781, -0.06207950595558278, -0.025371861536730603, 0.01536118027492688, -0.019552381762478754, 0.06453879844686786, -0.024165724021012854, 0.026790696064048707, 0.034843500748461355, -0.02250470341037808, 0.0017641086734458708, -0.06916762804802307, -0.0023627210928588837, 0.10162522177366819, -0.17375764026828722, 0.0129792639669773, -0.027866655131163424, -0.1059343626070505, -0.18976638983230806, -0.18725046997855777, -0.04512941247968864, 0.004381892279383131, -0.1976182448849573, -0.053523479521440376, 0.024624098475759824, -0.03278652796736013, -0.11677333880001631, -0.09270380044333876, 0.018225521498801113, 0.018692233967799125, 0.034135744278848364, -0.18110395904281656, -0.061997473405700304, 0.003362896846875587, -0.003629366087906654, 0.011460232281428357, 0.012523011576156378, 0.019700890662364048, -0.05938070321652315, 0.08027980482377996, -0.018059978091165597, 0.025725651854103686, -0.029719999604780212, -0.10249225696919399, -0.049724141728955604, -0.014203123667548067, -0.01540338778197635, 0.023039236828645834, 0.01654265624592559, 0.028640673886405683, 0.0295942190717589, -0.015655499339157736, -0.07598540811275149, -0.14057783174905647, 0.02015512706900223, 0.012453450403643108, -0.034536730615531634, 0.0313617005677373, 0.021110968309547708, -0.11164530180993387, -0.0003201652915664331, 0.04131080563798057, -0.031040372413512145, -0.1527304632142332, -0.06142297670395603, -0.1190996817885511, -0.011826086052104392, 0.035800082189704624, -0.0917107622496301, -0.12118677395870282, -0.14574728166599146, 0.08640867842801857, -0.010069412406958274, 0.052391975202027236, -0.004727117315851111, 0.004173895232693762, -0.025404814690689864, -0.0181824999997824, 0.013041231324488093, -0.06896589667779868, -0.04141515001038354, 0.01061353288534563, 0.020969328464630985, -0.11709372904185557, 0.015948504359604453, -0.1588909788997742, 0.007772906590154491, -0.08531974929153721, 0.022861129172024822, -0.04806401848185831, -0.05476699502857202, -0.051858438599508394, -0.045887768081477216, -0.18049208624325228, 0.014213093965284267, 0.008626925460692556, 0.03320720972920184, -0.226798919817053, -0.042709888047130565, -0.003909135727076463, -0.060509435111033995, -0.053938179437156446, -0.0534454315934074, -0.07587416937409047, -0.01534949461310773, -0.07391818159176831, -0.04160855810716519, 0.016569048230165175, 0.03572177437216382, -0.1069339348875345, 0.00827324684004969, -0.1068004279109683, 0.01003901151190124, 0.01819386714629315, -0.039860064808863185, -0.01212209163448997, -0.0943990778139752, 0.0014084335704033682, -0.03285959751156431, 0.021447751479614648, -0.004092650680337849, 0.013408946512001698, -0.12966246961726516, -0.020313125746765567, 0.024220196800779425, -0.013211972518817016, 0.025146555047083286, -0.0440782542125741, -0.12142894999251817, -0.06279096821807408, 0.009907104820814102, -0.07968742906836081, 0.010641437700143579, 0.06274827055117432, 0.034074618914717095, 0.04700399440535748, -0.0875199632596841, 0.00777319009919227, 0.024296285153436323, -0.0005106384173838002, -0.0012715585308822317, -0.0015181130439211967, -0.251746710546183, -0.10091622164063996, -0.003576476338569953, -0.11186239670303613, 0.03965799971788131, -0.028842860167766302, -0.018849086766487767, 0.03521027135734276, 0.044136905075937045, -0.005547802492991935, -0.12346566936494471, -0.05011484830497694, -0.22523950668258966, 0.015558900906056262, 0.04337324693840561, -0.07894669845311647, -0.06500638268932313, 0.013494202689809318, 0.021303845973494075, -0.1320858550146645, 0.02616566510662594, -0.1506998790640468, -0.05117330229408232, -0.11224886870199621, 0.015315736770340862, -0.17122892565427789, 0.026107274701877837, -0.015442738727143619, -0.05506678866410009, -0.0039787131098265325, 0.02688882962633704, 0.02172947089493156, -0.079765362252275, -0.029728235533699146, -0.1494024037600297, 0.06170882903741407, -0.18767214006615168, -0.03881011352184551, -0.33978526024634104, 0.0008342450475084598, 0.0544530360988219, 0.006449245596349848, 0.05698270404271927, -0.0625540231632177, -0.06230468435701685, -0.12337840673536428, 0.021050580471819, 0.03055353405667789, -0.07521271132393162, -0.023998146267633722, -0.11931056951468505, -0.10035264681274768, 0.013877858526287238, -0.09410031568942986, -0.10310946402827789, 0.02228417014955741, -0.03193943006884384, -0.08475882028466056, -0.03295193772997742, 0.015718531228471156, -0.11618077981369368, -0.03723658791077141, -0.10405709216029403, -0.015631757097452202, -0.09267792393256415, 0.03421827406885499, -0.09517878314914918, 0.007120262789380797, -0.02665773847791602, -0.012982314955147298, 0.012125200022336842, 0.026270480764605017, -0.2056115282887398, 0.00762997509831471, 0.0063668876993391785, -0.014473468108368965, -0.1396237153172973, 0.006445384744284217, -0.06618109778739896, -0.02638478028211524, 0.006956077146348132, -0.03778157107931752, -0.07143284870458488, -0.016793261883271543, -0.13978771542397203, -0.07238590119129334, 0.022646742516126285, -0.06604329827202199, -0.028056216087066738, -0.010241834772204826, -0.0003632390628050065, 0.03622527138617594, -0.01672303809354549, -0.19880309180915365, -0.21243810856984804, 0.012983464208808266, -0.04309816400396243, -0.015889979096570802, -0.027254079212537915, 0.033400763989624285, -0.015715352396649937, -0.060342493423437164, -0.07428664683295169, -0.04039026891436173, -0.00982958177205677, -0.004742733760485172, 0.02952677154981719, -0.11747179413509688, -0.14441725731549362, -0.05800000626175011, -0.1121506840780561, -0.057916056844962614, -0.04656045544822263, 0.04923339296932528, -0.06693046226080607, -0.08737628495003764, -0.026563929254127944, -0.04108532324743422, -0.037715170495690384, -0.05615176380880223, -0.0667729046433097, -0.011329275326101184, -0.09798853763633351, -0.10549979335802298, 0.08862511582688852, 0.01917072384052786, -0.025587213720557375, -0.06322930825080837, 0.012920775708870734, -0.09695160709531218, -0.07866032110850953, 0.028906607126425947, -0.035319557497339074, -0.1374124068586969, -0.18758374211991646, -0.04318518089511005, -0.024667313386579307, -0.060100673948005084, -0.043762131486075176, -0.045432549842175186, -0.15128837375279097, 0.0038013498191004683, -0.07073837308321475, -0.025175429551987618, -0.035421404440836066, 0.019860696780449393, -0.21101498139602287, 0.004161789408930174, -0.03791837964017723, 0.01714946905147655, -0.036413109691541636, -0.0925592330567954, 0.0406519604527374, 0.002096969477120021, -0.005826528703071769, 0.015335139427754456, -0.19928065575311615, -0.05227754485414704, 0.023685455593267103, -0.003083085710117635, -0.1580372069681429, -0.05196636533629681, -0.011137133932999881, -0.08537117969570733, -0.1984371015201807, -0.11454099481649624, -0.08746733149933848, 0.00930252353023712, 0.04487261685400697, -0.1882161527387183, -0.0731740013112686, -0.17630438268592147, 0.008256326646029264, -0.0010484985601948832, -0.07068929186034006, 0.0545393174311928, -0.0021870781676811956, -0.023579012802725532, 0.03713609897709964, 0.012801220302015654, -0.06403491165576733, -0.07891186243290473, -0.04039279724030745, -0.05380340376841421, -0.06867720591322093, 0.03843031140343822, -0.03411972983739035, 0.03585753112560542, 0.0034796255862382633, -0.007139924379822882, -0.09096057463575825, -0.06046475351757849, 0.05169935137531072, -0.045900997101077264, -0.017392397965633875, -0.11841751748078314, 0.0011990982293718997, 0.0024767728370020344, -0.018247230288673734, -0.07579906947865146, 0.01791344624903737, -0.14280557759622164, -0.15504870625681316, 0.027294779274844107, -0.13437355088805195, -0.0415321507813009, -0.007200176110855393, -0.028307145716484277, -0.057209739649210466, -0.11218193722061576, -0.08330955305608016, 0.03316243489443691, -0.04837661706970361, -0.1519733282661166, -0.07323580122486052, 0.04455734540373321, 0.003971802261605996, -0.22417807744477877, 0.002472851798058729, -0.086946444155597, 0.03224874819164571, -0.13714793522365673, 0.05764780370165544, -0.10407635100654794, -0.08283433604173839, 0.05897755921126966, -0.0638836508129583, -0.08614131909200733, 0.010102653197668173, 0.014628228052378968, -0.025848504456287013, -0.07245492869785705, 0.010823371319549753, -0.16224733265368882, 0.018263918413359625, -0.15235047945904062, -0.03660965128393554, -0.06405030509175019, -0.0508475398282934, -0.1548875082688949, -0.002777386898526376, 0.04549580612294122, -0.0387410608666618, -0.08968654401136447, -0.06436055511023203, 0.012790054616845507, -0.03931517013024135, -0.004431477567829326, -0.07578099314801752, -0.044864298428578815, 0.00025429523252865993, 0.028838947981474537, -0.16293344412902155, 0.015665425061181012, -0.14379692679556652, -0.0004829013451848932, 0.03701202079707525, 0.040875192491164516, -0.15584172949312677, -0.08570027962679784, -0.11463643162747683, -0.04989058057221298, 0.019101842863749628, 0.001807211368506603, 0.028567181005102544, -0.061464613864846813, -0.0065386552877766966, -0.08132110091987255, -0.05357923577104808, 0.028789075944045023, -0.00940787385026004, -0.06208045908715296, -0.08760765908065726, 0.04928145626672552, 0.02884710931484287, -0.023713301374848377, 0.016091381802215317, 0.01646597207878527, 0.05408250522980519, 0.008695578295890278, 0.052460669436182474, -0.003383128200356728, -0.093787642083726, 0.011664114814221535, -0.046426533456370454, 0.004971277497760856, 0.037196236112285695, -0.014836488269314043, -0.06999767772082095, -0.17010441941754065, -0.1527304632142332, 0.004173782884341022, -0.03128180582773973, -0.13216418927399726, -0.016890753484450485, -0.12840959651192763, 0.011918712027575296, -0.031237671382472184, 0.01566151177999425, 0.01443392353215847, -0.06025515743402791, -0.01374627279334167, 0.03896840172123074, -0.09039893092022491, 0.0392582308113438, 0.033418497354158895, -0.18915292443017379, -0.05112833815215247, -0.06940417088746785, -0.04959586069721056, -0.01650357536508402, -0.034623744825355136, 0.04431812283114894, 0.03709455755856962, 0.014988988791188141, -0.09866049145542301, -0.21059894641424765, -0.03380148753280656, -0.052465393883169474, -0.1288704238589212, 0.02803671703050375, -0.09022421705537144, -0.021743635877304372, 0.025618114923462644, -0.0674249179232146, 0.02121335508897271, 0.031693394662349196, -0.10661277152213468, -0.08274147505653753, -0.23322159841341264, 0.007275884548562811, -0.025792520579677486, -0.1476529937006964, -0.1720489173047968, -0.17308760124306044, -0.017236825816223354, -0.038484257780252225, -0.022150873834496175, -0.09915780278154024, -0.009997002193983005, -0.04392798612237046, -0.037427206543397734, -0.14658268709054703, -0.11957506212734365], \"yaxis\": \"y\"}, {\"hoverlabel\": {\"namelength\": 0}, \"hovertemplate\": \"cat=\\u9152\\u5e97<br>x=%{x}<br>y=%{y}\", \"legendgroup\": \"cat=\\u9152\\u5e97\", \"marker\": {\"color\": \"#00cc96\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"cat=\\u9152\\u5e97\", \"showlegend\": true, \"type\": \"scattergl\", \"x\": [-0.02184885056913037, -0.03208302045410213, -0.030591634941433396, -0.030878657941604104, -0.013925541456215279, -0.0931883283901381, -0.05316669857740879, 0.06135848257700234, -0.06495361272224322, -0.07836288954880553, -0.013743119689441866, -0.0401221641513256, 0.02226955086257546, -0.045922461308371416, -0.0633307871267819, 0.04941491990354203, 0.004673924880529483, -0.018032728077407266, 0.06873966771264585, -0.08801140130518967, -0.05638930371500715, 0.025216023685473658, -0.07952376258306595, -0.08500808166017919, -0.05300970695168949, -0.041867308914742704, -0.032295781707453786, -0.06964656995446282, -0.05247658131131188, -0.05239016493225636, -0.01862313922231587, 0.05046699737849336, -0.06803343224207176, -0.017409183637643256, -0.0396355083287726, -0.05151733986216963, -0.08499286223001706, -0.024691925295938114, -0.12000233823599737, 0.0392671370886468, -0.0400411315372587, -0.046499037231533404, -0.03529569866528735, 0.04036264400655281, -0.016350871747413447, -0.03751054407793666, -0.04625520650414617, -0.11235273207809955, -0.03603368808673682, 0.1806167581429813, 0.024746845446267242, -0.06094917220071506, 0.09067037283613008, 0.026887468663535898, -0.00701056052382324, 0.074411999496784, -0.047398139263469384, -0.03444333216741567, -0.0371991987320196, -0.09515813273932698, 0.3218368714963487, -0.07757402764759216, 0.054141142838947505, -0.040217604993017374, -0.06904254728282977, 0.04814365726264047, -0.025185440119591243, 0.02578817332687787, -0.03826107514627491, 0.09463280440904577, -0.04185925212707982, -0.005960006407397776, -0.030754294101801874, -0.09620888991765059, -0.03599358920802966, -0.04101777077083954, -0.04026061611180247, -0.03467541013438168, -0.07530705871200802, 0.06712832494934837, -0.039430401820919336, 0.01776984902713201, 0.10917985930013561, 0.05654781170799674, -0.07437468640169605, -0.059470079801179694, -0.0042664904963347315, -0.031965519977624555, 0.06438128298838464, -0.02253597201833301, -0.029851981507565423, -0.11798362794745676, -0.006635339645786181, -0.07259956324619402, -0.0036821256108019776, 0.05157313490404196, -0.020922794449023754, -0.12227683835172203, -0.011076552925842973, -0.03953685557402773, 0.00040100679094558575, -0.05871733649283548, 0.01736425888846811, -0.06429654153382912, -0.045535885824503686, -0.09541232688687937, -0.031384418780240515, -0.07319754044215547, 0.019153686595193006, -0.019541409868997963, 0.05034748933738855, 0.03415164286100021, -0.045440429083351865, -0.04441563293135775, -0.04164611082995992, -0.00022399101228070525, -0.046869859534377764, -0.019281770951331642, -0.01894773834732719, 0.03041728716035707, -0.0491124954947403, -0.00843347687353576, -0.030399947132801503, -0.05452995218774826, -0.018441614687558082, -0.03331434853199593, -0.010942701631908065, -0.032376710644872374, 0.06913083583688212, -0.020081930512595922, -0.0635104177141511, -0.07926648679523089, -0.03723132089672882, -0.03516880156243941, -0.09188388811637536, -0.018409936768621404, 0.016994375975170922, -0.006729713904908315, -0.04963820085854837, -0.10250422460631892, -0.03247103792577559, 0.11261457834130514, -0.024917586627122013, -0.0440211470895637, -0.02302618491219648, -0.019980438494122326, -0.008380426851309433, -0.030163636962942823, 0.002303067303334994, -0.021889547952841003, -0.047948645378064154, 0.001287784263655106, -0.02019656657858206, -0.09260662386930683, -0.06120527427485197, -0.08100311615365084, -0.02088167787991178, -0.1084780926843704, 0.06230297368194433, 0.05569316100335946, -0.03800461138720325, -0.027968447167574604, 0.07247659401862631, -0.04732072600590949, 0.020811794974388807, -0.06293108411783961, -0.1267736520748137, -0.062050292632815945, -0.06144179030569472, -0.021124371992374665, -0.07030439462228896, -0.017081258657473484, -0.03421126279421402, 0.17513984887484013, 0.06810391031496069, -0.06167986823073716, -0.03601835439277653, -0.06604949235721702, -0.040066541060799626, -0.028123231533720226, -0.10314897858114092, -0.05473718778036525, -0.004700001638756407, -0.025577759636051446, 0.017914198571807417, -0.0008157634042883578, 0.02568726570212881, -0.06990388538580944, -0.09244892726198548, -0.042080644505726274, -0.07782739771877482, -0.07698148384845635, -0.03942810801785913, -0.029863487760995025, -0.03630345111481162, -0.04688151576970198, -0.0398632167260853, -0.018094294245361346, -0.03807955334676139, -0.031910338231348145, 0.010367680046867065, 0.10450744420275122, -0.05710594739027969, 0.037594259654811425, -0.022004749400742356, 0.026997608288593124, -0.10384417037513563, -0.03381796705150721, -0.019443252598424707, -0.04684261161460338, -0.0338912170972683, -0.03303182769612561, -0.034716411326909766, 0.07641022432194355, -0.06285266830177227, -0.0778858932898197, 0.011800223673144322, -0.04749909342611796, -0.03398718504142588, -0.0596753648628829, -0.029211223568502143, 0.1187530346620489, -0.04610501856744672, -0.014781031523741136, -0.02434508093802024, -0.040191374863455157, -0.0658018566212521, -0.03773032792393665, 0.1450026251121437, -0.03061031337422947, 0.024200459657851794, -0.002356942194444961, -0.007167385278880196, -0.021461925757691187, 0.05185066318406046, -0.015523583836636665, -0.06828159533027657, -0.058406244557721365, 0.05429947488713137, 0.003546784563271205, 0.07866495143072796, -0.027817628463008903, -0.03922028840175424, -0.04044699986019843, -0.002433706250528872, -0.06998989087003263, -0.07449006005478152, -0.03643588851050936, -0.11912438240494705, -0.042069725437542144, -0.036957560465732286, -0.03748501058990122, -0.05691870705677827, -0.0031990107639709294, -0.08321044200579154, -0.046863680049174794, 0.04867407346222067, -0.043277883657753986, 0.06676412027748614, 0.03665067441948141, 0.01322596880750485, -0.054579314443250065, -0.01932444391284486, -0.008630929374812486, 0.06936842361171906, -0.06259533459230292, -0.04472960235801168, -0.004401775221377632, -0.041426455363722844, -0.06505005722685055, -0.035897606452482286, -0.08693545073401002, -0.08541641822284815, -0.04797541632230455, 0.0021493751927514813, -0.02251467806841351, -0.04403836710298312, -0.008544051684607278, -0.0726212892572273, -0.08017455779971501, 0.06762687889247596, -0.03248906161186591, -0.03704689944870411, 0.03676941555302224, -0.0202543495335063, 0.004464786850800672, -0.02754080494591771, -0.07367995915532109, -0.03511920079866439, -0.0771968225829319, 0.09931178382991326, 0.03407237797615463, 0.015816023122882626, -0.042896870662115824, 0.16208813330015076, -0.02730781801552461, -0.07536331901600504, -0.0403072885352823, 0.08083084640450805, 0.01736891573620301, 0.03689804051158914, -0.03287224157065663, 0.009667473060127895, -0.042155652174572435, -0.045775843059398955, 0.012214160397304562, -0.020603183510118184, -0.036876787092288095, -0.0577830520496329, -0.07603063820120586, -0.040289658134899285, -0.012865217832798048, -0.10316836955454022, -0.054957577237947516, -0.06075823465354116, -0.06406470299746664, -0.042811239461031, -0.11727034686559952, -0.09362119096213169, -0.08036035474124072, -0.016986965801814444, -0.03401699901529899, -0.05820528272657089, -0.011734545411654486, -0.03468620987612727, -0.050641929858490246, -0.058214166555324576, 0.03141345887568147, -0.029290859246002936, -0.04709790034317267, -0.0291699814515769, -0.016177256731571325, -0.035494531873660355, 0.08669209344088813, -0.023030595242045062, -0.06595108987022259, 0.01335773864449582, -0.013976593452860566, -0.0950060803729884, -0.04607948798555661, -0.031759243052443924, -0.028765058291244736, 0.021590250920161036, 0.03323117769296801, -0.06589309212359515, -0.020036518711278473, -0.10024925846994334, 0.18022785011844306, -0.001299415094548192, -0.023178658299180435, -0.07096162916063464, -0.0857748629707249, 0.023802361981684193, -0.0681356765818434, -0.050177109250569146, -0.04911337523650919, -0.03545588346330759, -0.08030254161512763, -0.05012328662135052, 0.04887216384474311, 0.02702291706009646, 0.11965608887964195, 0.0396793784401242, -0.009637391994471698, -0.06980489647549051, -0.03444043952678827, 0.04778499101531108, -0.07177086647810814, -0.03427244738702581, 0.09686500913608304, -0.02926205619267163, -0.032363992073363594, -0.019054795801674098, -0.05332150059087329, 0.058969248774796125, -0.022536938312116853, -0.05797293822806343, -0.04535870062135545, 0.20063695360576994, -0.005326062684880451, -0.05295819658943084, -0.043628001903894376, 0.12732830336730278, -0.023315052244945567, 0.0499164077101565, 0.15425613087033557, -0.060390722434462314, -0.022414232889884488, -0.05177694302177129, -0.01791885624481342, 0.13864004945195543, -0.024083962819835873, 0.047318442082226554, 0.028820652449018002, -0.0340939850645764, -0.05387049789104418, 0.07148447941709833, -0.0018537820098636717, -0.12371834772361762, 0.05696922184772215, -0.07170156585053865, 0.0031975722453631878, 0.013114112878990326, -0.032911369072180736, -0.06942198707940342, 0.04896486157541108, -0.08801850716627359, -0.013210475670664525, -0.048738582395270576, -0.06142212050960132, 0.06071650754192522, -0.036308758620846704, -0.013079269854393021, -0.05220741468614171, -0.050222531161088844, -0.10424228110641134, -0.04186662163513639, -0.025597989267695017, -0.051424642880666574, -0.04791934730343905, -0.03588271700138104, -0.026827369186467477, -0.10232415023083416, -0.10581961330781117, 0.12818328385353497, -0.05572722176127809, -0.03217755418271861, 0.08044516233339093, -0.13528215619182354, -0.07740206007265119, 0.06069348023414401, -0.03321653680219606, -0.09736375222402055, -0.04753991162438475, -0.03651394398602877, 0.003943024242759874, -0.09000079421344692, -0.05509464250679991, -0.054790883338971086, -0.05356249625046653, 0.00371839870688711, -0.035890621555349625, -0.0022202463957917444, -0.09446703705963266, 0.025012157687175563, -0.07770664644874747, -0.061370978191333254, 0.02501442810582482, 0.008369176994017615, 0.007447164423003076, 0.08355148105857114, -0.00643769514069002, 0.10287904180311759, -0.08182213885046788, -0.02519769728012751, -0.014727357060260052, 0.057380678717348886, -0.047114617753760295, -0.06952124678443634, -0.06273134221278487, -0.013545335769007306, -0.023588559862395324, -0.020966653589016213, 0.2646298200109072, 0.05926550107754993, -0.0518211247467724, -0.004593937582795396, -0.03594967986269803, 0.0529391606692854, -0.029974528560483965, -0.03951899376292066, -0.04229197137812278, -0.024005320985348393, 0.06591683207443821, -0.019872153793402347, -0.0597627185371495, -0.002439141762443427, -0.06333717335927697, -0.039054824306653246, -0.006575441486027515, -0.022695464017145544, -0.07370479344685832, -0.011375332268235509, -0.03527063459551871, 0.1349608627798564, -0.014822308868996624, -0.06069512926806608, -0.01599502981042942, 0.0767348285788391, -0.034349968710082564, -0.02212020534783074, -0.015817929598468946, 0.03904636928170072, 0.032909128195041944, -0.0712509641979014, -0.06309820254991093, 0.05658251950337377, -0.0028837961574960384, 0.038108907592539716, 0.02047256134815523, -0.09224312626709387, -0.09595943128697414, -0.006808634919951477, -0.06002072612502338, -0.03197232892091274, 0.009198268560658137, -0.05706684569665308, -0.05889740841381635, -0.04552070663626277, -0.020694595264161002, -0.10898341644662185, 0.010228790676186414, -0.03167651386286139, -0.053787323520347895, -0.0687574326659591, -0.03807292586967127, -0.030344534697294978, 0.07801073420559902, -0.03996099466548424, -0.025537997770324714, -0.0599767448279107, -0.08177266517660906, -0.014186589201801152, -0.053785915741076704, -0.08797583688916366, -0.09580248703851937, -0.0912041490547438, 0.010797375761451464, 0.05191549026227358, -0.055489363393951566, -0.033881427307079784, -0.05636533992708218, -0.04355675012314153, -0.032386377686285855, 0.013103342652639915, -0.021018106051148417, 0.04056738759070734, -0.061449675065615425, 0.006427176881710677, 0.0049675237910858745, -0.057513564370702146, -0.017476823986817793, 0.014993957304078745, -0.08180103395904466, -0.023222965075413972, -0.06531625348436948, -0.04942880895537067, -0.02470142119154875, -0.02204675208032145, -0.010252886757151875, -0.031586956247896945, -0.03325820818164039, 0.2765484700455877, -0.08717360774311367, -0.08268930484239449, -0.10658155008139501, -0.03782804900046787, -0.023710607983525363, -0.013823016579886946, 0.030155973411167155, 0.07197279013055949, 0.003604497986631157, -0.021574836510430757, -0.02284323557281356, 0.02721560778071247, -0.009264795769442186, -0.10989037114846924, -0.042087393073271966, -0.055072416330483376, 0.03174365304269156, -0.06754355011542762, -0.025904778198139404, 0.0011054155428146585, -0.029078422151074507, -0.028787180057293183, -0.051910544806141296, -0.006100124445492301, 0.0395551238232181, -0.015096723541044904, -0.034637255189403526, 0.009462499371918594, -0.06785659843000688, 0.011271499849962065, -0.025222137893404387, -0.08560246309557525, -0.0653444965827346, -0.045969587995120034, -0.04049175917730333, 0.06353339494927512, -0.10217575732350853, -0.018581850677017397, 0.01640298029685111, -0.007512863232019963, -0.03058655973114859, 0.13603071225455657, -0.050476369361827485, 0.0002716798838806071, -0.021981022815477553, -0.02880338672806613, -0.010904645699551149, -0.06845356641756711, -0.058800757430926325, -0.030506329963937262, 0.018882078316087438, -0.10654778547241377, -0.04576213184876075, 0.026040455191855178, 0.036947283507320454, -0.09059383210908323, -0.10853661470331888, -0.030570603400883054, -0.022559459896608083, -0.02469136810871555, -0.032646311045132985, -0.02660896957317215, -0.0772634648285265, -0.03505486984606829, -0.05276283648081154, 0.13120710176028916, -0.07778267096308769, -0.00031877420128694655, 0.07333815662640139, -0.04631774870279792, -0.07046475517969898, -0.04855003766767548, 0.029752125612486035, -0.0948044612671885, -0.08883785619858713, -0.030342956528203346, -0.11498585051696408, 0.054977459443616844, -0.05463756442785819, -0.026042927192353117, -0.08281817250794372, -0.0667909902290604, -0.015826263746202985, -0.12540887308424034, -0.041855683001915, 0.07776045891257108, -0.030108395019737087, -0.0304866401448525, 0.02839347280658939, -0.007996970083214818, -0.06819753174895443, -0.042620471818274226, 0.09079897821375296, 0.019036924189191614, -0.09087957266854595, -0.052866891330992, 0.03063009940181128, -0.038240115377507274, -0.01793654798169185, -0.07316055581914367, 0.0654791810252354, -0.12226758559717547, 0.00850558455810347, -0.03676556522949077, 0.003406554593308686, -0.05289962875983543, -0.015307979098167553, -0.034282071506280806, -0.010602084591927687, -0.019728375097409857, -0.08988396013033595, -0.03488419700147193, 0.01578736850388469, 0.1983521989849643, -0.03241316727623596, 0.020740170957511184, 0.011067928309436802, -0.10926995830159193, -0.04370276866885491, -0.03968391932543714, -0.010366385198658276, 0.0937467249352068, -0.04788057685846517, -0.05199528385925919, -0.024877714555234826, -0.07924564034621946, -0.06782666207239645, 0.15884100666065795, 0.09394656432957597, -0.021620836463033834, -0.019656490418251384, -0.039919303311544696, 0.03253950160597669, -0.051894109753067436, -0.05331861344837273, -0.05572864577306525, -0.023049138830052197, 0.11458146431797132, -0.021265086504126993, -0.01761542180770575, -0.04003919801015953, -0.061539355369704736, -0.04262359378490921, -0.08133399803849628, -0.06215297760105407, -0.03167294066157352, -0.06232958905687084, -0.014936678580301267, -0.04946092730837351, 0.0010343640774986157, -0.032349283013356794], \"xaxis\": \"x\", \"y\": [0.1497974294001999, 0.11835983689368174, 0.07312177738214064, 0.10479038193293848, 0.14099719304511085, 0.13598578788135263, 0.1377238857992983, 0.1781917688680039, 0.172016695349021, 0.16287615013067258, 0.13512339510401356, 0.13706486695895112, 0.08528492731285828, 0.15970649600930545, 0.12464450389956819, 0.06556641722505757, 0.17690893369490449, 0.15479433802119977, -0.02532790129583512, 0.2634135882497303, -0.039724341321057746, 0.26252589395771636, 0.16338325452746683, 0.1702766593797222, 0.06972007289322052, 0.0796681193098519, 0.11833121745221614, 0.15478372037320381, 0.21401104692559128, 0.04725392999095188, 0.1281824135280669, 0.25228844262150213, 0.12779694812409115, 0.13446119777461993, 0.14133372536779784, 0.06682926531485762, 0.11232663279411179, 0.15093586727427902, 0.07311030475870947, 0.10575985990030434, 0.14730909559315725, 0.1707321254036352, 0.07362182836388405, 0.09308564199108181, 0.2077987439518128, 0.1678610774084273, 0.1717431242717164, 0.10506793923992995, 0.11591243193975498, 0.09783845370690987, 0.19438390891653684, 0.19393884484870236, 0.2317352709484312, 0.25805926229654147, 0.1190865410449954, 0.05483809730525501, 0.10571219117676145, 0.20319458774082574, 0.05457870865515447, 0.27568389145674677, 0.07049984648474768, 0.06670714637939856, 0.059176266205751546, 0.27383572214456825, 0.10422509480589179, 0.16051090328216808, 0.08165313628111279, 0.08849913282921042, 0.09398868906705009, 0.1907676806162051, 0.10868306973363358, 0.04246019122706778, 0.1386001044761008, 0.22276753554702924, 0.15376532325147935, 0.17187123750255784, 0.09607514687245941, 0.07504702281024782, 0.09814510813629458, 0.11265180636226947, 0.1746052056707345, 0.027557512842820913, 0.1298887827031105, 0.09727681140188643, 0.08915841295263703, 0.049739300228798605, 0.11068211019138019, 0.19712910416306126, 0.11593786488461975, 0.16967192370424436, 0.19854112130978332, 0.19093261140786794, 0.19626734668965554, 0.3083355774375187, 0.1962873198670757, 0.19231434781929707, 0.1328926388363396, 0.053073300936372726, 0.14526285432764416, 0.20898108012097705, 0.18689326672703882, 0.16968790882326373, 0.2709449502770544, 0.21697844692077958, 0.2094131359980933, 0.1537125387278781, 0.08830410658446942, 0.123422928024659, 0.19917660088706546, 0.07901123990154303, 0.12700890020838124, 0.0645156624311361, 0.15384236110346441, 0.1922991868489217, 0.17383079050574887, 0.1023097680388529, 0.1879193964863853, 0.20967928276566414, 0.11491270784451305, 0.21608255140206503, 0.1700770832880764, 0.1298801354977063, 0.1792812626862398, 0.0988426994603609, 0.1846707243738751, 0.0060071441293791895, 0.09284727687339406, 0.056045223888506114, 0.08481217193191326, 0.02232412981057584, 0.2163214897932512, 0.12428970267088882, 0.04528025387183298, 0.10928149113152075, 0.20619511880889077, 0.08125378934173821, 0.11150106219746571, 0.1944264601222543, 0.08525147052957609, 0.3801020547750082, 0.1755251264876719, 0.13067680976991808, 0.12607648388865572, 0.11978370777947513, 0.1648621902919748, 0.13130842308008253, 0.04544945539465468, 0.14357285404035616, 0.0713025130893111, 0.10962038137268645, 0.09860766362990085, 0.15111487473804816, 0.13784134286351604, 0.059231771527203415, 0.18169252653782647, 0.12131067816989725, 0.06785586701223491, 0.1408383418002714, 0.12378019776782544, -0.007218623069141125, 0.23699532273351637, 0.18410217035701998, 0.14132708765136023, 0.20375501738214788, 0.09311385453081508, 0.05507297554032687, 0.19459440082488952, 0.20887723453701684, 0.28078250260544135, 0.12568976881096885, 0.1776740912024082, 0.1312658544109306, 0.13240183334754557, 0.2066073523361466, 0.13860447735329504, 0.16200144809879347, 0.25143074546203986, 0.14102775887995492, 0.1805370129842679, 0.16201175249386734, 0.1707224888783109, 0.14795714305344818, 0.16305618484177622, 0.1028712081093801, 0.06311615233513501, 0.09549226872770472, 0.10851127172101718, 0.19944032381835416, 0.25916044079704637, 0.29577604677954256, 0.0755673312513572, 0.21870973697787133, 0.10791406090113945, 0.19037945393398337, 0.20857310990245548, 0.08743234905601276, 0.10442670372207631, 0.12635176866811396, 0.02117644542315197, 0.04785485882019722, 0.21006050328646378, 0.10777388179265894, 0.14550112772592266, 0.10006479140730175, 0.09296724883295848, 0.0604125871867093, 0.1396201615742119, 0.1988708600687622, 0.1590405837441268, 0.10317382302123171, 0.10881734534084483, 0.1552820011254844, 0.20768071399667862, 0.21772063316589704, 0.03511592286544347, 0.10288543033723356, 0.21896884438770917, 0.05945991858960245, 0.09623660598245327, 0.12427924040076599, 0.21290148055816635, 0.08373503168113151, 0.26868828088146546, 0.03989480629636714, 0.20713964191577044, 0.09225373733656038, 0.0934437578579929, 0.12912372925933813, 0.1737123931034156, 0.18031078388658395, 0.11187555841400072, 0.10914715393108058, 0.1256219392603182, 0.14081363698639124, 0.09069271423956843, 0.10261103121967356, 0.08512697450149234, 0.0324756228843255, 0.18978794235050567, 0.08598302897169417, 0.14188424607888883, 0.15626837372602076, 0.04055851065087695, 0.14742061331883402, 0.13295896442337726, 0.177467563338559, 0.15676713822964897, 0.24383431887291804, 0.4262395099718178, 0.15744668932617284, 0.10037524870132648, 0.07440973555060093, 0.11350249320177891, 0.13065643180010292, 0.09164595566771615, 0.1702570113935651, 0.1127398678283308, 0.1974101124809933, 0.18366009634865346, 0.15224970350978287, 0.10031808085791388, 0.14651378567819204, 0.04272661748189635, 0.062168148952056045, 0.16473486919193028, 0.10237975997293307, 0.17828889518988777, 0.13129250237846493, 0.12021489398309586, 0.23495474611789893, 0.32893030373636606, 0.13311813978200263, 0.10589728589281343, 0.1713266652084765, 0.1299065295432377, 0.11506586157564293, 0.12203620008192831, 0.035797537267659184, 0.18069716733666752, 0.17804865206750903, 0.2258829661552876, 0.0842588311241029, 0.14309001924430487, 0.1622566763187886, 0.190233491403355, 0.09116864256955644, 0.08271597566683606, 0.23756559031302937, 0.09316026804416432, 0.16266381411263073, 0.16177278365586067, 0.16135252022056246, 0.16498954125836107, 0.12226727738089509, 0.14865305869318257, 0.11422602476807309, 0.14282787868340002, 0.14729249665192276, 0.17008062160104565, 0.15176261562539883, 0.054683615294507305, 0.14969010964154364, 0.1281468017307794, 0.13939820935931868, 0.13117934634105977, 0.16515868106312456, 0.046164540004565886, 0.19192993746979414, 0.03764438111239787, 0.13227397445288308, 0.11321437208849292, 0.13279120646755643, 0.04515433656284412, 0.2533999758651619, 0.1710780484795014, 0.009276918263172479, 0.1545387672203533, 0.10795125821159843, 0.17252524363167418, 0.2160242767224051, 0.044040315290091996, 0.17606164324785278, 0.1046489587065888, 0.23317123587169447, 0.05927288935207945, 0.10703338710867233, 0.18452067347881954, 0.1363595601843602, 0.13311599038215313, 0.14130473669708118, 0.11581100132818316, 0.13719783826107318, 0.050537002993425124, 0.17059964047831797, 0.0923530258260691, 0.0827675729287506, 0.11783894757411563, 0.15494363662878596, 0.16923539742852423, 0.1417627745704792, 0.218580476671398, 0.11442409925357208, 0.14013751699427918, 0.17658610817643355, 0.10977189048605089, 0.12639573116748182, 0.2354220047979703, 0.17836620380581641, 0.13544412465300337, 0.11761354927517811, 0.1832214399465504, 0.18689989190413187, 0.06760582156785266, 0.21401374740398943, 0.11895752818325296, 0.1381547395908228, 0.13372116125278236, 0.10305997605944664, 0.15138424182039092, 0.022806789265073735, 0.17718504043403777, 0.12683734040627948, 0.12874498874899795, 0.13329892345879069, 0.0213684618768811, 0.09343354566193615, 0.1948289681705488, 0.3218886380298996, 0.14285409193727405, 0.10574333574289083, 0.09423903127998605, 0.2015146301806372, 0.07405430025767945, 0.22062856747392706, 0.17031477244290436, 0.18588188910512787, 0.08411965212593997, 0.09357100626684996, 0.23454615123918607, 0.1734081164193096, 0.08980265015311804, 0.12016823882367778, 0.10626977034123837, 0.2826258868620077, 0.27244903226693484, 0.12724547620290175, 0.19390915657861396, 0.12584957408440023, 0.17261204148919615, 0.10429344995365965, 0.1434016760822423, 0.1088720603208379, 0.21950010230227968, 0.23770331023059552, 0.11203391939339855, 0.0749892959624858, 0.18957686446396868, 0.08761141681580312, 0.13183711824335917, 0.2133220503538212, 0.1185929640159717, 0.19870155487329105, 0.1468624664404918, -0.030214613728919214, 0.18207470756478333, 0.17705265208707602, 0.2622586195596852, 0.10212133639948923, 0.15541377618093566, 0.09376833923307032, 0.09566985342631468, 0.0789366881542106, 0.19216025374754125, 0.1027733496263848, 0.19826673848027002, 0.11837129777053361, 0.046656117905809646, 0.055266029470743896, 0.06117558865398517, 0.15336672266586088, 0.07610868121167078, 0.16903052137058966, 0.031481830419895954, 0.11004476045818229, 0.1084126266892824, 0.11908163666382933, 0.2057331487999052, 0.11215763118653663, 0.20207016482298873, 0.06388548297137668, 0.20999664439641397, 0.14858622288044215, 0.11685322520841786, 0.1386312884792367, 0.07778300592698924, 0.144624516002903, 0.23890138580420203, 0.04103268166310397, 0.15468646971543368, 0.12071621845703308, 0.14801436152004707, 0.20210866458246493, 0.08901070587902807, 0.1831800954084044, 0.0902440021121652, 0.11218180801836315, 0.35446495454066107, 0.12959762862043794, 0.08690411233489329, 0.13288955135703936, 0.12196244032367792, 0.24403071748995483, 0.08771313729050005, 0.12534795796874038, 0.0922093558844854, 0.16768244318027076, 0.1777065253611587, 0.27097999569838954, 0.0950088233042071, 0.23393047857796975, 0.04558039221409085, 0.056530719705659045, 0.1665105437873735, 0.10644786721153167, 0.13281599989478413, 0.0685874628937752, 0.07294984572368911, 0.31819875025547767, 0.057254773516790675, 0.23891950308371476, 0.2015013157686069, 0.12751131260239384, 0.10056783348152278, 0.19411488001569846, 0.13593727200028388, 0.0840614161758871, 0.033886576870652906, 0.22826475145074687, 0.044004478422693276, 0.19987936121329283, 0.1665270122884362, 0.06086672736284896, 0.11647552003746736, 0.1462856983778794, 0.09429340591991636, 0.09149435451991071, 0.12532782971839226, 0.03935269174874857, 0.12885931821314162, 0.17743236758105907, -0.0900379619620039, 0.1295626222979442, 0.06679705876040296, 0.08059666345353575, 0.3258093473860714, 0.08689203580005797, 0.20592916959721594, 0.0190204962584804, 0.11291393412536382, 0.0975119981721022, 0.18740936919399095, 0.0763254610224814, 0.02023492479404149, 0.07636574034916972, 0.07114933661796181, 0.10927729331815657, 0.12363008974705095, 0.27972212881974634, 0.20751045333373308, 0.06746792030585196, 0.21918837411926925, 0.1662110162928944, 0.07393287203131213, 0.10476460816286266, 0.08871244890128181, 0.1232732129784999, 0.09844321374651631, 0.18636406139993814, 0.10874350912239082, 0.18597337344239048, 0.24652288236597256, 0.09042556788023887, 0.13020780893159828, 0.08112240172340215, 0.2051521564499056, 0.07162680131445177, 0.1081921708244029, 0.03710593779664265, 0.19770393589157337, 0.10587245442062908, 0.11552761633525344, 0.20493391121753785, 0.1168889606233466, 0.25988549517236587, 0.0033815314656427675, 0.03819196529491415, -0.017568447115431225, 0.22504571674579482, 0.06145635452654063, 0.24415588770103872, 0.007117276390767599, 0.10478003542994503, 0.10331424529149841, 0.13789608318744562, 0.18433564275319883, 0.12923776594312902, 0.14061042613030558, 0.15672089013133111, 0.03638150233223395, 0.22177852034994633, 0.09815542275750597, 0.05478771388764137, 0.03009369046796935, 0.10983247749163726, 0.1400450717282833, 0.19559865945336466, 0.08077090433238489, 0.057793674832002914, 0.060939201901471125, 0.12117200406879125, 0.09211833464760087, 0.10369850898028976, 0.10392070294120119, 0.10622446749937425, 0.11728083818140216, 0.12342375531683329, 0.1716269295902779, 0.11864856092714483, 0.1765964213151864, 0.16345713261419328, 0.10566776656553563, 0.1660583131738434, 0.18442587186051596, 0.16704968342358595, 0.0937139501495521, 0.20416652301669355, 0.11092772555581047, 0.044110457781154244, -0.009367043689282331, 0.07420826041698313, 0.156757709504527, 0.1784000917776198, 0.17578764429621577, 0.20212376922481667, 0.30263204545871436, 0.1901487760478537, 0.16634473569669392, 0.2857538504201073, 0.15279402273971762, 0.12731166403579336, 0.1947120810348472, 0.06888516748837092, 0.17783748048896905, 0.008471900971207374, 0.010775863914004735, 0.19487202985059635, 0.09643267466413558, 0.11348304952979528, 0.04518202826695152, 0.04803925111285688, 0.14653473889803664, 0.053110850481084944, 0.18210637498574445, 0.01948573246350404, 0.19150853963363634, 0.12321671376340591, 0.21634370436813824, 0.1477376290885287, 0.15507609362924185, 0.20395992573417987, 0.0752314738697321, 0.08653453514660125, 0.07719031356578883, 0.21639379137471243, 0.1964777240690893, 0.1026043455817292, 0.18931297135635453, 0.14663337515857347, 0.23174908248205867, 0.11755575733660877, 0.13742726668868763, 0.09465284348328051, 0.13529306944297811, 0.14620524428740891, 0.14903041214079696, 0.1833268885881132, 0.22589869685174785, 0.18658966022279375, 0.1372102760519073, 0.15966221689561402, 0.15334790217249347, 0.12670198143015032, 0.243820122584994, 0.18088524873831413, 0.028550897685514023, 0.10905302088009562, 0.10566207447240861, 0.1394106060597023, 0.1317560984969776, 0.18326994602575403, 0.11893233011845412, 0.0872158205359374, 0.14376457563922904, 0.050964613851024916, 0.11660195742192898, 0.1595597727066141, 0.07911862001008896, 0.23698744844796815, 0.05059365351539855, 0.1310626331057727, 0.14560865210695828, 0.17101530675210141, 0.14438462841104596, 0.1029719388988532, -0.03202409485503402, 0.07273849338926973, 0.1575511959776825, 0.20451079958687385, 0.11222675707774675, 0.058087548370589326, 0.23470815042129894, 0.13556885642835062, 0.1449449535921959, 0.20818197002016606, 0.14675351717383706, 0.07875090756733294, 0.20821283333054968, 0.168137309205766, 0.06923507505203483, 0.06216212433692215, 0.16723901555182735, 0.17301543485083232, 0.13693529204210597, 0.22233182426298118, 0.17815461942621708, 0.08549447815655667, 0.1771580011782472, 0.0984465730056957, 0.13105587384067774, 0.1567011683805306, -0.13574396198501684, 0.015002469365075351, 0.12312995505166921, 0.07131227664603051, 0.16811772661290925, 0.13632038275639505, 0.14817524099248638, 0.10172018722994994, 0.09741217301484853, 0.03692935806902428, 0.09525351732389942, 0.03300339174071623, 0.04755739558814541, 0.1052343390182927, 0.09922805397429926, 0.11047423558950192, 0.17052074070254847, 0.17935619272220107, 0.14140100387977672, 0.2002817998015115, 0.03794615423199551, 0.09162909265560423, 0.04453904148327922, 0.04388112093477036], \"yaxis\": \"y\"}, {\"hoverlabel\": {\"namelength\": 0}, \"hovertemplate\": \"cat=\\u8863\\u670d<br>x=%{x}<br>y=%{y}\", \"legendgroup\": \"cat=\\u8863\\u670d\", \"marker\": {\"color\": \"#ab63fa\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"cat=\\u8863\\u670d\", \"showlegend\": true, \"type\": \"scattergl\", \"x\": [0.006410635278561735, 0.1559945097386163, 0.3163620485731804, 0.3595457622147959, 0.01876531101103561, 0.0698641115908429, 0.10457414525745551, -0.026518024642705647, 0.083745393844837, -0.07092222933988451, 0.08510169639537879, -0.03196241137136163, 0.27730646706104095, -0.031436499614932834, 0.25621821190211785, -0.02556641399805244, 0.16624132515820905, -0.028784672509271466, 0.0265304125379801, 0.016398128982448978, 0.07805257440049138, -0.03173990270635906, 0.11521300896713195, -0.04370752417127692, 0.09714836032166105, 0.09384568828092395, 0.02039303214963696, 0.04043919419204739, -0.06864028992828808, 0.16936584452461528, 0.0381365966162643, -0.032547977894484714, 0.03225426274619505, -0.04387100024417775, 0.4445364111954195, -0.05069239342506558, 0.06677980451547656, 0.0359698759525066, 0.3371284641425306, 0.3959629076332912, 0.2798645571355205, -0.04325261160348077, 0.17266466614858408, 0.43133450633293197, 0.23769582188065475, 0.07101154714089704, 0.4225289880365999, 0.38605252721040145, 0.3757692809112856, 0.4112388009395425, -0.06537175118339401, -0.09521894998378108, -0.019516458303502134, 0.02012957988891586, 0.23213691357234817, 0.0740233509193938, 0.2123525876247113, 0.08247583084839348, 0.2236881691100186, -0.013945792442265492, 0.1445107198213678, 0.0160295415999158, 0.003922135730153656, 0.0067445963525014615, -0.01740743497590688, 0.42704669358466035, 0.08357270851469087, -0.08401872319397877, 0.036039607977273146, 0.32426135282364493, 0.2753901331994818, 0.09517352721711089, 0.15330473926412858, 0.0001812690279119085, -0.01299596189975359, 0.45561624460504385, 0.04413788702411584, 0.04225860268224498, 0.3136368658491654, 0.17509719150394104, -0.034084032937617, -0.09250943667022742, 0.2604867194293138, 0.19639743620896605, -0.07740829613979332, 0.1329686618164126, 0.006123242322207538, 0.32468854694687466, -0.04192640842293806, 0.5141706881800346, 0.015328311529056638, 0.21289409585878494, -0.028658202627515058, 0.24930415573131068, -0.08844076276372452, -0.08037406503975221, -0.07677721940186606, -0.06147705637910851, 0.17497557568637376, -0.010023765950142788, 0.047072824517297336, -0.005840775008752981, -0.018684212462585777, -0.05289961940495202, 0.009494891149864603, 0.17141347929348494, 0.12640250707642312, -0.027533835629911088, 0.004443675139925544, -0.11112949397896606, 0.009244523071051473, 0.20984699637886473, 0.23359792155888073, 0.04184346461308182, -0.0008457076856985481, 0.2418531714484945, 0.2755688096338553, 0.12214612514104498, 0.13537677651594018, 0.19806709506440875, -0.011801956402349104, 0.017418688150108406, -0.012359834420305197, -0.010848922266282484, 0.04332510524769044, -0.04271800711499257, 0.3617311442806921, 0.17835598994531587, 0.11221838826259399, 0.16618846872650866, 0.31055214716249663, 0.4566544516898664, -0.07750288268038917, -0.032169316177433474, 0.2458897483409096, -0.02025649962150629, -0.08955692770979763, 0.05190552814581917, 0.26214184197544305, 0.006092673729557411, -0.07795657954852551, 0.33142373212780757, -0.06729695462917341, 0.319024594120912, -0.040848461932606966, 0.13073305046567818, -0.07454903727583771, 0.09146251947385359, -0.029553318575166458, 0.04272628643258879, 0.040759880091350786, -0.02283311809108119, 0.06437246905433669, -0.009494732776333645, -0.07080890442478932, -0.04791379486011032, -0.0728317572928601, 0.09599564659048332, -0.030159993358400573, 0.3551824757295012, -0.03411416214657327, 0.3675009161075074, 0.27217515263177733, 0.16435376153301576, 0.46235190335158777, 0.405614753447665, -0.005920453439327114, -0.008290478370852563, 0.07970747590005867, -0.016021426226062504, -0.03532845274571701, 0.16747273877682764, 0.37905710832970085, -0.027611403746793113, 0.01545738379639505, 0.2423132766884193, 0.020616564490397876, 0.018187108653607224, -0.02154545106600189, 0.19874171037852292, 0.41216854634726985, -0.028135684597132313, 0.12986902108322085, 0.0925717422663529, 0.22687628131514373, -0.06597131888779216, 0.006188137793287369, 0.12975804669975308, 0.18852292407467938, 0.2736968515291267, 0.29048457223926616, 0.37520578910431357, 0.0034376557615699663, 0.4105931289263059, -0.05126248850468242, -0.031249251133693635, 0.1295711305136607, -0.017271843475480594, 0.10913323849985547, 0.008044959286870285, 0.09125249360640446, -0.008160799707616037, 0.07300365603678452, -0.030020431242016685, 0.087602539857255, 0.05185912601608369, -0.08619430843172475, -0.049953814204974176, 0.2643473045023255, -0.010666351787560464, 0.14703168704332203, 0.03999551775523691, 0.09471251750737363, 0.10787209549864372, -0.03307004587562513, -0.01412832278500294, -0.04161740645717776, 0.009814920759329963, 0.14062969912494433, 0.004863313342798393, 0.06077665785193486, -0.042795314066230855, -0.002537807849493329, 0.28215092703622824, -0.09855849982347711, 0.09265695980173849, 0.0507659259971712, 0.15076211364362616, 0.09385268860936276, 0.08613763621798616, 0.45349303732947704, 0.06820275849912405, 0.3165301033937419, -0.036337148809633284, -0.013815747868365605, 0.20615289272431542, 0.019265702950352688, 0.1501563495016845, 0.018027976762164506, -0.08740855552154546, 0.08377431509370752, -0.003566708735490573, 0.28839290960528013, 0.20285630506676586, -0.0489895248116451, 0.1311407008940536, 0.06567847897597828, 0.06512859938952752, 0.2216795928168841, -0.05432344790101855, 0.26444861997628727, 0.35375169505675275, 0.31278523319174245, 0.07688098437683377, 0.1159448371678088, 0.24278858651712357, 0.3646309459262612, -0.10097078698113676, 0.4167563453985375, -0.05027852964505938, -0.023181373520779914, -0.040770385011217124, 0.006449655550171255, 0.3827862728639547, 0.13691816844716637, 0.24114400524486207, 0.050594849027988185, 0.42941077055702637, -0.00406208586086053, 0.0088073860962128, -0.020069293934411446, -0.015821264337235394, -0.026037593454447297, 0.06083730123452551, 0.13700624085780472, 0.4279040778230757, -0.019076922638309474, 0.13830461838010386, -0.03154643517024589, -0.02912693732339937, -0.006394810862083885, 0.21655323469167317, 0.14352218307826553, 0.031116106445649803, -0.009354706833073729, -0.034264708040824036, -0.07817219730249754, 0.19516733879463027, 0.10718882117451285, -0.044172917427435905, 0.03091045411935467, 0.2184983720062914, 0.17801561402826607, -0.07354934824629213, -0.004678684591489826, 0.20265289335952533, 0.04942339624058399, -0.02398932655062086, 0.043573628098446365, 0.05396390519662372, -0.07766884754877112, -0.012064092011063824, 0.0024150806635372473, 0.2534004416337626, 0.07840141301737663, -0.01602433882826906, 0.16936584452461528, -0.029494194436220134, -0.007725420430953847, 0.17236769398106572, -0.0011283375228261262, 0.08760279659624819, -0.025270481302733513, -0.009192450327912507, 0.04920271169702811, 0.03031390114122932, 0.0810387910321683, 0.07522827861954216, -0.007875918634162628, 0.05953609499893474, 0.23638960080660812, 0.17170839983168962, 0.03421176323287133, -0.0010043074148764488, -0.05317317912487043, -0.0033753700964083404, -0.03098966722519757, -0.00040937056078636243, 0.07851978266570431, -0.05551824248200185, -0.004026933579386579, -0.03243970738629277, 0.29796142253049407, -0.023128675927545816, -0.058766666042424524, -0.05858282252478405, 0.47081041094013737, 0.04929344700278722, 0.358809799737825, 0.35813491819587173, -0.03129892267153457, 0.13890146999445138, 0.23088778053157885, -0.060122537841538364, -0.012882514647563067, 0.013423686392199553, 0.19927460189752072, 0.03045313674229635, 0.3286783167300612, 0.33428365692651296, 0.04424990222034194, 0.2741716623215376, -0.02337839300717166, -0.1067873404320899, 0.05166427788486225, 0.3110916115864789, -0.0071876601979189, 0.2171961735825326, 0.1328955929614434, -0.02801568246512902, -0.05241762232361863, 0.3271789551006585, -0.005278967918215659, 0.05522643176060992, -0.007434961852029161, -0.013180067375311641, -0.06411416528700614, 0.008529902946320977, 0.04742954124278539, 0.19907036926701632, 0.08648114528405165, 0.0381051456856147, -0.016986017805605304, 0.22303906956111041, 0.1533838261152568, -0.06568035569061344, 0.16998757392848873, 0.1871795399724531, 0.1301804231621502, -0.08189366242078694, 0.024720245912233922, -0.016541680113380426, 0.25360809080164104, 0.014500783618618182, 0.023438397197983532, 0.012467343212654145, 0.08741630160666722, -0.1511114549990322, 0.14207605261694964, 0.21974666719843036, -0.03636966870223533, 0.03760946844848808, 0.0005249738629659096, -0.018828219751442155, 0.13435820092571504, -0.06553961129209043, -0.03433459960541528, 0.06331359049302543, 0.07020686326437979, 0.14005710278103606, -0.06323210894880169, -0.012289774672644754, 0.06364057987827831, -0.0198386707608657, -0.07154616184946924, 0.060246574472149865, -0.02895489471975182, 0.003915430639575895, 0.2835335639329734, -0.08175971003514076, 0.002493473440216002, -0.04707713093394206, -0.040829405796660244, 0.02384847841799957, 0.5003656803284627, -0.07570223888038258, 0.2789619381862007, 0.25708118269851427, -0.0021155635762255894, 0.23335199306183313, 0.19568498500599532, -0.07098276689536874, -0.023319784809216964, -0.049838272766376035, -0.044061921918359545, 0.27964024773146334, -0.03515090869825159, -0.02478890635439099, -0.13386031926040723, 0.2530937960382904, 0.2329430159710308, 0.04148008967060834, -0.05012316362422598, 0.1487125996523817, 0.1447342370991799, 0.21002917772471835, 0.21320004215790686, 0.07012350851888265, 0.20564956915122792, 0.263723936862117, 0.18891029524744943, 0.06752895557100935, 0.1229508703902658, 0.2579356134043411, 0.05018132785896111, -0.036748727039828455, 0.3656520178813852, 0.0019200602116839765, 0.1086057376072701, 0.2477039078852304, 0.1649200502721103, 0.04860733648455254, -0.05521314650663429, 0.04806979851091655, 0.030772581410741563, 0.07628210072041788, -0.02356178861798859, -0.07403648239881742, 0.20624285439454387, 0.01408255489000999, -0.049310098262985276, 0.2782924123647222, 0.36078412260108045, 0.054544428461651716, 0.08487901213668304, 0.3345622519874827, 0.12791113553112554, -0.023129373436155173, -0.0030147594027469518, 0.06427644401853287, 0.43948691953031493, 0.1752349434881238, -0.047339590482097675, -0.05321424842187245, 0.06616152645246816, 0.03393442409220162, -0.03419945532152182, -0.03096907763719896, 0.04049839334016686, -0.005152951445279463, -0.033592110230801574, 0.09637781595696009, 0.24829940746146184, 0.24154145459732157, 0.18897384949202206, 0.13621989101779497, 0.2625803798044384, -0.0432423236719272, 0.2534355827435477, 0.12444506825217423, 0.2805195925845096, -0.0014601827360551675, 0.47447217045810325, 0.21279783054821358, 0.23414084794825035, 0.2979261572323534, -0.07883049793436103, -0.05488289431828454, -0.05909307099061109, 0.024884302514689736, 0.38731445269873305, 0.17490628448692813, 0.3939502692547385, 0.16759254765467582, 0.4340601880441201, 0.0023552332306071492, -0.02876838700909241, -0.04034454368297569, -0.0017457849072572767, 0.37603462076000854, -0.02323331549995391, 0.23216970803365794, -0.05068747855854632, 0.001075159787626586, 0.06646949891614472, 0.061451223037061846, 0.25210136148079654, 0.04458705344584013, 0.049034683988148565, 0.24291138074293753, 0.2450430663938755, 0.1919417377971496, -0.08190455894581541, 0.11504567337102818, 0.04719410843117041, 0.16433716224118916, 0.20353493588739843, 0.2249289269081003, 0.11209749521000399, -0.05149996619382122, -0.04744861270306109, -0.048685166099601086, 0.24162554768368638, 0.33983843498214744, -0.06956384299810046, 0.0009893001514616469, 0.09518001511163983, 0.09389660845061641, 0.10506827366521614, 0.23631377091934821, -0.018889237708189364, 0.27890080115265076, 0.11274179419866866, 0.22327026201390424, 0.3254424068552799, 0.1149905506429023, 0.026512684593354544, -0.010057343589596297, -0.08934377180171107, -0.022469562838892686, 0.2545618771400271, 0.011980328818302894, 0.33127825666104843, 0.1016776359066114, 0.4296762969408676, -0.011309877312214228, 0.23924550118348875, -0.04791958198328651, -0.056332975662102734, 0.2499818814563213, 0.14887730376568928, 0.06855202047468752, 0.05350934521232353, 0.06760056483937003, 0.08134149773666927, -0.0500516266750359, 0.13283003598518292, 0.2521184504232699, 0.34323086971791233, -0.16468812366736857, 0.07494640549957461, 0.028760858446766662, 0.1520401970212117, 0.018497932011221685, -0.059979985798246926, 0.10521936918233658, 0.081523926179845, 0.31101340855551174, 0.28146989549905177, 0.02258538712456879, 0.3004586681705083, 0.08400428248700213, -0.06032824239019539, 0.45328417612571587, 0.4972849244599452, -0.04421008037300735, 0.1415719235232587, -0.06066690705689025, 0.15805129794366712, 0.22691739798349625, -0.009747515293143019, -0.10438109974262319, -0.008526820126941524, -0.0032789824888826984, 0.23509957069122447, -0.0816012913792053, -0.0651707638662694, -0.05301755313921448, -0.04665782130407247, 0.07024346070912083, 0.04481199715554766, 0.24928263946961413, -0.08438108442887558, -0.009662214858718621, 0.2369001086962632, 0.2128128976964086, 0.03313451076466814, 0.04045683299305308, 0.3234087550507868, 0.010868376034665228, -0.03864740548921643, 0.16195023538205283, 0.004033312774944505, 0.15840771683930346, 0.36675190735228597, 0.514534738061888, -0.03878375699644199, 0.1266328184007882, 0.13051304965851482, -0.05277205043709137, -0.004050005201717561, 0.1853091485340316, -0.04529512797480974, -0.016476193448784007, 0.3769830693478041, 0.16893757062326317, 0.606532085172806, 0.22630576026969038, -0.03793842855475034, 0.39278270350646394, 0.35463111019672633, -0.06710181544241717, -0.04065808554643117, 0.002577109450147008, 0.00821380248437998, 0.1866630453924962, 0.005628153045932956, 0.03084655104538223, -0.01933081966325239, 0.2028485765093806, -0.06475939906465238, -0.016988859866503964, -0.01867973396083675, -0.04286258835155874, 0.10984463019161508, -0.010077985084329744, 0.2852052017039163, 0.02304649532017067, 0.04471599405714642, -0.0399481088094126, 0.22552575432511357, -0.05555597467453322, -0.07433462664131066, -0.0803631251179914, 0.06839876837584533, 0.31864485894163075, -0.07246937171102627, -0.025929001886492057, -0.01754317899908506, 0.11349727737111566, 0.001027493719781822, -0.06750822700646945, -0.0332798302494905, 0.048374447539643976, 0.42622945140098967, 0.1950323079245934, 0.13537677651594018, -0.04854768076595635, 0.07880766627598661, -0.06786224394960047, -0.0632896194502403, 0.43765414820131965, -0.019217222983793504, -0.031960629770520436, 0.0928275633898176, -0.05059695695952122, 0.13220178984394362, 0.12031874579562128, 0.004232491476636779, 0.22628884000553307, 0.4179401622427124, -0.03186363757174696, 0.08530107738113248, 0.11023536277640016, -0.04637932379894187, 0.0545640408061247, 0.30404413742280434, 0.2873024269050766, 0.18737459957038924, -0.02352661517705362, 0.056443429814251565, 0.25573959503524724, 0.13715410531890998, 0.3968513209928936, -0.00514957226010659], \"xaxis\": \"x\", \"y\": [0.05195956450823347, -0.0061461656752288645, 0.005217478925292565, 0.003056861056054932, 0.09181471914023542, -0.0682293894733004, 0.03415837166920113, -0.02992372806041003, 0.06790206160738846, -0.04740213160472646, 0.033678577301307196, 0.0678264439741617, 0.01197733168905103, -0.05375974682826955, 0.007172698447845426, 0.019747737595084844, -0.05383135716445592, 0.15883841639549814, -0.0062661938868849906, 0.03393207907313913, -0.18074056827202747, -0.03532992827286786, -0.16047878925136994, 0.039960104974028236, 0.031089344996359375, 0.04695350189466615, 0.03509820281781654, -0.016922607835968974, 0.13870883883697524, 0.13940329574473143, 0.07477076873326413, 0.040147441329727376, 0.06027140768475983, 0.03519816042524103, -0.015058697606657378, -0.09673799828729927, 0.0002303364387421255, -0.03184541079247797, -0.10818180862420004, -0.03239189555927903, 0.047895010187961364, 0.00825548553725286, -0.10745788597689471, -0.02799039613800091, -0.03519487983028893, 0.0435816984008306, -0.021028830977295715, -0.018016746420433556, 0.025010698870581438, -0.16453997675538476, -0.04786651518730275, -0.07056081820362148, 0.08560703290075611, 0.01782602694156271, -0.010888438294207554, 0.007108129411267957, 0.0016226834748717795, -0.04222575760809218, -0.010327409604542457, 0.037957042966145896, -0.03867941578580647, -0.011094640553949008, -0.03627871153109218, 0.04133579928953109, 0.035421565957754114, -0.1150567236328279, 0.04801828528575765, -0.06558929390245337, 0.0385970212661293, -0.04056687738256721, -0.07165953292147247, -0.04294880407090059, 0.007841905533520576, -0.12512015305521804, 0.039749785704175034, -0.0213780156061484, 0.020194782762432847, -0.02943990030510212, -0.03176466897337622, 0.0009175168592680968, -0.05009863234383704, -0.02786491162732765, -0.011611786967769384, 0.01735729230356224, 0.09603812841778872, 0.03362955602341106, -0.0313234915965907, -0.07599050903997044, 0.04104964217052717, -0.08188692321859992, 0.032006003063955915, -0.025834408049221803, 0.029517303513250524, -0.006953256076917413, 0.1469293955142345, -0.04569062640511301, -0.09489052658409856, -0.05040278113256108, -0.012348388868778898, -0.004153642812518902, -0.09643000676414673, 0.03324775344461616, 0.003506103505011436, -0.05875328628158669, 0.019100976129270968, -0.026789352695672067, 0.030074471430211053, -0.015767398042762473, -0.07192918419569641, 0.014777792771893053, 0.05699965280537907, -0.011521438347370913, -0.05108098298843454, -0.048537085905109655, 0.06878924884785344, -0.0051535216789793635, -0.052637757492119186, -0.005494385379585597, -0.022351068739215604, -0.009781330746045105, 0.007096552678871525, -0.11914116466346679, 0.12805144063194032, 0.034394957506834356, -0.07855441698842218, 0.03386639739457378, 0.0001591193886613386, -0.06442084742626675, -0.14612536141234142, 0.004598015902112147, 0.0015092692571937823, -0.039220067901008314, 0.010459366592189013, 0.03151976148865085, -0.016924879142735834, 0.043340367660255175, -0.09137320447998647, -0.031076793752785714, -0.06421414973837956, -0.01668079714200327, 0.13420573900584534, 0.004408641094878444, 0.033844663298192953, -0.03907064750235482, -0.00926331357720789, 0.013329350940532132, -0.03807748721122108, -0.07696459152080576, -0.04399706410353928, -0.016421240925789442, 0.02221938292756786, 0.010307084080012156, 0.01050055584381491, -0.052614056033854606, -0.04529884944842846, 0.008163843353732405, 0.011507896429149157, -0.23715447060587516, 0.05704066335649967, -0.020729821089117, -0.006452300023967468, 0.006131039616358818, -0.09778399584640374, 0.04028078573118414, -0.03674182882102158, -0.006590172774240778, -0.06067510697948267, 0.04402598594626466, 0.020722365660960907, 0.0112347899030881, -0.04786881967386035, 0.01479279417338533, 0.0016589275318651976, 0.04130980266488728, 0.09209823402748023, -0.037058259171781, 0.011622929235854937, -0.03221349542491436, 0.006256510587493802, -0.09989804584008002, -0.017482366422459407, -0.036194624222018926, -0.11754998145459392, 0.04667739930053429, -0.09730832759361725, -0.08140554865086404, -0.02226503319469476, -0.08842046729013071, 0.005639081346835565, -0.00475421118946415, 0.05473864794575506, -0.005672113834964393, 0.015361180278777897, -0.020385933303423833, -0.10181754342624638, 0.02928799827319611, -0.03562672817305954, -0.09688130403502863, 0.04799632883781187, -0.03176326333797806, -0.03263480986934572, -0.05525454602262452, 0.03809347096562364, 0.016480482155063574, -0.22329809741312476, 0.08134109786375704, -0.17241895902218932, 0.006407387333184428, -0.02409696491647597, 0.11390794763988671, -0.00028969596839126623, 0.024001993972547007, -0.0287695206817145, -0.027235921422533834, 0.026945786335714716, 0.034618050747976005, -0.029652009594207388, 0.03526029198002233, 0.0026135936842740584, 0.03584957157506773, -0.14660306480843666, 0.019110025346256998, 0.004682858765940688, -0.15200040779358986, -0.06006233121872133, 0.015380767042094104, -0.05223124086707953, 0.022312628555716627, 0.008129116374488819, 0.0007140643067864463, 0.004862066012405158, -0.1933253302607448, -0.03498095318297547, -0.08491604521576541, -0.028207351773721967, 0.036119216507094515, -0.082569939882493, 0.023869261784265863, 0.018653229125181924, -0.026124018461100174, 0.0017525292665933525, -0.11852854045645181, -0.030495349322712473, 0.004361620864496572, -0.013595635405495247, -0.048646313793230894, 0.07740332392010174, 0.01835453937724974, 0.01564861559286261, -0.1630250820712897, 0.005082364590549357, -0.0007009558828316463, 0.0012629127672968128, -0.06257329405735347, 0.018045435690304715, 0.010578967444586094, 0.006332435067860644, 0.23719324083415763, -0.04199509851986486, -0.03053427239927891, -0.02438077167045169, 0.04953135833494877, -0.039996043447457594, -0.011984187986227236, -0.01843769527558687, -0.020555543667800606, -0.03626162378828119, -0.023107166937211297, -0.025756829158571345, -0.05005413646796369, -0.017490095625784595, 0.04840604341864208, -0.08620327239407692, 0.03668193822251226, -0.1307162020274519, -0.05936660830509858, 0.02660850265265216, 0.005550493078611969, -0.0775392132908151, 0.03456178466542507, -0.05113136065845055, -0.042469827990182125, -0.09160203333979464, -0.09177261956952562, -0.0012567029310124032, -0.018898491182286717, -0.006883177196293772, 0.03663164928971525, 0.023496075522605596, 0.08093246863401271, 0.005074278497943749, -0.045432236771958574, -0.052765878982356205, 0.07401402627094468, -0.007576769337295384, -0.10026001705005139, -0.1045307416388653, 0.029350486303492306, -0.10338882646523755, 0.019307448153771948, 0.020027248301372723, 0.03517156793652139, 0.00012705050949374708, -0.002836897367191843, 0.040909519610033876, 0.04350889963392444, 0.13940329574473143, 0.03578263263384074, 0.035808222494740756, -0.10429548428878131, 0.06100354716272202, -0.0047662983823064595, -0.028304088320732317, 0.058916930264767686, 0.01739865258721537, 0.036800323248272475, -0.03876240814690782, 0.043158980309661615, -0.04172390401313564, -0.00509024178901851, -0.01825302037143675, -0.015347748708056861, 0.015444544992374893, -0.004034839122083056, -0.05040426938278314, 0.018365823238388854, 0.04600872576120874, -0.0586609014119257, 0.036635487922180646, -0.06875921064312547, 0.02822273597668578, 0.022623722810438236, -0.052714564117215375, 0.022850452650315652, -0.03814503525058431, 0.014355602000208674, -0.003536297753240719, -0.04482285258618996, -0.04789729001706203, -0.05017756992447616, 0.05686607788218095, -0.15170239770370092, -0.0006384534716931077, -0.09256706625620746, 0.04731683685370832, 0.004029461960417142, 0.04965785445990388, 0.061319819005533585, -0.03802286769183873, 0.02443067596566773, 0.032104591251408264, -0.06392831860501777, 0.036983707517243364, -0.08902371906591046, 0.0006910822109593748, -0.005542381978905215, 0.035529837020585216, -0.040425439180390425, -0.012833164203046447, -0.03310292546039563, 0.001202209984912398, -0.07829654274789974, 0.024333849283505352, -0.010444869542164992, 0.026484870972886033, 0.0299246124389447, 0.024679243579338903, -0.03244061767102491, 0.03453727812924313, 0.014591889925952688, -0.005512691326333094, 0.01346300995975156, 0.04539792020087322, -0.12923068135259472, -0.0008788130265355938, 0.0071858212195227275, 0.04289378192691768, -0.08740020640504811, -0.028409316586805858, 0.04096614965264706, -0.01144475888512788, 0.031387754197605575, 0.005829515256216457, 0.008496079410063763, 0.01900729198164092, 0.008892487640163283, -0.012499419225464558, -0.20201231722479224, -0.09239851147607042, 0.016822147295064285, 0.10929205398135416, -0.020911544072944363, 0.0013360358803470942, 0.03030189025167717, -0.007804421053332367, -0.06907908148002126, -0.06451930740572066, 0.09223897430863094, -0.0530261160373234, -0.16868500806221282, -0.011620978687602661, 0.03620711027748687, -0.037439078342726556, 0.03165933625757661, -0.2120540154239479, 0.06711102668941857, 0.019144766652632098, 0.07733736480521067, -0.0034688361409697284, -0.1094400941194234, 0.02975486235121985, -0.007016867538580657, 0.03659947669021753, 0.03210035853086404, -0.05616466378328746, -0.16935005517814314, 0.06698416563122135, -0.08562414446529507, 0.029562164509866946, -0.02368890971649179, -0.007948845284033921, 0.09582173697553109, -0.02835440227816226, -0.004713400446335333, 0.029800723438139113, -0.04595328478963642, -0.013755850843706988, -0.04880871028034682, -0.11488952338750355, -0.029297421620404548, 0.02581306715982451, 0.035521249841924986, 0.11592913343895256, -0.06292316498685643, 0.010697123213431439, 0.003722739847650294, -0.10825982512304565, -0.046134762046737984, -0.050231044153848735, -0.1883109771011233, 0.008965323184628362, -0.07132261183225888, -0.017716561837513314, -0.05994256419520013, 0.03237037648319781, 0.04523753714183157, -0.009720635650396353, 0.11582662705537326, -0.08576898392539616, 0.012771579543360717, -0.06510351121326671, 0.07458834055488367, -0.00015065927011729353, 0.08332773492346779, 0.06973553918702202, 0.022706852686987297, 0.02210152935713811, 0.022083406365978494, 0.03212661416275049, 0.0015296331439749952, -0.019999102531509837, 0.03148958138111567, -0.08788119418585381, -0.04569777441334687, 0.013406617121111932, -0.005080665240811649, -0.07037769573685777, 0.03932816078259811, 0.021911086815429564, -0.05731757906476618, 0.02474503361323892, -0.002058111517278431, -0.0762339380934808, -0.13385206262090146, 0.09147343960339452, 0.010972470643750619, -0.08570376382486662, -0.059465297492458956, -0.05383603983670622, -0.07763517384276666, 0.01988819976697407, -0.118780130660792, -0.05501966164238222, 0.011694034052127187, -0.026805707364122463, -0.00837565037900639, -0.008533764219280012, -0.039634197226928916, -0.09616715307567761, 0.008717247499815655, -0.02970600833991989, 0.0109438761447971, -0.02712251030416956, -0.07031160304446407, -0.11476321694377219, -0.00744385603482789, -0.0643515805112239, -0.10424218620726379, 0.016851227673672538, 0.023721182210128706, 0.011241781446461973, -0.09159638442125462, -0.044090816977730714, -0.0862132404933174, -0.15106936180581257, 0.03335214476901735, 0.03967131793360985, -0.013207194286452844, 0.06396487104520678, -0.00937221433011332, 0.029359774138223056, -0.03582094524769978, 0.03790687347564915, 0.008475787280591902, 0.03432668200336679, -0.013133458749117902, -0.13836082881857092, -0.04681852246136868, 0.02287722877437571, -0.04786900622540974, -0.01801842765100272, -0.014538499204789902, 0.002511361588462403, -0.0060143600405537425, 0.051388998564234074, 0.015924339564782546, -0.025415903755522388, 0.0405503232064805, -0.10340813129084928, 0.07527877606308822, 0.02563841761828219, -0.026106731696543747, -0.046088118657721895, -0.1089621877154583, -0.09302638864112865, 0.05632867723872621, 0.011596315660132792, 0.038324242858107774, -0.05139275243279579, -0.01276405353794704, -0.03690346543197565, -0.05128207657916194, 0.0248101508882974, -0.006841898360870657, -0.12647133449205178, -0.015974128286205452, 0.031147376561773972, -0.022270313707350237, 0.02825167652884086, 0.023950702009711555, -0.05309481746000615, 0.10313127411139839, -0.07913542803295726, 0.010688135684299783, -0.10472196967215704, -0.01478749110513729, -0.00019344258044784018, -0.11807063127666283, -0.08828369601463937, -0.08529103315614346, 0.0018043370060269613, 0.015068235693124463, 0.007056307052625793, -0.04450662738204091, -0.04847610255124684, -0.05486741328410615, -0.06364502314807353, 0.025308076123256487, -0.029123422566937614, -0.35656855133329607, 0.038576001314809365, 0.059860532977597136, 0.03712377925095089, 0.017299248949246775, 0.05983829689939807, -0.07540282832184109, -0.026206111003459567, -0.055557459685007075, -0.058144775662189195, 0.011427447961819959, -0.025466758357396355, -0.051941137401247575, -0.08199684528868648, -0.05751499880981004, -0.06051749249184176, 0.03395588419210551, 0.02351230331493369, -0.08871224253359641, -0.1953939320175951, 0.012073235829446733, 0.03648282547667811, -0.27033339677552776, -0.03970064164374175, 0.12568610951851814, -0.09467244820854402, -0.026577754234597697, -0.06676397433188795, 0.032475974586266804, 0.03216046098567071, 0.03802058297297608, -0.11935861781529905, -0.0014651654132553798, -0.01344922098564219, 0.04619579744783754, -0.09410422843823313, -0.04007906875816796, 0.011276594985549627, 0.024460422696424747, 0.060409740454332075, -0.007161188512480392, 0.029331804651466373, -0.048544993353824666, -0.23866481113633353, 0.000905369448290899, 0.0036756600820333513, -0.11290417966994434, 0.06327473147875175, -0.07856862062387512, -0.02684637871580328, 0.008964848885918388, -0.11683263599791402, 0.024371951547166357, -0.02428032709697692, -0.02424987279056649, -0.19341164473191694, 0.01029351613083851, -0.043531336586499526, -0.0033128077441815225, -0.07938333360425416, -0.0017285322494284812, -0.06496714893482716, -0.03580554990190198, -0.027412100824162185, 0.005209586828293851, 0.034339218911661175, -0.10024999032402043, 0.03404028710354513, 0.10040308911060017, 0.07585981623069939, -0.013948786261425348, 0.16371394125764618, 0.032642091313840514, 0.033824567365787345, 0.06282938068922142, -0.12530214508671506, -0.007052952925927401, -0.016874323759460904, 0.07012587453151618, -0.0555474000428475, 0.028265327593270523, -0.06594978365992547, -0.009079147830569007, 0.04834692891208126, -0.05076101857665506, -0.09535286205585831, -0.10130377404922225, 0.010839120409357807, 0.03338313711503637, 0.047824625792326346, 0.026047553102552648, -0.03268419976424619, -0.022922425650551085, 0.017086543820955667, 0.023137415163794573, -0.19469149985283166, -0.053935778400766014, -0.022351068739215604, -0.011482950707571155, -0.05674311843483915, -0.09212842361475117, -0.006293809222502238, -0.07241013795610658, -0.01718705994297403, 0.032689792314835486, 0.17690066671782223, -0.08983910622021743, -0.006214934687524506, -0.23969445564959962, 0.03134008226640742, -0.0781266641246317, -0.006646633500605785, 0.015187327772332684, 0.018557336379763104, -0.054770797515679974, -0.018463854645319622, -0.06041143195984905, -0.02874959255534776, -0.008976175042017401, -0.2600829124926817, -0.0146009661014556, 0.017158716161693753, -0.015209576786303537, -0.08719867289663191, -0.00788588851149213, -0.022917358007127003], \"yaxis\": \"y\"}, {\"hoverlabel\": {\"namelength\": 0}, \"hovertemplate\": \"cat=\\u5e73\\u677f<br>x=%{x}<br>y=%{y}\", \"legendgroup\": \"cat=\\u5e73\\u677f\", \"marker\": {\"color\": \"#FFA15A\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"cat=\\u5e73\\u677f\", \"showlegend\": true, \"type\": \"scattergl\", \"x\": [-0.04460341375074778, -0.06509540656813427, 0.10203877550843059, -0.04177714126880423, 0.10330892265036382, -0.026676325933590067, -0.09079988970998995, -0.08713020602471068, -0.10260413489359003, -0.044411857059642436, 0.0033212506365632877, 0.05397220421214726, -0.061000736765372494, -0.10139957383441364, 0.20178802012819624, -0.0687165051209451, -0.04306919467308626, 0.14956349586039583, -0.020234569435764908, -0.05400459700237516, -0.021062193716613362, -0.02528395449558082, -0.12855905380179417, -0.03800522571004048, -0.06191350512621389, -0.06147059311760969, 0.04008795274214284, -0.04696430782046476, -0.03918954380973378, -0.01921888216646099, -0.061952869483886126, -0.1493857671234115, -0.010044094441455912, 0.3673519689855948, -0.048553161837975915, 0.08628878501170696, 0.14543805928900724, -0.014726293929892666, -0.06237092151921958, -0.06261286478042469, -0.11415331151797259, 0.08062205694092792, 0.010829137260342351, -0.06642494066464147, 0.02688709340300124, 0.0018182618385679567, -0.10318015940825659, 0.03808298584715919, -0.04196612247435676, -0.090294682610452, 0.03596922394148144, -0.02103621509148729, 0.20923976871450845, 0.04973402486063485, 0.0030171034778612893, -0.09159760264042494, -0.07381354878920161, -0.03956057955787322, -0.0413749669350436, -0.01722802503924632, 0.1078905026705044, -0.010142967050503535, -0.020747838953458022, -0.05017259603461722, -0.02362221277017114, -0.044029460687258326, 0.02771323113176153, -0.08072148171106215, -0.04477041848613227, -0.05816870355106515, 0.024443290129385904, 0.013956944368525046, 0.2421366483068628, 0.07137988205597182, 0.0023388772995703738, -0.04869071202192176, -0.05854020515311821, -0.014263738069547298, 0.007265308298311783, -0.05527755235779083, -0.10167262278544391, -0.04853061586788163, -0.03667436441150404, -0.05654490379805895, -0.015920546897994065, -0.018226281125839832, -0.05428927089985406, 0.1934323230828299, 0.0010651786982147549, -0.02257584135422615, -0.07253588000058102, -0.012852062398116515, -0.02142123183884055, -0.016861963431324487, 0.11488500622152598, -0.020434886532458202, -0.0835115002466227, -0.08708435896738978, -0.011044655755136959, -0.03263949203191567, -0.0547244263723647, -0.03541751986123961, -0.13679932735976116, -0.07314016412158014, 0.03230884847324041, -0.018325455031065294, 0.1520398720971235, 0.051365649160344994, 0.028823853849282002, -0.04241677529825623, 0.17227466530205213, 0.1341271889179857, -0.07603376277921353, 0.0794455860082602, -0.07234396821994284, -0.01134366553594515, 0.11979059497607421, 0.0345566737316679, -0.04889612851654817, 0.5920951637461059, -0.022090662963705078, -0.014715938325304065, -0.05410865059197378, 0.07174469738684904, -0.021826698003298495, -0.0061850733188131635, -0.07725459322707205, -0.05005762127256459, 0.018910455870577096, 0.14920200081419738, 0.15399068465316834, 0.08189104718570295, -0.07039604095754408, -0.05854899230917679, -0.13700112740911166, -0.05691250296604183, -0.02805541656083642, -0.002099365528710836, 0.01386878793091073, -0.043784599052288604, -0.07564496654411697, -0.028069502334657413, -0.04456122741970854, -0.05724317225059074, -0.12849940821282463, 0.2673364922050609, -0.07245853383272069, -0.03474846370568663, -0.08306038803547897, 0.10439930555998822, 0.12970379423924994, 0.053987909081529104, -0.07404695623312284, 0.14110024220218806, -0.01412832278500294, -0.06793386276611009, 0.11993067120449946, -0.04290418474879848, -0.06309926654253917, -0.029202799985037598, -0.03702956591748296, -0.048856902948961195, -0.01523553730974154, -0.04762694092743481, 0.1394841695855168, 0.10845459521990085, -0.047885909844221364, -0.01769212041904697, -0.02592645433071588, -0.019914749840952005, -0.07768001920526825, 0.025574207883693884, 0.13485391579357922, -0.03862560759623046, -0.04840035274169568, -0.02974769199495462, -0.01190279819734563, 0.20736830975506307, -0.018851678393076568, -0.07082577730594819, -0.0632169394345686, -0.009596094520863883, -0.027152170801685478, -0.07731067266675416, -0.052270445420118834, 0.013749051558779746, -0.044915230377272845, -0.04715972892827181, 0.005492996345682565, -0.06755516434526292, 0.04779803609610373, 0.07454050606395693, 0.030891280954487614, -0.0674934026104416, -0.03621620853936715, -0.10893374125042171, -0.005894258084254334, 0.019346047864187067, -0.027096357093674178, -0.049694692891812775, -0.03416388100224647, 0.03964501664208071, -0.001204183563950936, -0.04850478052820973, 0.028907661164633367, -0.043328046710052616, -0.026054328808313442, -0.02201103399699596, 0.45613868430808785, -0.04826617880224771, -0.15497597526249376, 0.05194804462426019, 0.12729738851973088, -0.030965070036102205, 0.025679129207554358, -0.03561263396576771, -0.07522957197154946, -0.03217721960684724, -0.009320453477710924, -0.017229277797041746, 0.11916232167622867, -0.05967825689261942, 0.02095755300584248, -0.07140445909901887, 0.04096601065687577, -0.02803044233095545, -0.0425932780204573, -0.005147744191833487, 0.0934041353367147, -0.0902896489991605, -0.03490537625487037, -0.0175766304709631, -0.04517115509159899, -0.025940272377331496, -0.029511532675113333, -0.006805109352932344, -0.08341374599326075, -0.023999595384005777, -0.004341536027034211, -0.009184089968042068, -0.040319275488424317, -0.027862990846450466, 0.018040860345373284, -0.030184423049818197, -0.10260137294160897, -0.03524849560806595, 0.024825655238839544, 0.02138884447913593, 0.021145908602913262, -0.10336867426682642, 0.08099691847318308, -0.10344408287468111, -0.04365755620782259, -0.06299754021748559, -0.010809039068265105, -0.07090517197052065, -0.06354001602878294, -0.03185820259053489, -0.06010526546045046, -0.01472078928385781, -0.07043870856727452, 0.030465324082154313, 0.05708580231541283, -0.02603090227453731, -0.06853658283510423, -0.0991526936732165, -0.021682704505095823, -0.07528108296525914, -0.10362702960413982, -0.014002592207850693, -0.009188052979759147, 0.004908020517616997, -0.05832800646114425, 0.02004117257198061, 0.10402702070511474, -0.11273236930502327, -0.10634963898558569, -0.0861807891433023, -0.09552743412796406, -0.06480219166899968, 0.5920951637461059, -0.010561365821300652, -0.024043720582324887, -0.05899020866613867, 0.02458054696678382, -0.05918246819343643, -0.08906804933178963, -0.01322039082436485, -0.057354802870399166, -0.029499172186657395, 0.01880422432527065, -0.07968403882886638, -0.06587000411371173, -0.01866777614531914, 0.026296275772645896, -0.0435545947290123, 0.0015517405017658507, 0.5920951637461059, -0.036367974615391006, -0.06602509872573546, -0.0449714612074145, 0.12980365969915858, 0.00694510631059807, -0.005634737188380364, -0.05184818905041771, -0.10389760784271644, 0.10075525545880676, -0.03271648674717404, -0.0567812697382905, -0.04975466806334017, 0.20320239019986278, -0.04544062581662634, 0.05535601129923969, 0.1745665516719963, 0.22973018251065164, -0.012874834139000207, -0.07982427588383345, -0.09015971945409794, -0.08075548749787048, -0.02050949561282509, 0.29817588310159376, -0.01161766198240114, -0.0532662218252603, -0.07575470899784276, -0.06639213650243606, -0.025729816363899045, -0.014045019125972056, -0.04291944019430104, -0.06848198070816015, 0.009456482405176441, -0.013236751442910694, -0.04518973256728307, 0.09935585810321021, 0.01951236511422067, -0.01813508549995744, 0.11455225861878432, -0.08265089192442518, -0.020960237955076743, -0.01780789508696337, 0.014184816725618611, 0.06440241346482943, 0.06990248963117814, -0.1424293040414454, -0.11356734102296111, -0.03215394993282604, -0.03676652996425828, 0.10312964460100664, 0.12682199415286474, 0.0693492232512343, -0.07259877034417193, -0.10731142326024047, -0.02890128874949641, 0.1252416358587299, -0.07998796221264066, -0.09359997004985436, -0.03953906928194801, -0.0692021212682664, -0.0012947791134105746, 0.07255300862053513, -0.04528652213634818, -0.006535972481729172, 0.04306906101208152, -0.014741606991280303, -0.026848680912119414, -0.03772643678678193, 0.1000992590884356, -0.08196391549553825, 0.03795315297404129, -0.015801622471228485, -0.07293132301115765, 0.05876923903564364, 0.22917559303644672, -0.08379658430685516, -0.04473170992222686, 0.06839391174225586, -0.03074903313972581, -0.03337493571429365, -0.0321331973110805, -0.07626237799627773, 0.07034691225727516, -0.0256825361952751, -0.03537430315547428, -0.0029481232036161435, 0.4084177590804986, -0.0711006684792083, -0.029766382439115346, -0.03586373921844368, -0.016195059945033577, -0.030744199930489824, 0.05754517633936146, 0.1565578498642207, -0.028278222656762057, 0.24068763964279177, -0.0423345360045511, 0.10870231721185883, -0.0007221812464041708, 0.16257629337720345, -0.006957513632189027, -0.062439186028611766, -0.004299982818701085, 0.11849212521412043, -0.0306583500334887, -0.08665018794958969, 0.17319069972727058, -0.06913318050789001, -0.025694412046211416, 0.04707130547827376, -0.07974121267675191, -0.08810761919240273, -0.013077472912632462, -0.034260957217010175, -0.054568654282654394, 0.02450959485407802, -0.07214925979219977, 0.15181649320880056, -0.02306567466061357, 0.03515616779021852, -0.0975160686534667, -0.03808076335734595, -0.02442509599816157, 0.07683123919672229, -0.021460916463345435, -0.015991535684615338, -0.07206363048972707, 0.011265504283855554, -0.014510047116401355, -0.030677908751003048, -0.14927860949544916, 0.10649968057575361, -0.08784161476167886, -0.09010588575275828, -0.05974219364377385, -0.12383712041807815, -0.03410669609670913, -0.054946479197140755, -0.11803946798625908, -0.02148324249401722, -0.04718458238860775, -0.05524107440218896, 0.038967244607232575, -0.019536421672261023, -0.04327773483159413, -0.040884469865729356, -0.11957102457121219, -0.01835132239695636, 0.2274718752388787, -0.028666468698732794, -0.0350291403288999, -0.028251157559993564, -0.0684382282975863, -0.05974711886424758, 0.005730301831336494, -0.018854606560217875, -0.02953066219795389, 0.016987359653326106, 0.15243898750947557, -0.010603620325349023, -0.09133408576247115, -0.0078073351025654564, 0.012116717064711131, -0.011692047491745254, 0.0710574492158593, -0.10235674922102621, -0.07273185716841886, -0.014104531422390876, -0.028752952191328535, -0.08379800368771548, -0.02997635387373708, -0.04281054376301915, 0.06172436148372946, -0.03276704989162726, -0.02180920461421802, 0.04447792597593373, -0.025761905726339022, 0.06841904535675951, -0.06294168062765246, -2.9869240412867258e-05, -0.03971982333932302, 0.03702903211903412, -0.04744072797355364, 0.009963260139209172, -0.052459464700790236, -0.13910669398618306, -0.050547827530051874, -0.011263407046287081, 0.13309346421574916, -0.10047461054442278, -0.08754067545304978, -0.044002642967605275, -0.04587962156924845, -0.033193458179835066, -0.10677944213637985, -0.03334135749979406, 0.08582987510948464, 0.1668791967528297, -0.057374382351867155, -0.0610473706819861, -0.07052405491988183, -0.03158024420084086, -0.043621885258496675, -0.06596319953903673, -0.08076780519242933, 0.0390441719068612, -0.025930797276780573, -0.09998200245130631, -0.0967847943518539, 0.0057050599430636785, 0.04535992124984589, -0.02165838536300174, -0.09958192325466635, -0.06458552048923887, 0.1572478970614506, -0.07641965043136009, -0.04190339237203275, -0.026166795503318917, -0.04816014444089832, 0.02392849369254471, 0.006796205696277025, -0.05153128118996675, 0.003118487285833916, -0.08843348844276153, -0.03299810563273848, -0.01983742939533668, 0.08651241154889835, -0.026094000737873962, -0.05965744145386048, -0.16354968909410694, -0.024184342505460373, -0.060713337288551564, 0.019531449553407076, 0.06774741648755073, -0.04782058189253495, -0.09875614511363251, -0.06930619545877821, 0.2034417595852486, -0.05966742334854839, -0.030575888239071026, -0.08085986140374057, 0.16991636521501333, -0.10934985173887238, -0.04341133156156959, -0.03280594811001356, 0.04243972001675937, -0.08228543886641765, -0.041185136859327155, -0.07310711287937485, -0.06299963036932055, -0.06958418867077035, 0.010467912187749152, -0.08836860796247531, -0.03169564472863954, 0.20334559567626276, 0.040054267165027156, 0.0736754236457745, -0.07729731571620879, -0.030814737204386496, -0.003930850680528591, -0.0026546930432754688, -0.04679550535657264, 0.22614645028028785, 0.1498371362515962, -0.024498374033406014, -0.045022257410308485, -0.028371980716932525, -0.07234665691662655, -0.06515627320675721, 0.1738728808792207, 0.09141901434361799, -0.08945091788584938, 0.014287541796188425, -0.021302119201224472, 0.11815857677047727, -0.0637948105739752, 0.16849991287819407, -0.0843762624792354, -0.055646846576996076, -0.0755567977762715, -0.004780493765399023, -0.010248286698257902, -0.02801794114745936, -0.11677166616826963, -0.014201898515065014, -0.041759562141671196, -0.05069439395586878, 0.2441330807561216, -0.030644684443688685, -0.044752469423042604, -0.08110042875793189, -0.14786280290219123, -0.07640716336679475, -0.0319989458521742, 0.07938131497172964, 0.22032294613123796, -0.08344285663344332, -0.012513638620681097, 0.24440984504359914, 0.08820004061073604, -0.07340474330636187, -0.012513638620681097, -0.03903862627633895, 0.18513908878105592, -0.112845896166004, -0.019907209174345197, 0.025559388654673738, -0.07672510216906966, -0.07459340261549577, 0.1784060762180884, 0.02850865658547641, 0.12388236272370473, -0.07904154086672253, -0.03819782136877887, -0.010284431387677309, 0.12003924710843339, -0.004386766472297704, 0.06919827139242718, -0.0825960751958661, 0.000807364330920104, -0.001105550475440194, -0.04557823274429611, -0.06655306879711052, -0.010607737384843868, -0.10982727681364933, -0.05413165066351359, -0.07098434084551053, -0.0071815504462842015, -0.06298688274428103, -0.05438499336410097, -0.057492575763101204, -0.01902034930025648, -0.03099076264244524, -0.02927087870442502, -0.07314589994652305, 0.024382460291477646, 0.07518828756534864, 0.0718752759261603, -0.1463555964074465, -0.03440472622779246, 0.012763836908739869, 0.011520299381996639, 0.08720878958143445, -0.03681850116923686, -0.08043323837388916, -0.10346925166139341, -0.0844355982217134, 0.07878063156945468, 0.1475495271260637, -0.04698316471857688, -0.015293910722094402, 0.1589369439477588, -0.055886516132175046, -0.013443958041009715, 0.08581593366991302, 0.17049267501942458, -0.07901874739246952, 0.04587899726442034, 0.0321169639593918, -0.1279818313501487, 0.028340930699863213, -0.018456827159518664, 0.42924945322345087, -0.10260202164661053, 0.1371706867557533, -0.10696346162706487, -0.05669818227598378, -0.02666389793776743, -0.06986072886459752, 0.17068743958944435, -0.03234526461009543, -0.024062162024310482, 0.025536033769825477, 0.05809823862022913, -0.021216435642000573, -0.12181514253823657, -0.03698183508264767, 0.10543970125304707, -0.02779976149470589, -0.015129978831396851, -0.01043535548101905, -0.00015164302283437176, -0.002711789208628973, -0.11896604331614433, 0.0960803043946773, -0.01129407595907808, 0.11026277272323176, -0.11848479387515051, -0.08794414407942253, -0.029039111868543004, 0.11963663555201265, 0.043338943903999644, -0.09021405685667679, -0.10622166007713382, -0.028893394200068274, 0.18823804656480597, -0.06970426541793524, 0.0026502755659493244, 0.02432981906702283, -0.09933855927022114, -0.06303600147837334, -0.031019743138550788, 0.11699875040505674, -0.048135165505437415, 0.02807426946545985, 0.13758204819591427, -0.04985285131944276, -0.06573256484176337, -0.07582959951539914, 0.00762350783710311, 0.1677454269700482, -0.009591528228096237, -0.052013375222586075, -0.04058499042302413, -0.021481646422848312], \"xaxis\": \"x\", \"y\": [0.04280254912205563, 0.034205379662848506, -0.017242710271142582, -0.018963150198026107, 0.004279255624742263, 0.03677226360327877, -0.05962081071167486, -0.11681112231093578, -0.1337956918014346, -0.010680789086667126, 0.07310859114072686, -0.06672994913133237, -0.011408416263403817, 0.012042248918524812, -0.010863640940172566, 0.04954935661578948, 0.0430511235947433, 0.004197140971389559, 0.039555692027583786, 0.030214553606755446, 0.010649176776348381, 0.016736439357339507, -0.11559979579791654, -0.05445029798693473, 0.04579121088897451, 0.015446636796776854, -0.002703206451806229, 0.06916475902192161, 0.06485400439041374, 0.02229100723935703, -0.03893378268147784, -0.04400063649250435, -0.03471445573335483, -0.09088469303521551, -0.042559875426151, 0.009683224606731015, -0.009563284627351972, 0.0380715063226495, -0.00045547462917204973, -0.07707295245273367, -0.018325428711972264, -0.0222396311648551, 0.002940018902452003, -0.11821010244782648, 0.01632692746796509, -0.06563484642226419, -0.026974079017024862, 0.004477093153854834, 0.032567547352143476, -0.1461983524792857, -0.0001871425467071968, -0.001769174876847772, -0.030862950105915943, 0.020784133220276984, -0.006147886592717472, -0.04651565730570041, 0.02872985784070786, 0.013156646620805599, 0.08066553170415726, 0.025024161366553485, 0.036396212717321345, 0.03638421380264995, 0.03294153405134258, 0.035933781563208374, 0.04901021428711113, 0.09130194835710909, 0.011356955446828904, 0.01643558289157409, 0.04217373347711698, -0.010242865456256568, -0.04980617583393693, -0.03176435253463715, -0.08641203460458427, -0.03004536546577878, -0.049802469506182084, -0.0165259839376479, -0.03315908600754882, 0.022305021985690916, 0.015929588564729515, 0.01904370368625217, 0.008236509570347727, -0.023892403831971252, 0.008196232979126437, -0.07995139782963741, -0.11968832211300792, 0.054041067254039586, 0.013079858485680954, -0.01984248431450207, 0.022056932737947614, 0.014628631751683344, -0.06853570852788714, 0.03300176042705959, 0.02852239663509813, 0.03593670309397971, -0.008656930674286413, 0.03484537652101195, 0.0339870368338407, -0.007417514070571883, -0.012878073721885912, -0.005857742871094879, 0.022373216077376527, -0.05911320949467016, -0.11902298044537886, -0.05954896736995568, -0.0006350040449589463, 0.018244621800217742, -0.043215747594827866, 0.0195176848944297, 0.003316055754909077, 0.020237841789284467, 0.018307105229093294, 0.017256542160799127, -0.030333946547543258, 0.02780917675753416, 0.08694241437331167, 0.04571211572247935, 0.01896635424042893, -0.022741096389176076, 0.026180776839412778, -0.024979268951373996, 0.04378812018526407, 0.036810715181630656, -0.03715771162922924, 0.0401328479907952, -0.06101426859414696, 0.028611044879047358, -0.005499311492634257, -0.10158371671873802, 0.011997738624505316, 0.006648682146962719, -0.012483098249534044, 0.009070951333503876, -0.04196123249240248, 0.08031670163149175, -0.12782214689759022, 0.0010365480865858953, 0.010725848170715641, -0.18455532018663326, 0.0650468193452169, 0.05150199685548269, 0.11261034858428151, -0.07319265131580312, -0.03281784091741651, -0.07536909775583973, -0.09825008054242554, -0.03655091987202007, -0.04301367039272088, -0.034574801741793504, -0.0033885448911717777, -0.03918704189243808, 0.01726019338616711, -0.0619951688274352, -0.044690074571724296, -0.09294636003563846, 0.034618050747976005, -0.020170218425784956, 0.042253329363554915, 0.04925769812805386, 0.03290634425331339, 0.005132964557727689, 0.005954033575682443, -0.0201569585162594, 0.009855681252961598, 0.026090762888843152, -0.0497313929506261, 0.004055250856659237, -0.0662100141064954, 0.038119276877517275, -0.02148307114592243, 0.04341795633867716, -0.0032173708415899806, -0.04131533035050612, 0.018744714566395895, -0.07143662910977111, -0.13658161239926428, 0.028109721958696387, 0.03833821404103863, -0.00848806501429399, 0.047903883139870845, -0.24936014378945776, 0.10224786750367584, 0.03514053460323652, 0.008965265349122706, 0.0026536717857498803, -0.14214023172801746, -0.1317881551573969, 0.049194020626673336, 0.008242314353122093, -0.07173788304142574, -0.03171963565985111, -9.421003105937932e-05, 0.061249459762080974, 0.004944810507745959, -0.027694215738500266, 0.037992344334624444, -0.07030052447057435, 0.03739308535619553, 0.05158252079825514, 0.004806965523656555, -0.006148721801382595, 0.03203957087906747, -0.020403109007620792, -0.10864742598822175, -0.06098997543916577, -0.006956049094644623, 0.02027387128698334, 0.027261465240306943, 0.04292930885024759, -0.042557619372430806, 0.1348719905260353, -0.06481084592721544, 0.08634915241089494, -0.013210347228516132, 0.004642265073691639, 0.009611866148666423, -0.05371326024499826, -0.07148103663643746, 0.01927432533023018, -0.012600451187052265, 0.03309748227406985, -0.02081178963539552, -0.011283497356329926, 0.014988600377206906, 0.015551119202866778, -0.11608752054446583, -0.007028370835311746, 0.004191210275109616, 0.011742676139136172, 0.0167658196228886, -0.05576872680090197, -0.011906639858683826, 0.017819706516404882, 0.04344903225068216, 0.012946006642687672, 0.03359507790299764, -0.07621528688408982, 0.0395993520740711, 0.03355846359641637, -0.02446023685744695, 0.0365934785788628, -0.028114662496194546, 0.02195260598835676, 0.002515271590740463, 0.04141405271255163, -0.08295090304836826, 0.023714775575822664, 0.04083450245804251, 0.03379166313588186, -0.006280462914642793, -0.03983598246331096, -0.02627505726039203, -0.0015637106668969865, -0.014709704164439834, -0.0647533942392482, -0.049405903591811774, -0.028205163278014516, 0.034276104550102215, 0.012345906375056306, -0.045952323672541616, -0.08400071803830847, -0.0586460607048781, 0.005924554619080171, -0.0663592786752252, 0.0037334205785505828, -0.008485607548863596, -0.16371564135187494, 0.03440437496347575, 0.0763929880931404, 0.00972676082346361, 0.0640184735978714, 0.027369571461947878, 0.002426390528066206, 0.03282394318539544, 0.03466085128298162, -0.1163356364829695, -0.027622821385265492, -0.04083861507209537, -0.14372102501039158, -0.09144086113005105, -0.008892504678791175, -0.024979268951373996, -0.0004362272856239175, 0.036921031960754094, 0.10812501717351818, -0.002585230460039882, 0.022209805710305067, -0.038852958094603644, -0.04305820792570529, -0.013687799751512223, -0.045042386390732166, 0.010759783108317909, -0.019600843267406087, -0.06557488036128004, 0.0025714910668887796, 0.03498821351761551, 0.009828712229992473, 0.05647627445641182, -0.024979268951373996, 0.008769590592789955, 0.031328109223663844, -0.1370503001179541, -0.0463497375448653, 0.017116967431878582, 0.04234076154510384, 0.016414843788884012, -0.05617537970963465, -0.09694871692054056, -0.11244308858095789, -0.0008349396917567677, 0.0371294847711313, -0.09005247425288973, 0.03771753214629484, -0.07499737219862831, -0.0003748288484440123, -0.023592073525241914, 0.03947420668832647, -0.033423776583877594, -0.03412097117064778, -0.09507992550825048, 0.02784526076673136, 0.03332970610605081, 0.03291789017253876, -0.01721403001541283, -0.017928612632988325, 0.012130710638742905, -0.05028480070893761, 0.017562337073058615, 0.009013220539227056, -0.0770397806722537, -0.020524361751972937, -0.11966577979384549, -0.0023850747972572845, -0.06247746478392003, 0.010503928472175463, -0.03726984549949443, 0.003814261201239576, -0.06806266406476447, 0.029419344927061993, 0.05709599378447115, 0.02498804816041554, -0.055386965203491775, 0.029845493204037753, -0.07891495905657125, -0.06401937042155384, 0.03213184469029533, 0.06506430176962279, 0.013393335113127692, -0.06132620121476026, -0.029254867875185785, -0.19953657857083323, -0.031294027139407425, 0.03438249378447815, -0.04914788635813593, 0.028485221163101038, -0.16652227968784694, 0.00010800021674105754, -0.03340915858143766, 0.033145113389676455, -0.007093700927155849, -0.02401417786030808, 0.020099553959620882, -0.13452267413200183, 0.03549719967070978, 0.03967851100256492, -0.00721174598181106, -0.13019341869856613, 0.15215447057667647, 0.027616475056668404, 0.035124028618565666, -0.09017546056445265, -0.055847991331214715, -0.026730023618794736, -0.017570227702631204, -0.02403113283406898, -0.017594875587825867, 0.037472419855963106, 0.033224389809177135, -0.040709324778508915, -0.09988485187150253, -0.017547551648408696, 0.013828762502129518, 0.02897933453722198, 0.008383491272837894, -0.08345816188540622, -0.08809031152088992, 0.039970854737505904, -0.027306615455749517, 0.033864509522842784, 0.0004074686848063843, 0.00018446909292538788, -0.18489656756882433, 0.023872419915598617, -0.13159792668640394, -0.08867099785320635, 0.033186923198448966, 0.02374213202142806, 0.016739680400423902, 0.058465429202060375, 0.0034121485554625365, 0.03679654941042794, -0.0018398652573340625, -0.0028007719841175757, -0.0710910924878939, -0.07682927713493734, -0.19175081020959367, 0.018763500952348928, -0.002734447791356221, -0.1042336815048513, -0.009223188914425228, 0.021972948051703186, 0.042881493265666946, 0.012895509731762312, 0.013729236208575465, 0.003542045140917243, -0.032353862015413556, 0.043524293556970656, -0.09995281984340484, -0.03551006562570334, 0.03786853749918029, 0.03901658185053856, 0.03661487754524752, 0.03543111262038591, 0.03405980166057225, -0.034421876708484754, 0.017859317827993947, 0.03568598596719077, -0.010960710656377317, -0.14202830934102043, 0.0236950111730261, -0.09659397545163273, -0.07094633304908315, 0.015238749705491293, -0.10378373446976781, 0.03225069064688258, -0.06915638558763529, -0.11886574341372486, 0.012558350516327858, 0.03285827333121971, 0.04472707588888705, 0.02000067560433477, -0.14461552018923507, -0.01242001886403278, 0.03200570871439148, -0.1037269308454205, 0.03823677256445008, -0.0235910386521631, 0.02608586845819591, -0.020768853267853167, 0.06234703988340265, -0.008126632947913433, 0.023695076325522032, -0.01545403044483796, 0.03690845896533874, 0.04242925785735833, 0.0005603159652677378, 0.005159153885115916, 0.045908303255619114, -0.07275698382737512, -0.09031774574561614, 0.08468477427962012, 0.02369315284392566, -0.06914391741194226, 0.026048568237711393, -0.006628388007864335, 0.03314925039906536, 0.024206354772735802, 0.05841989177395227, 0.011103117241408975, -0.06510009395953727, -0.006770295700166382, 0.01647848031470523, 0.03322732307219786, 0.028906610457667143, 0.036530235384982956, 0.012629861416631999, 0.029765568114683963, 0.034081673833904765, 0.02171805718743039, -0.019117179740775453, 0.00908411319885964, -0.10523932630206344, 0.0322003924544565, -0.1659280714675891, -0.01157945226954467, 0.042707302866865506, 0.09156989005997207, -0.013027660335812577, -0.043124067273376906, -0.006556855827629904, 0.06439924763411008, -0.08094446815458038, -0.16388046449138513, -0.03054867726432812, 0.019693407651824583, 0.002134359737611461, 0.0622236949745824, 0.09795515057036466, -0.02123442151318105, 0.03500018028367718, -0.03857750927248092, -0.03654681613417934, -0.10420885475773198, 0.002547768354901008, 0.0561682594311996, -0.029625143365893888, -0.041663900067460714, 0.03073505820222575, 0.024138758631101077, 0.031045189553166503, -0.08796218247544063, -0.08764347420209163, -0.009342329195666226, -0.05608606830149318, -0.025508571798829722, 0.030851866021240498, 0.018370773403510638, 0.04546053447725662, 0.026110253003383654, 0.027358216842819527, 0.043963118234919774, -0.05322193693485743, -0.007983101476525447, 0.027399260043100136, 0.00432542516599797, -0.0067375487673803506, -0.03674418782108952, -0.04050120994284819, 0.06220258677042791, -0.008134641872632792, 0.0023618907380075402, -0.08015122236945693, 0.052561153615971246, -0.0825494963118366, -0.0545882943353694, -0.01728579380367004, 0.04334785114710743, 0.05327549003441064, -0.058942309496559826, 0.00860958650242872, -0.07443880114906623, -0.031748140318319615, 0.03298478166455999, 0.05040324518750422, 0.024756168831725177, 0.046372215253816365, 0.05008050091347838, 0.003632791024079416, 0.0012709329521533555, -0.0021204066611617113, 0.016242078336731882, -0.0181824999997824, -0.030961682506658338, 0.07466754889935072, 0.021069542922644492, 0.005026995081834031, 0.028540136305693117, 0.015363442109172346, 0.03627834002096017, 0.004888598929026556, -0.007740904625952128, -0.020643128248585604, 0.04252039016751202, 0.021281712475729124, 0.027059276545095358, -0.021223408197110622, -0.01647179032224842, 0.04076407717647689, 0.02346234403914534, -0.04828263101054743, 0.028128884050751268, -0.007916013785553525, -0.003719255916210053, -0.015144140643040266, -0.05951775862225949, -0.04367631123130299, 0.08302026155827148, -0.0076129589360563075, -0.04875210733966546, -0.017985364261288633, 0.014389881934295117, 0.03187521504224927, 0.019866133788706764, 0.007498253538218627, 0.009954750769654758, 0.005246189936277428, -0.0047395583589603645, 0.06659391641316514, 0.05628910709578948, -0.11010864778995094, -0.0096139440247861, 0.08461868516445024, 0.029448206351448382, -0.0074285678440721955, -0.041413942657017984, 0.03881251485170444, -0.05353587263335218, 0.014658944039652374, -0.004470969180776416, 0.03881251485170444, 0.026120001840697724, -0.015204306777417189, -0.016507916099094796, 0.029805313523122313, -0.1065863272232113, 0.01441505373991657, -0.020892819999108126, -0.024463548161114342, 0.020817541088176423, -0.0035001986892784624, -0.13540186498561801, 0.0047677133815967054, -0.04680098224634797, -0.11672646375349023, 0.05173065078661172, 0.0030593351364941136, -0.061167729422237006, 0.00678727279993166, -0.014539254203334795, 0.015923351857072988, -0.09346803627961409, 0.04059583307993064, -0.07718502186741089, 0.08516225066263002, -0.08328327450151615, -0.001158348524542727, -0.02833853958397904, 0.1369208873035577, -0.0555406183732259, 0.03947656395739011, 0.0365990782476425, 0.03948080705278996, -0.04496188411014935, -0.05254456409308958, 0.022149101883268265, -0.03204892152272463, -0.05347595124916879, 0.0425569995735274, 0.024905477916721364, -0.04455722880304948, -0.015176680339178846, 0.02614167026384495, -0.011795561073262897, -0.017374241582903505, 0.02282827899288766, 0.01418809323980932, -0.0685556476982118, 0.057707914518509804, 0.03578213420739907, 0.013359157278231667, -0.005840414889659092, -0.0722303007781312, -0.0793552698534253, -0.03419621358481851, -0.07011459677538985, 0.0016054695988704883, -0.00879678560170639, 0.0019492808926547477, -0.059873466588988414, 0.04228655628283491, -0.10038626246881835, -0.040694458230193395, -0.006843099737799093, -0.1275298620167284, 0.03733034172578586, -0.0006643652988845768, 0.0031448596865835386, -0.04299879437570668, 0.020744423610779706, -0.0725435315564923, 0.006116378161608407, -0.032771680006345213, 0.03305245942726851, -0.06705587320968086, 0.08469699507892252, -0.09277229452683862, 0.019401430512088914, 0.03767716289102875, 0.025908355006714612, 0.005077646298970699, 0.040882233652988764, -0.11211926376399292, 0.0350753678084592, 0.05140229733062117, 0.0005356111074688652, -0.0641665150818051, -0.03455022038079516, 0.035271505645140956, 0.01680052666914898, 0.016116788184409977, -0.023434143323147864, -0.026157382485369366, 0.02589653166510197, -0.02662491194955105, 0.11969788357156984, -0.01419890244836437, 0.012219973523434948, -0.024824031922095254, -0.02317816140109172, 0.03802855083424715, 0.007920649918670216, -0.015253826512660008, -0.009237763446928313, 0.021332675542419053, -0.015757689048329155, 0.061688796604196204, -0.07369375900863889, -0.014858343108302157, 0.035786113871595956, 0.029019835264622755, -0.03794492767156051, 0.004983532622959111, 0.0428861094195463], \"yaxis\": \"y\"}],\n",
              "                        {\"legend\": {\"tracegroupgap\": 0}, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"\\u5546\\u54c1\\u8a55\\u8ad6\\u8a13\\u7df4\\u96c6PCA\"}, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"x\"}}, \"yaxis\": {\"anchor\": \"x\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"y\"}}},\n",
              "                        {\"responsive\": true}\n",
              "                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('4199c138-a69f-4c05-81e8-45d42eb4af38');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })\n",
              "                };\n",
              "                \n",
              "            </script>\n",
              "        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "hero.scatterplot(train, col='pca_from_tfidf', color='cat', title=\"商品評論訓練集PCA\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 542
        },
        "id": "Lzf2L_ZUpnYh",
        "outputId": "571554f9-33da-43f8-8b40-6e0bd22b92c9"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<html>\n",
              "<head><meta charset=\"utf-8\" /></head>\n",
              "<body>\n",
              "    <div>\n",
              "            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax) {MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>\n",
              "                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
              "        <script src=\"https://cdn.plot.ly/plotly-latest.min.js\"></script>    \n",
              "            <div id=\"e9c5ad4e-e3a2-41a4-8074-fabcf5b7c6ec\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>\n",
              "            <script type=\"text/javascript\">\n",
              "                \n",
              "                    window.PLOTLYENV=window.PLOTLYENV || {};\n",
              "                    \n",
              "                if (document.getElementById(\"e9c5ad4e-e3a2-41a4-8074-fabcf5b7c6ec\")) {\n",
              "                    Plotly.newPlot(\n",
              "                        'e9c5ad4e-e3a2-41a4-8074-fabcf5b7c6ec',\n",
              "                        [{\"hoverlabel\": {\"namelength\": 0}, \"hovertemplate\": \"x=%{x}<br>y=%{y}<br>label=%{marker.color}\", \"legendgroup\": \"\", \"marker\": {\"color\": [1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1], \"coloraxis\": \"coloraxis\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"\", \"showlegend\": false, \"type\": \"scattergl\", \"x\": [0.006716820914296273, 0.02415329977205472, 0.1342925286456653, -0.02184885056913037, -0.03208302045410213, 0.006410635278561735, 0.1559945097386163, -0.02470403113615121, -0.030591634941433396, -0.030878657941604104, -0.04460341375074778, -0.06509540656813427, 0.10203877550843059, 0.3163620485731804, -0.10677944213637985, -0.04177714126880423, 0.003888065427254825, 0.10330892265036382, -0.026676325933590067, -0.013925541456215279, -0.09079988970998995, 0.3595457622147959, -0.030876236188153484, -0.08713020602471068, 0.01876531101103561, -0.07510853749125106, -0.008922272115162916, -0.052634260484167, -0.020261489333105866, -0.0931883283901381, -0.10260413489359003, -0.12971551125226266, -0.10746837614609514, -0.044411857059642436, -0.05316669857740879, 0.0033212506365632877, -0.04847514201118354, -0.08893277430076016, 0.021263492672449112, -0.12402050394808499, -0.07108270367453373, 0.0698641115908429, 0.06135848257700234, -0.06495361272224322, 0.05397220421214726, -0.03485281867220487, -0.03355128241722638, -0.07836288954880553, -0.09628215100944543, -0.061000736765372494, -0.013743119689441866, 0.10457414525745551, -0.026518024642705647, -0.10139957383441364, 0.20178802012819624, -0.058460303816676024, -0.11115584711058563, -0.0687165051209451, -0.04306919467308626, 0.083745393844837, -0.05237959568291481, 0.016065475522662808, 0.14956349586039583, -0.19304831304769848, -0.020234569435764908, -0.07092222933988451, -0.02783155860697278, -0.03759143251458817, 0.08510169639537879, 0.021292351314530063, -0.05400459700237516, -0.021062193716613362, -0.0401221641513256, -0.09835299152264719, -0.08007847559858738, -0.052908154561807655, -0.08624225859299957, -0.037894175455055044, -0.03196241137136163, -0.05773925590178017, -0.06611902950127899, 0.02226955086257546, 0.27730646706104095, -0.031436499614932834, 0.25621821190211785, -0.02556641399805244, 0.0017313929175520018, -0.045922461308371416, 0.16624132515820905, -0.028784672509271466, 0.0265304125379801, -0.0633307871267819, 0.04941491990354203, -0.02528395449558082, -0.0383281764349479, 0.016398128982448978, -0.12855905380179417, 0.07805257440049138, -0.03800522571004048, -0.03173990270635906, -0.03175424175347286, -0.06191350512621389, 0.004673924880529483, -0.06147059311760969, 0.04008795274214284, -0.14157987246830242, 0.03381253563152528, -0.018032728077407266, -0.05623889174637782, -0.04696430782046476, 0.11019199072484867, 0.0394806118432948, -0.007782898284959778, -0.03918954380973378, -0.058906364266006116, -0.07902459940817816, -0.10727076036661967, 0.2656633261950162, -0.08537944755752302, 0.11521300896713195, 0.06873966771264585, -0.04370752417127692, -0.01921888216646099, 0.09714836032166105, 0.011561911907421153, -0.0603327388838532, -0.08801140130518967, -0.05239285126494742, 0.09384568828092395, -0.09130694692652423, 0.02039303214963696, -0.051156019089722574, -0.061952869483886126, -0.05638930371500715, 0.025216023685473658, 0.04043919419204739, -0.1493857671234115, -0.010044094441455912, 0.3673519689855948, -0.07952376258306595, -0.048553161837975915, -0.10170539217503638, -0.08500808166017919, 0.08628878501170696, -0.05300970695168949, -0.052006710082322966, -0.017252648044387794, 0.14543805928900724, -0.011484290822299225, -0.041867308914742704, -0.014726293929892666, -0.032295781707453786, -0.06964656995446282, -0.022144944286730115, -0.1227911313258988, -0.05247658131131188, -0.06237092151921958, -0.06864028992828808, 0.16936584452461528, -0.05239016493225636, -0.021157966816563074, 0.0381365966162643, -0.06261286478042469, -0.014248154484171369, -0.044616034675703777, -0.11415331151797259, -0.01862313922231587, -0.032547977894484714, 0.05046699737849336, -0.06394120750447792, 0.08062205694092792, 0.010829137260342351, 0.03225426274619505, -0.04387100024417775, 0.4445364111954195, -0.06803343224207176, -0.017409183637643256, -0.024717834442748228, -0.07316295262084895, 0.02449054234753958, -0.06642494066464147, -0.05069239342506558, -0.0396355083287726, 0.02688709340300124, -0.05151733986216963, -0.08499286223001706, 0.06677980451547656, 0.0359698759525066, 0.0018182618385679567, -0.10318015940825659, -0.04246935873029277, -0.06422510687513723, -0.07400030646129625, 0.03808298584715919, -0.10613139258039664, -0.04196612247435676, 0.3371284641425306, 0.3959629076332912, 0.03667218221942502, -0.090294682610452, -0.024691925295938114, 0.03596922394148144, -0.12000233823599737, -0.03035091372229078, -0.02103621509148729, -0.06599412149054194, 0.2798645571355205, 0.00046393763197546484, -0.09555916750779828, -0.1180520411781864, 0.20923976871450845, 0.04973402486063485, -0.021465117301315135, 0.0030171034778612893, -0.09159760264042494, -0.07381354878920161, -0.056093200083805826, -0.06383796986182916, -0.03956057955787322, -0.0413749669350436, -0.08316700475305777, -0.01722802503924632, 0.1078905026705044, -0.024875947655974427, -0.010142967050503535, 0.12354416456241667, 0.010379677643497178, -0.04325261160348077, 0.17266466614858408, -0.020747838953458022, 0.05330354685327187, -0.14185720726104645, -0.12582195352124345, -0.05017259603461722, 0.43133450633293197, 0.23769582188065475, 0.1448354190695758, 0.07101154714089704, 0.0392671370886468, -0.1108399809981128, -0.015239095531174045, -0.02362221277017114, -0.03440454507926396, -0.0400411315372587, 0.4225289880365999, -0.07719452735194399, 0.38605252721040145, -0.04728965230209502, -0.08380796229117034, -0.046499037231533404, 0.06952111362420133, -0.044029460687258326, 0.3757692809112856, 0.02771323113176153, -0.06440677143361515, -0.09406748440092497, 0.4112388009395425, -0.03529569866528735, -0.0013725661003196705, 0.04036264400655281, -0.016350871747413447, -0.03751054407793666, -0.08072148171106215, -0.042901801587027985, -0.13333709499104054, -0.04477041848613227, -0.021144721258360435, -0.05816870355106515, -0.030856698892314207, -0.04625520650414617, -0.0625204599829412, -0.07213191133803973, -0.06537175118339401, -0.09521894998378108, -0.11235273207809955, -0.09262443228040738, 0.024443290129385904, -0.019516458303502134, -0.06676982348896092, 0.02012957988891586, 0.07373724876857608, 0.033880280391248664, 0.23213691357234817, -0.05925825762398807, -0.03603368808673682, -0.014427405723369905, 0.0740233509193938, -0.07685147499220786, 0.1806167581429813, -0.0510394514510572, -0.06306332372413082, 0.024746845446267242, 0.013956944368525046, 0.2421366483068628, -0.08198119712504161, -0.037542145597541134, -0.06094917220071506, -0.18632027999038794, 0.07137988205597182, -0.0552794114433355, -0.07422451431990111, 0.09067037283613008, 0.2123525876247113, 0.0023388772995703738, -0.04869071202192176, 0.08247583084839348, 0.019265878414504025, 0.026887468663535898, 0.2236881691100186, -0.033096745192185424, -0.057532484412711485, -0.06021462841182305, -0.013945792442265492, -0.00701056052382324, 0.1445107198213678, -0.08387383375968911, 0.074411999496784, 0.04962785189409256, -0.047398139263469384, 0.0160295415999158, -0.10722804302167276, -0.05323706805659475, -0.03444333216741567, 0.003922135730153656, -0.05854020515311821, 0.023402864259550522, 0.07924281987301644, -0.06404036735862305, -0.07415032209839097, -0.014263738069547298, -0.031574986768346615, 0.0067445963525014615, -0.01740743497590688, 0.007265308298311783, -0.0371991987320196, -0.10989667483693447, -0.04128110705146857, -0.03390766156835956, -0.05527755235779083, -0.08269103165506383, -0.020468252090220337, 0.11142466100738119, -0.10167262278544391, -0.04853061586788163, 8.231585614611378e-05, 0.03342586587829781, -0.055460822813773396, 0.42704669358466035, -0.03667436441150404, -0.09515813273932698, -0.11511844844019514, -0.029857968576297588, 0.08357270851469087, 0.12164244211506228, -0.05654490379805895, 0.3218368714963487, -0.08401872319397877, -0.015920546897994065, -0.03066437474727548, -0.07757402764759216, 0.011928741811211861, -0.1600719509235999, -0.10330771385703606, -0.03785157040407431, -0.09662638290407016, -0.018226281125839832, 0.036039607977273146, -0.05428927089985406, 0.32426135282364493, 0.1934323230828299, 0.2753901331994818, 0.0010651786982147549, -0.02257584135422615, 0.054141142838947505, 0.025198653606106937, -0.040217604993017374, 0.09517352721711089, 0.15330473926412858, -0.04444519349302274, -0.06904254728282977, 0.0001812690279119085, 0.04814365726264047, -0.09558661812772651, -0.025185440119591243, 0.07013433695712831, -0.040547246009394924, 0.02578817332687787, -0.06606386209736612, -0.03826107514627491, 0.09463280440904577, -0.04185925212707982, -0.07822492904274168, -0.01299596189975359, -0.029221060324551296, -0.006029447439236462, -0.07253588000058102, 0.45561624460504385, -0.005960006407397776, -0.030754294101801874, -0.02481359431432341, 0.04413788702411584, -0.06177274912582219, -0.09620888991765059, -0.02839827456109694, 0.020616609401440125, 0.08354201638855692, 0.04225860268224498, -0.11345465803838162, -0.09981961996392232, -0.03599358920802966, -0.04101777077083954, -0.04026061611180247, -0.11277908631270922, -0.03467541013438168, -0.02619637853719201, -0.17239807245140376, -0.06081857355395206, -0.012852062398116515, 0.12124359305647549, 0.3136368658491654, -0.07530705871200802, 0.17509719150394104, -0.02142123183884055, -0.016861963431324487, -0.034084032937617, 0.11488500622152598, -0.020434886532458202, 0.06712832494934837, -0.0835115002466227, -0.08708435896738978, -0.09250943667022742, -0.10515612605060011, -0.011044655755136959, -0.039430401820919336, 0.048834216118313255, -0.014496802830199019, 0.01776984902713201, 0.10917985930013561, 0.05654781170799674, -0.04549869206504621, 0.034546971929295206, 0.2604867194293138, -0.07437468640169605, -0.059470079801179694, -0.10054630820042146, -0.05005055993843267, -0.1087648246138575, -0.03263949203191567, -0.0547244263723647, 0.19639743620896605, -0.0042664904963347315, -0.03541751986123961, 0.03961564769342998, -0.031965519977624555, -0.07740829613979332, -0.005991499230453761, 0.1329686618164126, -0.03765875266007768, 0.006123242322207538, 0.32468854694687466, -0.0408920941467643, -0.046349793932091425, -0.051928922859489905, -0.026020285138811175, 0.06438128298838464, -0.07527906726654345, -0.115016677393589, -0.02253597201833301, 0.12121769440897986, 0.04420764227641567, -0.05420454563808286, -0.029851981507565423, -0.04192640842293806, -0.13679932735976116, 0.5141706881800346, -0.035162495033693175, -0.07314016412158014, -0.11798362794745676, 0.015328311529056638, -0.006635339645786181, -0.07259956324619402, 0.21289409585878494, -0.0036821256108019776, 0.03230884847324041, 0.05157313490404196, -0.028658202627515058, -0.037107033539463395, -0.0032134286874236076, 0.24930415573131068, -0.018325455031065294, -0.08844076276372452, 0.1520398720971235, -0.055689438360508596, 0.051365649160344994, -0.05184930297953428, -0.08037406503975221, -0.08162631149824034, -0.07677721940186606, -0.020922794449023754, 0.028823853849282002, -0.07599081599642153, -0.06147705637910851, -0.04241677529825623, 0.17227466530205213, -0.08130959854616808, -0.12227683835172203, -0.011076552925842973, 0.17497557568637376, 0.1341271889179857, -0.015155078540079948, -0.03953685557402773, -0.06349237210672963, 0.04372831845469649, -0.01877485563855039, -0.07603376277921353, 0.00040100679094558575, 0.0794455860082602, -0.07234396821994284, -0.07520762724997157, 0.11104136853617604, -0.010023765950142788, 0.05370408741338455, -0.08937473761625306, -0.10470845755965953, 0.047072824517297336, -0.05871733649283548, -0.005840775008752981, -0.018684212462585777, -0.05289961940495202, 0.009494891149864603, 0.01736425888846811, 0.17141347929348494, 0.12640250707642312, -0.015632704527651642, -0.01134366553594515, -0.08597459306995885, -0.04162715597905164, 0.11979059497607421, -0.027533835629911088, -0.06429654153382912, -0.07149114661696576, 0.0345566737316679, -0.03528585491783801, -0.04889612851654817, -0.045535885824503686, -0.06380274454846584, 0.004443675139925544, -0.09888421121922286, -0.11112949397896606, -0.038263448361881086, -0.09541232688687937, -0.03431314748581591, 0.009244523071051473, 0.08438730343089244, -0.03779750123016249, -0.1482168707852017, -0.014860401055775038, -0.02323669843007064, -0.031384418780240515, -0.01661388234356965, 0.0019776724915377243, 0.20984699637886473, -0.07319754044215547, 0.019153686595193006, -0.019541409868997963, 0.23359792155888073, -0.04560919294300134, -0.0022134428315604884, -0.04793341986837338, 0.5920951637461059, 0.05034748933738855, 0.04184346461308182, -0.022090662963705078, -0.014715938325304065, -0.07350386475342555, -0.05410865059197378, -0.0008457076856985481, 0.07174469738684904, 0.03415164286100021, -0.04166966773963971, 0.2418531714484945, -0.017369799143146097, -0.021826698003298495, 0.2755688096338553, -0.03896724456389239, -0.0061850733188131635, -0.0982806861970633, -0.07725459322707205, 0.12214612514104498, -0.045440429083351865, -0.04441563293135775, 0.08876427813070041, -0.04164611082995992, -0.0394779505433253, -0.03756740495669209, 0.13537677651594018, -0.07292733312395966, -0.00022399101228070525, -0.05591091328438309, -0.046869859534377764, -0.05005762127256459, -0.04510032503567081, 0.018910455870577096, 0.19806709506440875, -0.07006302765697828, -0.12955154488200496, 0.14920200081419738, 0.15399068465316834, -0.011801956402349104, 0.07690615694234647, 0.08189104718570295, -0.07044609249474992, 0.017418688150108406, -0.019281770951331642, -0.012359834420305197, -0.010848922266282484, 0.04332510524769044, -0.01894773834732719, -0.06773204664366372, -0.04271800711499257, -0.07039604095754408, 0.3617311442806921, 0.17835598994531587, 0.11221838826259399, -0.03250389457523184, 0.1507322769409681, 0.03041728716035707, -0.0491124954947403, -0.01054953804898221, -0.00843347687353576, 0.16618846872650866, 0.047064744758237714, -0.09347471092968428, 0.31055214716249663, 0.021220354425182848, -0.05854899230917679, -0.030399947132801503, 0.002459567203106742, 0.4566544516898664, -0.05452995218774826, -0.0771137570473524, -0.07750288268038917, -0.13700112740911166, -0.018441614687558082, -0.05691250296604183, -0.03331434853199593, -0.010942701631908065, 0.038988855721138936, 0.1133049414264158, -0.09269676121439786, -0.08930138583949061, -0.004806190391017812, -0.02805541656083642, 0.013800908717165546, -0.0024826849512074632, 0.029482689333066022, -0.08940415161011324, -0.027697148961887644, -0.002099365528710836, 0.01386878793091073, -0.032376710644872374, -0.043784599052288604, -0.1432245356965889, -0.032169316177433474, 0.18297262260337005, -0.1477449835476396, -0.07564496654411697, -0.028069502334657413, -0.05621574580150369, -0.040834908603083975, 0.06913083583688212, -0.020081930512595922, 0.2458897483409096, -0.0635104177141511, -0.02025649962150629, 0.010182259896954341, -0.04456122741970854, -0.07926648679523089, -0.05724317225059074, -0.03723132089672882, -0.03516880156243941, -0.08955692770979763, -0.12849940821282463, -0.00437722074954283, 0.2673364922050609, 0.05190552814581917, -0.07023572350947568, -0.07245853383272069, -0.03816950233618851, 0.26214184197544305, 0.025469158312844284, -0.09188388811637536, -0.2000312122224593, 0.014100611145719205, -0.03474846370568663, -0.17860444630042924, -0.030119769556111828, -0.08306038803547897, 0.006092673729557411, -0.018409936768621404, 0.13283343495432964, -0.030023579476382057, -0.07795657954852551, 0.016994375975170922, 0.33142373212780757, 0.10439930555998822, -0.006729713904908315, 0.12970379423924994, -0.06729695462917341, 0.053987909081529104, -0.08041130137023887, 0.319024594120912, -0.040848461932606966, -0.04963820085854837, -0.12768081784689145, -0.10250422460631892, -0.018691436827992007, -0.052539350007542604, -0.07404695623312284, -0.03247103792577559, 0.050252559119822585, 0.11261457834130514, -0.024917586627122013, 0.14110024220218806, -0.0440211470895637, -0.02302618491219648, -0.049205391462956376, -0.06354108241512808, -0.02800362216289958, 0.13073305046567818, -0.019980438494122326, -0.008380426851309433, -0.01412832278500294, -0.06830672680838687, -0.07454903727583771, -0.030163636962942823, -0.06793386276611009, -0.01251732128182942, -0.07754484963267927, -0.17039297786677582, -0.11835544216077762, 0.09146251947385359, -0.022661967026531536, -0.029553318575166458, 0.002303067303334994, -0.021889547952841003, 0.11993067120449946, -0.04290418474879848, -0.042467526660369376, -0.047948645378064154, -0.06309926654253917, -0.1116813836501302, -0.029202799985037598, -0.022275794269256778, -0.08081793491286805, 0.04272628643258879, 0.040759880091350786, -0.12242158296768699, -0.02283311809108119, 0.06437246905433669, -0.06559147624650383, 0.001287784263655106, -0.061481323022521614, -0.1239965289349194, 0.20161241573452193, -0.02019656657858206, -0.09260662386930683, -0.06120527427485197, -0.03702956591748296, -0.009494732776333645, -0.08100311615365084, -0.039285767134133, -0.07080890442478932, -0.02088167787991178, -0.04791379486011032, -0.07263263437871965, -0.03368353347733756, -0.1084780926843704, 0.09207831936955521, 0.06230297368194433, 0.05569316100335946, -0.0728317572928601, -0.0952250538937133, -0.00029210863581662696, 0.29806679618390536, 0.09599564659048332, -0.07429448423530899, -0.030159993358400573, -0.048856902948961195, -0.03800461138720325, 0.3551824757295012, -0.03411416214657327, -0.019918890922353407, -0.027968447167574604, 0.07247659401862631, -0.06980686402210848, -0.07216547965044239, 0.07717216991980122, 0.3675009161075074, 0.27217515263177733, -0.007488676892215015, 0.16435376153301576, 0.021623413272395597, 0.46235190335158777, 0.405614753447665, -0.005920453439327114, -0.04732072600590949, -0.08321181645067975, -0.008290478370852563, 0.020811794974388807, 0.07970747590005867, -0.002203060735046089, -0.06293108411783961, 0.06285652527954827, -0.01523553730974154, -0.04762694092743481, -0.12146198638046647, -0.1267736520748137, -0.016021426226062504, -0.03532845274571701, 0.16747273877682764, 0.1394841695855168, -0.062050292632815945, -0.06144179030569472, 0.03615866129295931, 0.10845459521990085, 0.37905710832970085, -0.027611403746793113, 0.01545738379639505, -0.010664387265308554, -0.04433256059623096, -0.06488196038227659, 0.2423132766884193, -0.021124371992374665, 0.020616564490397876, -0.07030439462228896, -0.040182711400272436, 0.018187108653607224, -0.017081258657473484, 0.03221824152626607, -0.03284420645822316, -0.047885909844221364, -0.01769212041904697, -0.061478137796699334, -0.03421126279421402, 0.17513984887484013, -0.02592645433071588, -0.019914749840952005, -0.02154545106600189, 0.19874171037852292, 0.41216854634726985, -0.06160584165235196, -0.03303184224551585, 0.010442537389135519, 0.06810391031496069, -0.1073813513048864, -0.1083077735373772, 0.09446870228206869, -0.028135684597132313, -0.06167986823073716, 0.12986902108322085, -0.03601835439277653, -0.07768001920526825, -0.05629269560932108, -0.12857442244192493, -0.06604949235721702, -0.06253460605548913, -0.06900602918181171, 0.11793327048861614, 0.0925717422663529, 0.025955993041708196, 0.22687628131514373, -0.1461878412694662, -0.038118975561341265, -0.040066541060799626, -0.028123231533720226, -0.15359328341071177, -0.10314897858114092, -0.02196408650908488, -0.024957116189310517, 0.025574207883693884, -0.0948215990940116, -0.05473718778036525, 0.13485391579357922, -0.06597131888779216, 0.006188137793287369, 0.08325973869019551, -0.06972672649232385, 0.12975804669975308, -0.03862560759623046, -0.04840035274169568, -0.035941493286426276, 0.07269314935988425, -0.16903789546406495, -0.04095916972852507, -0.02974769199495462, 0.15409753557249742, -0.13082130684569712, 0.0003068944324737506, 0.18852292407467938, 0.2736968515291267, -0.0937812094224528, -0.004700001638756407, -0.01190279819734563, 0.20736830975506307, -0.018851678393076568, -0.059580255510043846, -0.07082577730594819, 0.29048457223926616, -0.07774953036187247, -0.07967534170189815, -0.0632169394345686, 0.37520578910431357, -0.009596094520863883, 0.0034376557615699663, -0.027152170801685478, -0.01843920974107349, 0.4105931289263059, -0.025577759636051446, -0.07731067266675416, -0.05126248850468242, 0.017914198571807417, -0.031249251133693635, -0.04134068695399718, -0.0008157634042883578, -0.045876217222417796, 0.0007617431216477616, 0.1295711305136607, 0.02568726570212881, -0.04489041239254691, -0.06990388538580944, -0.02340274762853165, -0.01986762138130456, -0.0922083433439702, -0.049343703540505796, -0.013266092008129195, -0.037656774253665835, 0.08956546387300615, -0.06563801012832055, -0.09244892726198548, -0.018709740457406122, -0.04406063945120746, -0.042080644505726274, -0.12238687046544608, -0.04909073469716441, -0.052270445420118834, -0.011804132487195132, -0.07782739771877482, 0.007786820965092408, 0.17345250884014227, 0.013749051558779746, -0.017271843475480594, -0.044915230377272845, -0.04715972892827181, -0.07357472322833206, -0.09981035266309839, 0.005492996345682565, -0.06755516434526292, -0.07698148384845635, 0.10913323849985547, -0.03942810801785913, -0.06629132163732855, -0.029863487760995025, -0.03630345111481162, -0.011562019372941495, -0.04688151576970198, -0.0398632167260853, -0.018094294245361346, 0.1900455348017372, -0.016497318714967162, 0.04779803609610373, -0.03807955334676139, 0.00498878483399512, 0.07454050606395693, 0.030891280954487614, -0.05335981873910533, 0.008044959286870285, -0.031910338231348145, -0.1118770224713136, -0.07305530693944588, -0.0674934026104416, -0.02712768412908098, 0.09125249360640446, -0.021421143600734376, -0.008160799707616037, 0.07300365603678452, -0.059722327302532206, -0.16119869605587933, 0.010367680046867065, 0.10450744420275122, -0.033022826678829754, -0.03621620853936715, -0.04172488935355256, -0.030020431242016685, -0.08317525446375115, -0.07482007038597098, -0.10893374125042171, -0.005894258084254334, -0.05710594739027969, -0.053447432890649786, 0.019346047864187067, -0.07604289775537558, -0.027096357093674178, 0.037594259654811425, -0.049694692891812775, -0.03416388100224647, 0.087602539857255, 0.05185912601608369, 0.03964501664208071, -0.022004749400742356, 0.026997608288593124, -0.006534556225702221, 0.24265259525247423, -0.00506998767493999, -0.001204183563950936, -0.12496713563972686, -0.10384417037513563, -0.04850478052820973, -0.03381796705150721, -0.08619430843172475, -0.07781476114273646, -0.15147437070083808, -0.07397873780564389, -0.010433289115964485, -0.049953814204974176, -0.1474395079882587, -0.019443252598424707, 0.13367908519431737, 0.028907661164633367, -0.043328046710052616, -0.07860968691294666, 0.05615134814360592, -0.04684261161460338, 0.2643473045023255, -0.0338912170972683, -0.03303182769612561, -0.03548873494670194, -0.0632285214562837, -0.010666351787560464, -0.08440635748210279, -0.06523208861884297, -0.034716411326909766, -0.1043577237500677, 0.14703168704332203, -0.026054328808313442, 0.0034376557591527675, 0.03999551775523691, 0.07641022432194355, 0.09471251750737363, -0.12653612075492754, -0.057447962920728575, -0.05316621232002623, 0.10787209549864372, -0.06285266830177227, 0.09789225021102749, -0.0778858932898197, -0.10303458268168099, 0.011800223673144322, -0.04749909342611796, -0.03307004587562513, -0.10233904038147634, -0.01412832278500294, -0.06781381181192939, -0.07649591568254456, -0.1046132614105131, -0.03398718504142588, -0.0596753648628829, -0.04161740645717776, -0.05369536165184451, 0.009814920759329963, 0.020406684286440324, -0.03354594538342447, 0.14062969912494433, -0.04839899026113371, -0.029211223568502143, -0.016005160331377576, -0.09078171355186329, 0.1187530346620489, 0.004863313342798393, 0.0036793038283242222, -0.04610501856744672, -0.0056735676809343254, -0.01549483356757144, 0.06077665785193486, 0.0031543088057370407, -0.1022727573576898, -0.01850308574085689, -0.04955340284310686, 0.032566888139445804, -0.02201103399699596, -0.06489046719469295, -0.09644253973111032, 0.45613868430808785, -0.014781031523741136, -0.04826617880224771, 0.18891313912306512, -0.042795314066230855, -0.002537807849493329, -0.02700337050331596, -0.02434508093802024, -0.040191374863455157, -0.15497597526249376, -0.0658018566212521, -6.461394695823921e-05, -0.003502163981536938, 0.05194804462426019, -0.027933216760702973, 0.28215092703622824, 0.1674713939054074, -0.03773032792393665, -0.011563225851613467, -0.09855849982347711, 0.09265695980173849, -0.08607604167260431, 0.12729738851973088, -0.06023219711169039, 0.0507659259971712, -0.06167161960907697, -0.030965070036102205, 0.025679129207554358, -0.04849352456256114, -0.07594950242907296, 0.055887163366514024, -0.05604669986066584, -0.05065528265257278, -0.028771973939384802, 0.032440361595456144, -0.03561263396576771, 0.15076211364362616, 0.1450026251121437, -0.0679389212769898, 0.09385268860936276, 0.08613763621798616, -0.03061031337422947, 0.45349303732947704, -0.04915598895524259, -0.07522957197154946, -0.03217721960684724, -0.014672809113583134, 0.024200459657851794, 0.06820275849912405, -0.10210841305765553, 0.035223537203217976, -0.009320453477710924, -0.017229277797041746, -0.002356942194444961, -0.08302166324285783, -0.09280530757962635, 0.11916232167622867, -0.05967825689261942, 0.3165301033937419, -0.04226083087513571, -0.007167385278880196, 0.02095755300584248, -0.050093993348544634, -0.036337148809633284, -0.07140445909901887, -0.021461925757691187, -0.013815747868365605, -0.0009685683271931058, 0.04096601065687577, -0.07862066326636756, 0.05185066318406046, -0.1268252962471544, -0.12326019889089218, 0.04496560587080021, -0.02803044233095545, -0.054734416828074346, -0.004321329200740937, -0.0352751062990502, 0.20615289272431542, 0.019265702950352688, 0.08621543398998888, -0.05218865359380022, -0.0425932780204573, -0.015523583836636665, -0.005147744191833487, 0.1501563495016845, -0.06828159533027657, 0.018027976762164506, -0.08740855552154546, 0.08377431509370752, -0.051079710295426836, -0.058406244557721365, 0.12683998062650645, -0.10559852825998516, -0.010665623415839256, 0.05429947488713137, 0.003546784563271205, 0.07866495143072796, 0.0934041353367147, -0.002438701851108554, -0.022110175532604262, -0.003566708735490573, -0.027817628463008903, -0.0902896489991605, -0.03977622890797324, 0.026689287845030225, -0.06589614285960142, 0.28839290960528013, -0.03490537625487037, 0.11293051475495754, -0.03922028840175424, -0.04044699986019843, -0.002433706250528872, -0.07290207605780168, -0.028243555309152527, -0.06998989087003263, 0.046763146815181546, 0.015304967350199065, -0.07449006005478152, -0.020557810766467486, -0.0175766304709631, -0.04517115509159899, -0.025940272377331496, -0.08110367390528575, 0.20285630506676586, -0.029511532675113333, 0.013390138716182705, -0.15846031749591202, -0.006805109352932344, -0.03643588851050936, 0.14186366288257654, -0.08438732803599683, -0.08341374599326075, -0.11912438240494705, -0.042069725437542144, -0.023999595384005777, -0.05026421299680681, -0.004341536027034211, -0.036957560465732286, -0.009184089968042068, -0.03748501058990122, -0.06529474535969358, -0.09286506077635334, -0.040319275488424317, -0.024424552690185033, -0.07240659252560129, -0.027862990846450466, -0.05691870705677827, 0.018040860345373284, -0.08313718114013884, -0.030184423049818197, -0.0031990107639709294, -0.020473354953311203, -0.10260137294160897, 0.09635137678139263, -0.03524849560806595, 0.17179116744802525, -0.05344469330776379, -0.0489895248116451, 0.024825655238839544, -0.02211336329119536, -0.12201795481411454, 0.12330987482822364, 0.008000657354545661, 0.02138884447913593, 0.021145908602913262, -0.10336867426682642, 0.1311407008940536, -0.10477819938620436, 0.08099691847318308, -0.10344408287468111, 0.06567847897597828, 0.06512859938952752, 0.2216795928168841, -0.050648713887753355, -0.05432344790101855, 0.26444861997628727, -0.042484287502487686, -0.08321044200579154, 0.35375169505675275, 0.2497471238894583, 0.31278523319174245, 0.0868012378188407, 0.07688098437683377, -0.04365755620782259, -0.1951423762946137, -0.06299754021748559, -0.010809039068265105, 0.1159448371678088, -0.046863680049174794, -0.04113698255719733, -0.035983927684890116, 0.08720431629622827, 0.24278858651712357, -0.09378425361677625, -0.07090517197052065, -0.06354001602878294, -0.03185820259053489, 0.3646309459262612, -0.06010526546045046, -0.01472078928385781, -0.02898066027536759, -0.10097078698113676, -0.06761976027553243, 0.4167563453985375, -0.10318095367132409, 0.04867407346222067, -0.05027852964505938, -0.07043870856727452, -0.05801203112135309, -0.023181373520779914, -0.026113798706761043, 0.030465324082154313, -0.056771623012782664, -0.023874303486364726, -0.02277898762136689, 0.010976700704608093, -0.043277883657753986, -0.015922407398853165, -0.05390703391838776, -0.014947886572177799, 0.05708580231541283, -0.041773725129252724, 0.06676412027748614, -0.030107200641594944, -0.040770385011217124, -0.025219895759732113, 0.03665067441948141, 0.006449655550171255, 0.01322596880750485, 0.043625511403735245, -0.02603090227453731, -0.07300840367273526, -0.06485341896335305, -0.054760730223885994, -0.06853658283510423, -0.0991526936732165, -0.021682704505095823, 0.3827862728639547, -0.07528108296525914, -0.054579314443250065, 0.13691816844716637, -0.01932444391284486, -0.0524765191033535, -0.008630929374812486, 0.06936842361171906, -0.01653916257178847, 0.24114400524486207, -0.06259533459230292, -0.04472960235801168, -0.004401775221377632, -0.050908103031331586, -0.041426455363722844, -0.10362702960413982, -0.014002592207850693, 0.050594849027988185, -0.009188052979759147, 0.42941077055702637, -0.06118569429790668, -0.027519436229396455, -0.06505005722685055, -0.047467149602385315, -0.07116196852130466, -0.035897606452482286, -0.00406208586086053, -0.04701265665242429, -0.027367722182746957, 0.0088073860962128, -0.030816281810345694, 0.004908020517616997, -0.10635060298638854, -0.04185346747377329, -0.020069293934411446, -0.08693545073401002, -0.08541641822284815, -0.015821264337235394, -0.026037593454447297, -0.05832800646114425, 0.06083730123452551, -0.04797541632230455, 0.0021493751927514813, 0.02004117257198061, -0.02251467806841351, 0.10402702070511474, -0.04403836710298312, 0.055442949435720534, 0.13700624085780472, -0.08381774250426556, 0.4279040778230757, 0.11706613161655818, -0.005919679791951667, -0.02306529629129697, -0.019076922638309474, -0.12813367911520143, -0.008544051684607278, -0.0977879268560033, -0.0726212892572273, 0.12124165185230967, 0.13830461838010386, -0.08017455779971501, 0.06762687889247596, -0.02827420272342144, -0.08831729649909655, -0.11273236930502327, -0.03154643517024589, -0.02912693732339937, -0.006394810862083885, -0.03248906161186591, -0.03704689944870411, 0.03676941555302224, -0.0202543495335063, 0.020997016472902603, 0.004464786850800672, 0.21655323469167317, -0.1043846433732964, 0.005805583634009732, -0.02754080494591771, -0.07367995915532109, -0.026645198588121328, 0.14352218307826553, -0.03511920079866439, -0.0771968225829319, 0.011754296367973033, -0.10634963898558569, 0.09931178382991326, -0.0861807891433023, 0.031116106445649803, 0.03407237797615463, -0.09552743412796406, -0.12798280353630886, 0.015816023122882626, -0.009354706833073729, -0.03004251853966626, -0.016321215922745207, -0.03621184861109452, -0.03039916904894711, -0.06480219166899968, 0.11519588416609591, 0.06927160692527243, 0.5920951637461059, -0.042896870662115824, 0.16208813330015076, -0.08206811918567658, -0.010561365821300652, -0.034264708040824036, -0.024043720582324887, -0.02730781801552461, -0.05899020866613867, 0.02458054696678382, -0.060194983512658555, -0.07817219730249754, -0.05918246819343643, 0.07962204531067996, -0.08906804933178963, -0.07536331901600504, -0.08330224271072874, 0.19516733879463027, 0.10718882117451285, -0.044172917427435905, -0.07348327599467251, -0.0403072885352823, -0.09260802021385986, 0.03091045411935467, 0.08083084640450805, 0.19156324309544273, -0.01322039082436485, -0.057354802870399166, -0.029499172186657395, 0.01736891573620301, -0.01673468010262602, 0.2184983720062914, 0.09260756466120569, 0.03689804051158914, -0.05556185553435324, 0.01880422432527065, -0.07968403882886638, -0.09010905996572115, -0.03287224157065663, 0.009667473060127895, -0.06587000411371173, -0.12991356942915808, -0.1691964300436622, -0.01866777614531914, 0.17801561402826607, -0.042155652174572435, -0.03169564472863954, 0.026296275772645896, -0.030026186374425383, -0.045775843059398955, -0.07354934824629213, 0.012214160397304562, -0.03238227674869349, -0.004678684591489826, -0.01742183031785441, -0.050231287230954255, -0.0435545947290123, 0.1390361671802394, -0.0938366300081756, -0.020603183510118184, 0.0015517405017658507, -0.05386656442520383, 0.5920951637461059, 0.20265289335952533, -0.12031795189730532, -0.036367974615391006, 0.04942339624058399, -0.02398932655062086, -0.06602509872573546, 0.043573628098446365, -0.009354559067371595, -0.0449714612074145, 0.12980365969915858, -0.09123802812702128, 0.05396390519662372, -0.052772051797492094, -0.07766884754877112, 0.0021357767399165337, 0.00694510631059807, -0.012064092011063824, 0.0024150806635372473, -0.036876787092288095, -0.005634737188380364, -0.07301074752173892, 0.07289323894525734, -0.0577830520496329, -0.07603063820120586, -0.04545433189740327, -0.040289658134899285, -0.05184818905041771, -0.02540878226759495, 0.2534004416337626, -0.0034106542834367037, 0.07840141301737663, -0.08138080132920303, -0.012865217832798048, -0.04526139964945801, -0.09075233627571984, -0.08180183174220065, -0.10389760784271644, 0.05316431118550644, 0.10075525545880676, 0.09753643044853703, -0.10316836955454022, 0.001804489580669696, -0.07980193955497072, -0.03271648674717404, 0.009306487808944492, -0.07208908162885241, -0.04160071353130132, -0.0567812697382905, -0.04975466806334017, -0.1556414654570998, -0.01602433882826906, 0.20320239019986278, -0.055403829318394916, 0.16936584452461528, 0.13443759862024032, -0.029494194436220134, -0.13629296834348395, -0.061391322355444256, -0.04544062581662634, 0.05535601129923969, -0.010648981824824067, -0.054957577237947516, -0.06075823465354116, -0.040495777791011364, 0.1745665516719963, -0.15729835554633612, -0.007725420430953847, -0.06406470299746664, -0.042811239461031, -0.11727034686559952, 0.17236769398106572, -0.09362119096213169, -0.08036035474124072, -0.0011283375228261262, -0.017728624892516014, 0.22973018251065164, 0.08760279659624819, -0.03182769947938746, -0.025270481302733513, -0.009192450327912507, -0.016986965801814444, 0.04920271169702811, 0.03031390114122932, -0.012874834139000207, 0.0810387910321683, -0.07982427588383345, -0.03401699901529899, -0.07456992697623767, 0.07522827861954216, -0.08569774126676231, -0.09015971945409794, -0.007875918634162628, -0.03596157013914913, -0.05820528272657089, 0.08531878432659419, 0.05953609499893474, 0.23638960080660812, -0.08075548749787048, -0.08684019078357261, -0.02050949561282509, 0.29817588310159376, -0.01161766198240114, 0.17170839983168962, -0.09653367042474577, -0.011734545411654486, 0.06835768073924724, -0.06396537045270644, 0.03421176323287133, -0.0642154941339569, -0.10117205713657769, -0.0532662218252603, -0.08142093584504634, -0.09762369252519462, -0.07575470899784276, -0.06639213650243606, -0.11682202561729468, 0.15462862247996864, -0.025729816363899045, -0.043870851578572304, -0.0010043074148764488, -0.05317317912487043, -0.0033753700964083404, 0.005897156315580785, -0.03468620987612727, -0.1052773264589829, -0.020078813767163143, -0.050641929858490246, -0.13242596504508952, -0.058347557869624526, -0.014045019125972056, -0.058214166555324576, -0.03098966722519757, 0.03141345887568147, -0.00040937056078636243, -0.04291944019430104, -0.0860398241159455, 0.07851978266570431, -0.06848198070816015, -0.029290859246002936, -0.02134690061231394, -0.05551824248200185, -0.09113085075432058, 0.009456482405176441, -0.04709790034317267, -0.004026933579386579, -0.05795464126775504, 0.04953341868130158, -0.013236751442910694, -0.16734972526165603, -0.0291699814515769, -0.03030985617206465, -0.04666724494851749, -0.04518973256728307, -0.03243970738629277, -0.011565097493261782, 0.09935585810321021, 0.29796142253049407, -0.023128675927545816, 0.01951236511422067, -0.06955396844102413, -0.01813508549995744, -0.016177256731571325, -0.03285770993090527, 0.11455225861878432, -0.058766666042424524, 9.168899259036177e-05, -0.06722649703676459, -0.008019570195333767, -0.05858282252478405, 0.47081041094013737, -0.03901979536259889, 0.04929344700278722, -0.018084635031321898, -0.035494531873660355, -0.08265089192442518, 0.08669209344088813, -0.020960237955076743, -0.09024021142844878, -0.07497683070543411, -0.13788199329130682, -0.01780789508696337, 0.358809799737825, -0.009779226441051224, 0.014184816725618611, 0.000969649280785319, 0.06440241346482943, 0.35813491819587173, -0.002619757470826245, 0.07249697547336943, -0.019522978897612023, -0.023030595242045062, -0.03129892267153457, -0.06595108987022259, 0.01335773864449582, -0.030363791329381377, -0.018947274681977648, 0.13890146999445138, -0.013976593452860566, -0.0950060803729884, 0.06990248963117814, 0.015465540519404445, 0.001195173103845552, 0.23088778053157885, -0.04607948798555661, -0.031759243052443924, -0.1424293040414454, -0.01881821801288362, -0.04071687187042988, -0.028765058291244736, 0.021590250920161036, -0.15362478169378058, -0.11356734102296111, 0.03323117769296801, -0.060122537841538364, -0.06832105564872691, -0.012882514647563067, -0.06589309212359515, -0.020036518711278473, -0.03215394993282604, -0.03676652996425828, -0.10024925846994334, 0.013423686392199553, 0.18022785011844306, 0.10312964460100664, -0.08986014295440847, -0.001299415094548192, -0.023178658299180435, -0.06608537997591261, -0.07096162916063464, -0.05033829563854631, 0.19927460189752072, -0.0008809915776903465, 0.12682199415286474, -0.0857748629707249, 0.0693492232512343, 0.023802361981684193, 0.002949349665479442, -0.03233067101959407, 0.03045313674229635, -0.07259877034417193, -0.05495604579395904, -0.09497612583318361, -0.0681356765818434, -0.010643762328039173, -0.027890647522477944, -0.0249218714748775, -0.050177109250569146, -0.09867979023573545, -0.005170785965502586, -0.10731142326024047, -0.02890128874949641, 0.3286783167300612, -0.03332532460710183, 0.33428365692651296, -0.04911337523650919, 0.1252416358587299, -0.07998796221264066, -0.03545588346330759, -0.08030254161512763, -0.09359997004985436, 0.04424990222034194, -0.03953906928194801, -0.05012328662135052, 0.2741716623215376, 0.04887216384474311, -0.017286403502466925, -0.02337839300717166, 0.02702291706009646, -0.1067873404320899, -0.014271544839661416, -0.0692021212682664, 0.11965608887964195, 0.05166427788486225, -0.034055216742838575, -0.0012947791134105746, -0.16780255103035294, -0.028980242383990877, 0.3110916115864789, -0.04969663799625574, 0.0396793784401242, -0.0071876601979189, 0.07255300862053513, -0.046137190109942, -0.026817935654610457, 0.2171961735825326, -0.05046517599947705, 0.1328955929614434, -0.05687992888186097, -0.09829057400786333, -0.030019015890235742, -0.02801568246512902, -0.05241762232361863, -0.04528652213634818, 0.3271789551006585, -0.005278967918215659, -0.006535972481729172, 0.12492356327918995, 0.05522643176060992, -0.009637391994471698, -0.007434961852029161, 0.18675931393439402, -0.15683097202361018, -0.06980489647549051, -0.10243613491946098, -0.03444043952678827, 0.04778499101531108, 0.01738162883706886, 0.04306906101208152, -0.1007528471181667, -0.013180067375311641, -0.06411416528700614, -0.07248289041975801, -0.02502413870623432, -0.11519721143347586, 0.008529902946320977, -0.12050740199097447, -0.00110491372443901, -0.08863231038322918, -0.014741606991280303, 0.04742954124278539, -0.026848680912119414, 0.19907036926701632, -0.07177086647810814, -0.03427244738702581, 0.08648114528405165, 0.015580800055793402, -0.034061228360039976, 0.030600635857934023, -0.03772643678678193, -0.08939319143980419, -0.07112362776007894, -0.05876523384519566, 0.1000992590884356, 0.0381051456856147, 0.09686500913608304, 0.07527495216380024, -0.016986017805605304, -0.08196391549553825, 0.22303906956111041, 0.03795315297404129, -0.015801622471228485, -0.02926205619267163, -0.07293132301115765, 0.05876923903564364, 0.22917559303644672, -0.08958248315464566, -0.032363992073363594, -0.019054795801674098, -0.04310226745332892, -0.08379658430685516, -0.031300656377378194, -0.05332150059087329, 0.058969248774796125, -0.04473170992222686, 0.09846161941046147, 0.06839391174225586, -0.022536938312116853, -0.05797293822806343, 0.1533838261152568, -0.06568035569061344, -0.014231770094194133, -0.02855666779945155, -0.04282083744740798, 0.03267799922681067, -0.06009281612200393, -0.01816974995646757, -0.04535870062135545, -0.04217276390464083, 0.20063695360576994, -0.005326062684880451, -0.03074903313972581, 0.16998757392848873, 0.12496078185955645, -0.05295819658943084, -0.06231322371882318, -0.043628001903894376, 0.1871795399724531, -0.03337493571429365, -0.0321331973110805, 0.1301804231621502, -0.05779298650133669, -0.05186033293719671, -0.07626237799627773, -0.08189366242078694, 0.12732830336730278, -0.023315052244945567, -0.03284560737925786, 0.024720245912233922, -0.016541680113380426, 0.19147067893339637, 0.0499164077101565, 0.07240462415995312, 0.25360809080164104, 0.15425613087033557, -0.04803749820656433, -0.060390722434462314, -0.0748372092629015, 0.029983225581197388, -0.015440962208127006, -0.08781664052757032, -0.022414232889884488, -0.10579493419483349, 0.014500783618618182, 0.023438397197983532, 0.07034691225727516, 0.012467343212654145, -0.05177694302177129, -0.04935522279613288, -0.045093942905752644, -0.0256825361952751, -0.01791885624481342, -0.03537430315547428, -0.07319891668993318, 0.04157755659615456, -0.05093129587568894, 0.13864004945195543, -0.09621273759461016, -0.024083962819835873, -0.0029481232036161435, 0.08741630160666722, -0.01518225674287973, -0.1511114549990322, 0.14207605261694964, -0.06744247805855483, 0.4084177590804986, 0.21974666719843036, -0.0711006684792083, -0.009984842146515023, -0.064428671856129, -0.03636966870223533, -0.12635894496794067, -0.03988204692273854, 0.03760946844848808, -0.05919716309695937, -0.029766382439115346, 0.047318442082226554, -0.03968439629500191, -0.08240342050885438, -0.03586373921844368, 0.0005249738629659096, -0.016195059945033577, 0.09434770628362062, 0.028820652449018002, -0.030744199930489824, -0.08653623423757359, -0.0340939850645764, 0.05754517633936146, -0.05387049789104418, -0.045288068190282804, 0.026925844895082093, -0.01509478812158582, -0.15429578271345296, -0.11743889636698361, 0.1565578498642207, 0.07148447941709833, -0.04423257088967927, -0.028278222656762057, -0.13286611141495278, -0.0018537820098636717, 0.09251201306722455, -0.018828219751442155, 0.24068763964279177, -0.12371834772361762, 0.05696922184772215, 0.13435820092571504, -0.06553961129209043, -0.04135890379612377, -0.0423345360045511, -0.03325090288548212, 0.12148869796038968, -0.03433459960541528, 0.06331359049302543, -0.017472185728388906, 0.10870231721185883, 0.0337917044162835, 0.07020686326437979, 0.14005710278103606, -0.06323210894880169, -0.07170156585053865, -0.012289774672644754, -0.09261367149610474, -0.0007221812464041708, 0.16257629337720345, 0.01858839731437371, 0.0031975722453631878, 0.06364057987827831, -0.06841634288811362, -0.006957513632189027, -0.05137638689857272, -0.062439186028611766, -0.11491495339857082, 0.013114112878990326, -0.032911369072180736, -0.06960943502349552, -0.004299982818701085, 0.11849212521412043, -0.06942198707940342, -0.0306583500334887, -0.0198386707608657, -0.06525596085716831, -0.07154616184946924, -0.08665018794958969, 0.060246574472149865, 0.17319069972727058, 0.04896486157541108, -0.06913318050789001, -0.02895489471975182, -0.08801850716627359, -0.013210475670664525, -0.06217007311810443, -0.048738582395270576, 0.003915430639575895, -0.025694412046211416, -0.06785328002756286, 0.04707130547827376, -0.06142212050960132, 0.06071650754192522, 0.038975864962961196, -0.07974121267675191, -0.036308758620846704, -0.08810761919240273, -0.013079269854393021, -0.06126211558834578, -0.05220741468614171, -0.050222531161088844, -0.10424228110641134, 0.23727569200573945, -0.07842367191380924, 0.2835335639329734, -0.013077472912632462, -0.034260957217010175, -0.04186662163513639, -0.054568654282654394, 0.02450959485407802, -0.08175971003514076, -0.025597989267695017, -0.020388489423243752, -0.07214925979219977, 0.15181649320880056, 0.002493473440216002, -0.02317690811532964, -0.0268564203361237, -0.04707713093394206, 0.021710223336243235, -0.02306567466061357, 0.03735115401540964, -0.053789745627417354, 0.03515616779021852, -0.040829405796660244, -0.051424642880666574, -0.07924645389420905, -0.056339679358975575, 0.015398906899347033, -0.11905815430282747, -0.0975160686534667, -0.03808076335734595, 0.02384847841799957, 0.06750187837151515, 0.5003656803284627, -0.02442509599816157, -0.04791934730343905, -0.03588271700138104, -0.07570223888038258, 0.2789619381862007, -0.09865005673692849, 0.07683123919672229, -0.021460916463345435, -0.015991535684615338, -0.07206363048972707, 0.25708118269851427, -0.08636942039579885, -0.02108594608906573, -0.0021155635762255894, 0.011265504283855554, -0.026827369186467477, -0.02512993567154584, -0.088833897545268, -0.014510047116401355, 0.23335199306183313, -0.10232415023083416, 0.19568498500599532, -0.0642086632202324, -0.07098276689536874, -0.030677908751003048, -0.14927860949544916, -0.08955950203901297, 0.10649968057575361, -0.10581961330781117, -0.023319784809216964, 0.12818328385353497, -0.05572722176127809, -0.03217755418271861, -0.02190669556268454, -0.06668851809763841, -0.0815024794924351, -0.05364388067480157, -0.049838272766376035, 0.005163032993485084, -0.0563026861097374, -0.044061921918359545, -0.08784161476167886, -0.09010588575275828, 0.0660742206372989, -0.05974219364377385, 0.27964024773146334, -0.12383712041807815, -0.06385158841636109, 0.08044516233339093, 0.024529672541223943, -0.012513638620681097, -0.03515090869825159, -0.02478890635439099, 0.045480359207475465, -0.13386031926040723, 0.2530937960382904, -0.014172581066391344, -0.03410669609670913, 0.009240633570189463, -0.13528215619182354, -0.07740206007265119, -0.054946479197140755, -0.02269684747414754, 0.2329430159710308, 0.06069348023414401, -0.03321653680219606, -0.02755236198849565, 0.04148008967060834, -0.08497397715331426, -0.04638741133549772, -0.05012316362422598, -0.11448688689138428, -0.09736375222402055, -0.11803946798625908, -0.1179133900522281, -0.02148324249401722, -0.04753991162438475, -0.1316895760633947, -0.04880849852329912, -0.03651394398602877, 0.1487125996523817, 0.003943024242759874, -0.018344156901287437, 0.17624091196616065, 0.01130503599062234, 0.1447342370991799, 0.21002917772471835, -0.04718458238860775, -0.04928767302117602, -0.05524107440218896, -0.09000079421344692, 0.038967244607232575, -0.019536421672261023, -0.11348308008327558, -0.05509464250679991, -0.04327773483159413, -0.040884469865729356, -0.08509786250512014, -0.054790883338971086, -0.04304095528897512, -0.08342634878503415, 0.21320004215790686, 0.06308778373479827, -0.1665061751088814, 0.005423868502776387, -0.11957102457121219, -0.12937123896476221, 0.07012350851888265, -0.1331330430291034, -0.01835132239695636, 0.20564956915122792, 0.263723936862117, 0.2274718752388787, -0.038304033514133806, -0.05356249625046653, 0.18891029524744943, -0.03878550520008994, 0.055873378003756254, -0.06810314395946288, -0.028666468698732794, 0.06752895557100935, 0.08176488385122083, -0.008120159264852923, 0.00371839870688711, 0.1229508703902658, -0.03724515812087039, 0.10249278111822192, 0.2579356134043411, -0.0350291403288999, -0.028251157559993564, 0.05018132785896111, -0.035890621555349625, -0.0022202463957917444, -0.011683550675945953, -0.09446703705963266, -0.1000928174412921, -0.06012047295792063, -0.06254316520746056, -0.036748727039828455, 0.3656520178813852, -0.08876166051842563, -0.0684382282975863, -0.07460023802396018, 0.0019200602116839765, -0.06304044079590514, -0.05974711886424758, 0.1263621113629622, 0.1086057376072701, 0.07239018284791496, -0.026603691240685158, 0.2477039078852304, 0.005730301831336494, 0.025012157687175563, -0.018854606560217875, -0.048656333331733595, -0.02953066219795389, -0.07293029535965469, 0.016987359653326106, 0.1649200502721103, -0.07770664644874747, -0.061167852291642656, -0.061370978191333254, 0.02501442810582482, 0.15243898750947557, -0.016878265549561462, -0.04802071569705854, -0.031023679635478016, 0.008369176994017615, -0.010603620325349023, 0.04860733648455254, -0.05521314650663429, -0.09133408576247115, -0.0078073351025654564, 0.012116717064711131, -0.011692047491745254, 0.04806979851091655, 0.007447164423003076, 0.08355148105857114, 0.030772581410741563, 0.0710574492158593, -0.01965409544415952, -0.022152304830219866, -0.00643769514069002, 0.10287904180311759, 0.07628210072041788, -0.08182213885046788, -0.06389290474754157, -0.10235674922102621, -0.06981059403388977, 0.16504603570906737, -0.02356178861798859, -0.021078254918731092, -0.07273185716841886, -0.02519769728012751, -0.07403648239881742, -0.04496371111542667, -0.15656854831449793, -0.014727357060260052, 0.15679772423289276, 0.20624285439454387, 0.06704545443451805, -0.07262605588968296, 0.03065474437690901, 0.01408255489000999, -0.049310098262985276, -0.08655266830722254, 0.04655435002105945, 0.057380678717348886, -0.09335216591006067, 0.2782924123647222, -0.04831927735207882, -0.019223303180033464, -0.014104531422390876, -0.0364020625971142, -0.047114617753760295, -0.06952124678443634, 0.36078412260108045, -0.05942052257939232, 0.0019376040846570603, -0.0605869380309026, 0.054544428461651716, 0.08487901213668304, -0.028752952191328535, -0.02815832724944907, -0.07008212359443902, -0.06273134221278487, -0.08379800368771548, 0.3345622519874827, -0.02997635387373708, -0.07975281034535239, -0.013545335769007306, 0.13642896366420973, -0.023588559862395324, -0.04281054376301915, -0.028808988285706997, -0.019596131395818835, -0.020966653589016213, 0.12791113553112554, 0.2646298200109072, 0.05926550107754993, -0.023129373436155173, -0.059899893239229646, -0.012589359111174299, -0.0030147594027469518, 0.028278736661555493, -0.15036731392789904, -0.05527980603750396, -0.0518211247467724, -0.13363517240168446, -0.004593937582795396, -0.022980336537332804, 0.06172436148372946, 0.06427644401853287, -0.03276704989162726, -0.007006781694893177, -0.02180920461421802, -0.03594967986269803, 0.0529391606692854, -0.0884248920044392, 0.04447792597593373, 0.02672534990815651, -0.09603107357285262, 0.43948691953031493, -0.018511210096444436, -0.05907710183170294, -0.05627515409763512, 0.1752349434881238, -0.029974528560483965, -0.03951899376292066, -0.047339590482097675, -0.04229197137812278, -0.024005320985348393, 0.06591683207443821, -0.05321424842187245, -0.019872153793402347, -0.025761905726339022, 0.10935912760153843, 0.06841904535675951, -0.0532923824257885, 0.06616152645246816, 0.03393442409220162, -0.06294168062765246, -2.9869240412867258e-05, -0.03971982333932302, -0.046683830850612645, -0.0597627185371495, -0.002439141762443427, 0.03702903211903412, -0.012513638620681097, -0.05437926140053627, -0.04744072797355364, -0.06333717335927697, -0.03419945532152182, -0.039054824306653246, 0.009963260139209172, -0.03096907763719896, -0.004418050642357223, 0.07298422950258604, -0.052459464700790236, -0.05479592498059548, -0.006575441486027515, -0.13910669398618306, -0.10977270693825343, 0.04049839334016686, -0.07373475937113859, -0.005152951445279463, -0.050547827530051874, -0.08207937122081668, -0.022695464017145544, -0.010336940345180203, -0.1001705325171616, -0.011971110114784257, -0.07370479344685832, -0.06173800010217422, 0.08861739771914545, -0.11936516130979231, -0.06966110363930739, -0.011375332268235509, -0.12650554627965188, -0.113605052246179, -0.033592110230801574, -0.08491514633898457, -0.03527063459551871, -0.011263407046287081, 0.13309346421574916, -0.10047461054442278, 0.031864447432801146, -0.0072642237934443625, -0.03344768190438679, 0.1349608627798564, 0.09637781595696009, 0.09198997927719009, -0.014822308868996624, -0.08754067545304978, 0.24829940746146184, -0.044002642967605275, -0.04587962156924845, -0.1735363526354762, -0.10731521136365045, -0.033193458179835066, -0.10677944213637985, -0.04449683312292586, -0.03334135749979406, 0.08582987510948464, -0.06069512926806608, -0.0408938481722749, -0.09658062667462625, 0.24154145459732157, -0.07840235126583711, 0.15353693927738754, -0.01599502981042942, 0.0767348285788391, 0.1668791967528297, -0.11771362559273837, 0.06976781965188779, -0.057374382351867155, -0.14961254130725235, -0.00496442779873586, -0.10142050574441056, -0.034349968710082564, -0.06900929566581782, 0.18897384949202206, -0.06469490903843159, -0.04285192348989644, -0.0610473706819861, -0.016204267391844735, 0.06578638027784942, -0.02212020534783074, -0.07052405491988183, 0.13621989101779497, -0.03158024420084086, -0.043621885258496675, -0.015817929598468946, 0.2625803798044384, -0.11069290902734803, -0.09267146782758288, -0.0432423236719272, 0.03904636928170072, -0.03461646606647831, -0.09194469476021239, 0.12978849969022743, 0.032909128195041944, -0.11407584732987723, -0.0712509641979014, -0.06309820254991093, -0.028650513654291186, 0.2534355827435477, -0.047906604375693274, -0.07280911294661463, 0.12444506825217423, -0.06596319953903673, -0.052855455478024525, 0.05658251950337377, -0.06038221704036398, -0.08076780519242933, -0.17326740715666225, 0.0390441719068612, -0.025930797276780573, 0.2805195925845096, -0.03542871863049157, -0.09998200245130631, 0.13197051898650164, -0.0014601827360551675, -0.047313817686503355, -0.0967847943518539, -0.023804985496994016, -0.0028837961574960384, 0.038108907592539716, 0.0057050599430636785, 0.04535992124984589, 0.02047256134815523, -0.06668663577901751, 0.07140251065838694, -0.017794721564655218, -0.09224312626709387, -0.10398512998576387, 0.47447217045810325, -0.09595943128697414, -0.03648810935628538, -0.02165838536300174, -0.006808634919951477, -0.09958192325466635, -0.06002072612502338, -0.06458552048923887, 0.21279783054821358, 0.05139374728249991, 0.1572478970614506, -0.03197232892091274, -0.07762368818317159, -0.001566975807222653, -0.04746277973987406, 0.23414084794825035, -0.07641965043136009, -0.04190339237203275, 0.22915723972897828, -0.08091550100223122, -0.07239091035600514, 0.009198268560658137, -0.011974008769060122, -0.026166795503318917, -0.06634938422129093, -0.04816014444089832, 0.0940681520646036, 0.2979261572323534, -0.05706684569665308, -0.06596811647697123, -0.07883049793436103, -0.0019597057554454123, -0.09201262315920244, -0.05889740841381635, -0.04552070663626277, -0.020694595264161002, 0.05024841710546535, -0.056001956424210214, -0.0737357943582764, -0.05488289431828454, -0.10898341644662185, -0.08416046483729953, 0.02392849369254471, -0.05909307099061109, -0.13944745790780394, 0.006796205696277025, 0.008278061250649909, 0.010228790676186414, -0.03167651386286139, 0.14153214051147503, 0.014347906442967406, -0.04283985803344131, -0.053787323520347895, 0.024884302514689736, 0.38731445269873305, -0.06094835942241012, -0.0687574326659591, -0.05153128118996675, 0.17490628448692813, -0.05857358405705903, -0.03807292586967127, 0.15595624029763022, 0.003118487285833916, -0.05777965913319507, -0.030344534697294978, 0.15450130599560138, -0.08843348844276153, -0.03299810563273848, -0.05215525392045059, -0.071688240205779, 0.07801073420559902, 0.3939502692547385, -0.08596772593405683, -0.01983742939533668, 0.16759254765467582, -0.038128781106029554, -0.03996099466548424, -0.025537997770324714, -0.0599767448279107, 0.4340601880441201, -0.07087895248509708, 0.0023552332306071492, -0.09528867665114686, -0.09692780844826467, -0.02876838700909241, 0.08651241154889835, 0.0006192213285246486, -0.021767230873882524, -0.03376925021492424, -0.026094000737873962, -0.04034454368297569, -0.05965744145386048, -0.08177266517660906, -0.11986429640605863, -0.16354968909410694, -0.024184342505460373, -0.014186589201801152, -0.054637545806774684, -0.060713337288551564, -0.05964320366957784, -0.019282215586797033, -0.07158474052541987, -0.02953051520342418, -0.05882814013365672, -0.053785915741076704, 0.058730723485333654, -0.0017457849072572767, 0.019531449553407076, 0.06774741648755073, 0.37603462076000854, -0.06046216531975093, -0.0224993530644505, -0.02323331549995391, -0.08797583688916366, -0.09580248703851937, 0.23216970803365794, -0.0912041490547438, 0.010797375761451464, -0.05068747855854632, -0.05256733970207618, -0.03159869414291584, 0.001075159787626586, 0.05191549026227358, -0.055489363393951566, 0.1577194192710188, -0.04782058189253495, -0.11850880212533554, 0.06646949891614472, -0.038953913021707955, 0.003017261854448211, -0.09875614511363251, -0.06930619545877821, 0.061451223037061846, 0.013469516898991984, -0.09310570751004801, 0.25210136148079654, 0.2034417595852486, -0.014145883438539682, -0.05966742334854839, -0.03919458232275365, -0.033881427307079784, -0.0350393054215711, 0.0631892931263815, -0.030575888239071026, -0.05636533992708218, 0.04458705344584013, 0.049034683988148565, -0.08085986140374057, -0.05030205966344309, 0.24291138074293753, -0.05234775896404357, -0.04355675012314153, 0.06145582552151685, 0.2450430663938755, 0.16991636521501333, -0.057057675909379664, 0.1919417377971496, -0.032386377686285855, -0.07915586771261934, -0.10934985173887238, -0.08190455894581541, -0.04341133156156959, -0.03280594811001356, 0.04243972001675937, -0.057422342196273235, -0.08228543886641765, 0.013103342652639915, 0.032425875496993176, -0.021018106051148417, 0.11504567337102818, 0.04056738759070734, 0.04719410843117041, 0.16433716224118916, 0.12656704600386742, 0.20353493588739843, 0.04378253728431059, -0.04463523197156736, -0.061449675065615425, 0.2249289269081003, 0.006427176881710677, -0.041185136859327155, -0.07310711287937485, -0.016281988798412934, 0.2301453647282284, 0.0049675237910858745, -0.06299963036932055, -0.0550671902572124, -0.06958418867077035, 0.010467912187749152, -0.04781157161177016, -0.08836860796247531, -0.03169564472863954, -0.05395769484837065, 0.20334559567626276, 0.11209749521000399, -0.05149996619382122, 0.040054267165027156, 0.0736754236457745, 0.2003699996856521, -0.04744861270306109, -0.057513564370702146, 0.02662007094933188, -0.048685166099601086, -0.12575766195963003, -0.08370782177785605, -0.0748706255024465, 0.24162554768368638, -0.1094990260381142, 0.33983843498214744, -0.017476823986817793, 0.014993957304078745, -0.08180103395904466, -0.0673183137392374, -0.07729731571620879, 0.16566774545102214, -0.11201471972620528, -0.041349701934900514, -0.06135169823593629, -0.06956384299810046, -0.023222965075413972, -0.1566614352092577, 0.09252396695141878, -0.030814737204386496, -0.016125611561259565, -0.003930850680528591, 0.0009893001514616469, -0.11422034179040533, -0.04323275494953939, -0.027166874171674096, -0.06531625348436948, -0.04942880895537067, -0.02470142119154875, 0.09518001511163983, 0.09389660845061641, -0.02204675208032145, -0.10219882964696173, -0.0026546930432754688, 0.10506827366521614, -0.010252886757151875, 0.23631377091934821, -0.018889237708189364, -0.14740964302949477, 0.27890080115265076, -0.05078741013059994, -0.027924474911485326, -0.031586956247896945, -0.06207950963731414, -0.05619114793817676, -0.04289976979159607, 0.14008820702502728, -0.04679550535657264, 0.22614645028028785, 0.1498371362515962, -0.0693135686430406, -0.03325820818164039, 0.2765484700455877, -0.024498374033406014, -0.0841156656626105, 0.11274179419866866, -0.03310125848831602, 0.22327026201390424, -0.045022257410308485, 0.3254424068552799, 0.1149905506429023, -0.0703568235503839, -0.03653123744336197, -0.08330771345846624, -0.1543826127021251, 0.026512684593354544, -0.028371980716932525, -0.08717360774311367, -0.07234665691662655, 0.022382329441433924, -0.08268930484239449, -0.05998369989836083, -0.10658155008139501, -0.04098460744433922, -0.1116850929780764, -0.06515627320675721, 0.1738728808792207, 0.09141901434361799, -0.010057343589596297, -0.035326749229199365, -0.03782804900046787, -0.06141199324100484, -0.08934377180171107, -0.023710607983525363, -0.022469562838892686, -0.08945091788584938, -0.02173837876103404, -0.03899290866366594, 0.2545618771400271, 0.011980328818302894, 0.33127825666104843, -0.04789336554907352, -0.09521794946928981, 0.014287541796188425, 0.1016776359066114, -0.021302119201224472, -0.09941003582793143, -0.12791847417767027, -0.05076741766938838, 0.4296762969408676, -0.011309877312214228, 0.07950454439115913, -0.03487885776808544, -0.05506005688695036, -0.04615619926822698, 0.11815857677047727, -0.013823016579886946, -0.07973324274187271, -0.04157933628860834, 0.029059148046224358, 0.030155973411167155, -0.011200505936034747, 0.07197279013055949, 0.5920951637461059, -0.014447628958620042, 0.003604497986631157, -0.021574836510430757, -0.0637948105739752, -0.02284323557281356, 0.23924550118348875, 0.16849991287819407, 0.13517405324644957, -0.0843762624792354, -0.055646846576996076, -0.03246091396261476, 0.06930205748999588, -0.0755567977762715, -0.07611641549201043, 0.02721560778071247, -0.04791958198328651, 0.028922242342134755, -0.009264795769442186, -0.056332975662102734, 0.2499818814563213, -0.10989037114846924, -0.02050697068692121, -0.042087393073271966, -0.05146704911554045, 0.14887730376568928, 0.06855202047468752, 0.010550347115307366, -0.004780493765399023, -0.010248286698257902, 0.05350934521232353, -0.03643426310138275, -0.01946348250926406, -0.02801794114745936, 0.10676809828849317, -0.03648897836838233, 0.06760056483937003, -0.11677166616826963, -0.014201898515065014, -0.015866101381355878, -0.1967943910017797, -0.041759562141671196, -0.05069439395586878, 0.08134149773666927, -0.0500516266750359, -0.07909273686851906, 0.2441330807561216, 0.13283003598518292, -0.030644684443688685, -0.055072416330483376, 0.2521184504232699, 0.03174365304269156, 0.009222592823642697, -0.06754355011542762, -0.025904778198139404, 0.0011054155428146585, -0.029078422151074507, -0.10890099025851194, 0.010090885025912478, -0.044752469423042604, -0.08110042875793189, 0.34323086971791233, -0.028787180057293183, -0.10302170843290262, -0.14786280290219123, -0.051910544806141296, -0.16468812366736857, 0.23064708839133002, -0.006100124445492301, 0.07494640549957461, -0.06267605413587027, 0.0395551238232181, -0.07640716336679475, -0.015096723541044904, -0.034637255189403526, -0.06775967310319637, -0.01710227459981375, -0.09956967402064415, -0.0319989458521742, 0.01690999538230466, 0.07938131497172964, 0.028760858446766662, -0.09356805009579137, -0.00787861205006641, -0.07020856138298476, 0.22032294613123796, -0.08344285663344332, 0.009462499371918594, -0.06785659843000688, 0.011271499849962065, 0.1520401970212117, -0.025222137893404387, -0.012513638620681097, 0.018497932011221685, 0.24440984504359914, -0.08560246309557525, 0.08820004061073604, -0.0653444965827346, -0.08497007918067062, 0.04049644883010845, -0.07340474330636187, -0.012513638620681097, -0.045969587995120034, -0.059979985798246926, -0.05340690101949835, 0.07083156747290487, 0.10521936918233658, -0.04049175917730333, 0.06353339494927512, -0.041368171938447745, -0.10217575732350853, -0.018581850677017397, 0.19313415638733333, 0.01640298029685111, -0.007512863232019963, -0.03903862627633895, 0.18513908878105592, -0.03058655973114859, 0.081523926179845, 0.13603071225455657, -0.050476369361827485, -0.112845896166004, 0.31101340855551174, -0.07013733161734408, -0.09237676803865802, -0.019907209174345197, 0.025559388654673738, -0.031863294073162, -0.023732095482619318, 0.17165416755364252, -0.12615143497938772, 0.0002716798838806071, -0.012062806522033972, 0.28146989549905177, -0.021981022815477553, -0.020328598883056832, -0.07556050345797691, 0.02258538712456879, -0.11146332842568875, -0.04526873744778281, -0.045573173225019796, -0.07672510216906966, -0.02880338672806613, 0.3004586681705083, -0.010904645699551149, 0.08400428248700213, -0.07459340261549577, -0.06845356641756711, -0.040997773600099124, 0.018558430664512324, -0.058800757430926325, 0.1784060762180884, -0.030506329963937262, -0.03785899817736198, -0.061999819003860784, 0.02914512644325999, 0.02850865658547641, 0.018882078316087438, -0.10654778547241377, -0.06032824239019539, -0.04576213184876075, -0.045666544644539786, 0.026040455191855178, -0.043934553249828444, 0.45328417612571587, 0.028506845975701647, -0.05233352758518622, -0.05993853938193171, -0.015889370129403176, 0.4972849244599452, 0.04939691113045645, 0.08658061831159417, 0.010931360580179583, -0.04421008037300735, -0.12449046048081898, -0.10476636102326868, 0.12388236272370473, 0.1415719235232587, -0.12439090247037694, -0.06066690705689025, -0.07904154086672253, -0.09856476084311899, 0.15805129794366712, -0.03819782136877887, 0.22691739798349625, -0.010284431387677309, 0.036947283507320454, -0.09059383210908323, -0.009747515293143019, -0.07042431755659329, -0.016188810909035733, -0.09332519111650407, 0.12003924710843339, -0.02787109213604012, -0.10438109974262319, 0.006501583139418173, -0.06232862375297552, -0.046521319265650576, -0.004386766472297704, 0.06919827139242718, -0.10853661470331888, -0.0825960751958661, 0.000807364330920104, -0.030570603400883054, 0.05262592012691085, -0.008526820126941524, -0.0032789824888826984, -0.001105550475440194, -0.046176296428458476, -0.022559459896608083, -0.04557823274429611, -0.06655306879711052, -0.03853250813202597, -0.010607737384843868, 0.23509957069122447, -0.10982727681364933, -0.03734855270729889, -0.0816012913792053, -0.14624690897040138, -0.13522929207976225, -0.0651707638662694, -0.05301755313921448, -0.02469136810871555, -0.032646311045132985, -0.02660896957317215, 0.02232718039608877, -0.05413165066351359, -0.06793243234103505, -0.04665782130407247, 0.07024346070912083, -0.04567856578817038, -0.11174431230637044, -0.0772634648285265, -0.07098434084551053, 0.04481199715554766, -0.04550442509800637, 0.24928263946961413, -0.03505486984606829, -0.08438108442887558, -0.05031755610516636, -0.07071639993192091, -0.0071815504462842015, -0.009662214858718621, 0.2369001086962632, 0.2128128976964086, 0.03313451076466814, -0.06885798051742303, -0.0054986912086328055, -0.06298688274428103, 0.04045683299305308, -0.06294919012496614, -0.05438499336410097, -0.057492575763101204, -0.0490535983490695, -0.03390901750895921, 0.3234087550507868, 0.010868376034665228, -0.17552948168929836, -0.0665505001475061, -0.01902034930025648, -0.03864740548921643, -0.03099076264244524, -0.02927087870442502, -0.1340149829115096, -0.036535567388438085, -0.07314589994652305, -0.05276283648081154, 0.0600886053631801, -0.12933601213108364, 0.16195023538205283, -0.008294080368321795, 0.13120710176028916, 0.024382460291477646, 0.004033312774944505, -0.15941893508234775, 0.07518828756534864, 0.15840771683930346, -0.07778267096308769, -0.036252701360224585, -0.02915019548395862, -0.03434896997095022, -0.00031877420128694655, 0.07333815662640139, 0.0718752759261603, 0.36675190735228597, -0.04155305539048789, -0.05316421478422375, 0.20852338743568322, -0.11702004682235648, -0.04631774870279792, -0.04626473418977322, 0.514534738061888, -0.07046475517969898, -0.03878375699644199, -0.037010443431811625, 0.1266328184007882, -0.04855003766767548, -0.056657921871062875, -0.0713800338700689, 0.029752125612486035, 0.05160947320332042, -0.0948044612671885, -0.1463555964074465, -0.08883785619858713, -0.0899171000509959, 0.13051304965851482, -0.03440472622779246, -0.035066301803293586, -0.04517240695140008, -0.05277205043709137, 0.012763836908739869, -0.004050005201717561, 0.1853091485340316, 0.0500745009690797, 0.06826330990690789, -0.030342956528203346, -0.04529512797480974, -0.017485418171510468, -0.016476193448784007, -0.054324613979306706, 0.09675277554207408, -0.10237084282923858, 0.3769830693478041, 0.011520299381996639, -0.005745949255231788, -0.11498585051696408, 0.16893757062326317, -0.038710637777663495, -0.03771170915972763, -0.12457943327580479, 0.08720878958143445, 0.606532085172806, 0.22630576026969038, -0.03793842855475034, 0.39278270350646394, -0.03681850116923686, 0.35463111019672633, 0.054977459443616844, -0.07388085806279245, -0.009518514689046362, -0.08043323837388916, 0.07400230530600754, -0.03787381965543503, -0.06710181544241717, -0.10346925166139341, -0.04065808554643117, -0.0844355982217134, -0.05463756442785819, -0.026042927192353117, -0.08281817250794372, 0.002577109450147008, 0.00821380248437998, 0.07878063156945468, 0.1866630453924962, 0.1475495271260637, -0.04698316471857688, -0.06461720825622291, -0.015293910722094402, -0.12425187375488664, -0.05164515937195789, 0.1589369439477588, -0.030256703850438445, 0.005628153045932956, -0.0667909902290604, -0.04263048677584027, 0.03084655104538223, -0.01933081966325239, -0.055886516132175046, -0.015826263746202985, -0.013443958041009715, 0.02836052889655556, -0.11522787440518677, -0.12540887308424034, -0.09752274392481125, 0.2028485765093806, -0.06475939906465238, 0.033622179146232146, -0.016988859866503964, -0.041855683001915, -0.01867973396083675, 0.07776045891257108, -0.040719471424965015, 0.15522268722475388, -0.09686078112014046, -0.005574090115609488, 0.02433778595807677, 0.08581593366991302, -0.030108395019737087, -0.03317958280123942, -0.025207431564263232, -0.04286258835155874, -0.0304866401448525, 0.17049267501942458, -0.07901874739246952, 0.02839347280658939, -0.06479798056679707, -0.038903298704325746, -0.03617257561994495, 0.10984463019161508, -0.020115020119821143, -0.08473773587351517, -0.007996970083214818, 0.04587899726442034, -0.10479641381460249, 0.023070778388209704, -0.010077985084329744, 0.0321169639593918, -0.016564864577177024, -0.046722121044711504, -0.06819753174895443, -0.05889778185641523, 0.05524685141226298, -0.08192260281383336, -0.06806247945830322, -0.04427472523024787, -0.10869883034205907, 0.09261811624219526, -0.042620471818274226, 0.2852052017039163, -0.1279818313501487, 0.028340930699863213, -0.021278015990552648, 0.05024266792557882, -0.024925782466171317, 0.02304649532017067, -0.03823470317316581, 0.09079897821375296, 0.019036924189191614, -0.003562946563921247, -0.018456827159518664, -0.12439298740083098, -0.050065052999049946, 0.04471599405714642, 0.42924945322345087, -0.0399481088094126, -0.05278671020172816, -0.09087957266854595, -0.052866891330992, 0.22552575432511357, -0.004866481252974527, -0.04085127111731736, -0.03774515845670896, -0.005750312254706303, 0.03063009940181128, -0.10260202164661053, -0.03660770955751842, -0.038240115377507274, 0.1371706867557533, -0.05555597467453322, -0.10696346162706487, -0.05669818227598378, -0.01793654798169185, -0.07316055581914367, -0.02666389793776743, 0.0654791810252354, -0.06986072886459752, -0.12226758559717547, 0.00850558455810347, -0.03676556522949077, -0.024321529045680593, 0.17068743958944435, -0.04472734246789088, -0.09203698953483858, -0.09536512737928805, -0.06480805324913304, -0.021125441431379854, 0.003406554593308686, -0.07433462664131066, -0.0803631251179914, -0.029342161536255938, -0.05289962875983543, 0.06839876837584533, -0.036089884993268956, -0.07463020022348758, -0.03234526461009543, -0.05117831589978076, -0.024062162024310482, -0.015307979098167553, -0.034282071506280806, 0.31864485894163075, -0.010825062459203758, -0.06353029740159699, -0.010602084591927687, -0.07246937171102627, 0.0173693966305595, -0.030010628146141853, 0.025536033769825477, -0.0017660599035080192, -0.025929001886492057, -0.02204679486183041, -0.01754317899908506, 0.11349727737111566, 0.05809823862022913, -0.021216435642000573, -0.019728375097409857, 0.0016698828415771289, -0.09330895642977985, -0.12998522245720373, 0.01966901737105013, -0.08922637389729692, -0.0053730091816996004, -0.12181514253823657, -0.08988396013033595, -0.03698183508264767, -0.03488419700147193, 0.001027493719781822, 0.10543970125304707, -0.02779976149470589, -0.0420701367929998, -0.06159032725534795, -0.0977879268560033, -0.015129978831396851, -0.012701193271425255, -0.06750822700646945, -0.0332798302494905, 0.01578736850388469, -0.01043535548101905, 0.048374447539643976, 0.1983521989849643, -0.0016773776626785332, -0.00015164302283437176, -0.04257817091397308, -0.04915125586659071, -0.002711789208628973, -0.03241316727623596, -0.118427183905515, 0.01676687404957156, 0.020740170957511184, 0.013957899535882898, -0.04849224968667403, -0.0379635125368252, 0.42622945140098967, -0.04650766120148757, 0.011067928309436802, -0.020346381732679757, -0.11896604331614433, -0.10926995830159193, -0.029478745640405236, -0.027227963808192826, -0.0260193018896233, -0.015185865031670054, 0.0960803043946773, -0.04370276866885491, 0.1950323079245934, 0.0437132862189398, -0.01129407595907808, -0.03968391932543714, 0.11026277272323176, 0.13537677651594018, -0.03279306513453775, -0.11848479387515051, -0.010366385198658276, -0.03789036652339161, 0.027956782579834256, -0.08794414407942253, -0.1108112776734657, -0.04854768076595635, -0.029039111868543004, 0.07880766627598661, 0.08959090345436801, -0.06786224394960047, 0.0937467249352068, -0.07591773146586227, 0.11963663555201265, -0.044877720325487626, -0.0632896194502403, 0.43765414820131965, -0.09116014570865288, -0.05237986440660097, -0.04788057685846517, 0.0017493925571166774, -0.019217222983793504, 0.16656934733947026, -0.035952668494336745, -0.05199528385925919, -0.031960629770520436, -0.06954305417840204, 0.0928275633898176, -0.05059695695952122, -0.1796261393773729, -0.08938338090206108, 0.043338943903999644, 0.13220178984394362, -0.09411608838540063, -0.012849037196812829, -0.024877714555234826, -0.09021405685667679, -0.07924564034621946, -0.08816974082873377, -0.10622166007713382, -0.06782666207239645, 0.020496335716464947, -0.027049800317856865, 0.15884100666065795, 0.12031874579562128, -0.06349837434490431, -0.028893394200068274, -0.16992344637593126, -0.09433162244770436, -0.00742742609596885, 0.18823804656480597, 0.004232491476636779, 0.22628884000553307, 0.06768211374470211, -0.06970426541793524, 0.09394656432957597, -0.05208245757989025, 0.4179401622427124, -0.021620836463033834, -0.03186363757174696, -0.12603344471887476, 0.0026502755659493244, 0.08530107738113248, 0.02432981906702283, 0.11023536277640016, 0.016741536714080966, -0.09933855927022114, -0.04637932379894187, 0.1213438642305583, -0.019656490418251384, 0.0545640408061247, -0.06303600147837334, -0.039919303311544696, 0.03253950160597669, -0.051894109753067436, -0.11107043135825312, 0.30404413742280434, -0.05331861344837273, -0.031019743138550788, -0.08421914525946961, 0.11699875040505674, -0.05633674689655862, 0.01526599924349722, 0.0010722289437762077, -0.03085311537236602, 0.003298801458623643, 0.2873024269050766, -0.05245003055113736, -0.048135165505437415, -0.05572864577306525, 0.18737459957038924, -0.03614110787903443, 0.02807426946545985, -0.023049138830052197, -0.036762912365475194, -0.022705444975240553, 0.11458146431797132, 0.09961828378805605, -0.10086302516363695, -0.04955423475885177, -0.08865099412085077, -0.029266275532151834, -0.02352661517705362, -0.025407046801106933, -0.10695359058424819, -0.021265086504126993, -0.04462640508641994, 0.13758204819591427, -0.04373060005026447, -0.04985285131944276, 0.009370764840449769, -0.01761542180770575, -0.020305975483177197, -0.0672827126133222, 0.056443429814251565, -0.0725913792337562, -0.06573256484176337, -0.07582959951539914, -0.06899806398646133, -0.04003919801015953, -0.061539355369704736, 0.004722601578069483, -0.04262359378490921, 0.018663678813562033, -0.07548752771444123, 0.00762350783710311, -0.02101303181381677, -0.016459614627990827, 0.1677454269700482, -0.03337599739004804, -0.08754225568040254, -0.009591528228096237, -0.08133399803849628, -0.052013375222586075, -0.0552961097699059, -0.11347617244020741, -0.06215297760105407, 0.25573959503524724, -0.07221619427869917, 0.13715410531890998, -0.04058499042302413, -0.03675810294642958, -0.1548072177522591, -0.03167294066157352, 0.3968513209928936, -0.07065901936559207, 0.02055468140321909, 0.05890737248703631, -0.06232958905687084, -0.04810224415502861, 0.019076357235784765, 0.16722440897510701, -0.014936678580301267, -0.00514957226010659, -0.04946092730837351, -0.13742402521571767, -0.08925241272027393, -0.058113686047697635, -0.0934570569480075, 0.03567021612071249, -0.033385881479503686, 0.0010343640774986157, -0.032349283013356794, -0.021481646422848312, 0.09462444035495124], \"xaxis\": \"x\", \"y\": [-0.0560315979574663, -0.05604308636447672, -0.07852365418078935, 0.1497974294001999, 0.11835983689368174, 0.05195956450823347, -0.0061461656752288645, 0.027672108330283635, 0.07312177738214064, 0.10479038193293848, 0.04280254912205563, 0.034205379662848506, -0.017242710271142582, 0.005217478925292565, -0.16388046449138513, -0.018963150198026107, -0.007410750190371259, 0.004279255624742263, 0.03677226360327877, 0.14099719304511085, -0.05962081071167486, 0.003056861056054932, 0.03323264200879534, -0.11681112231093578, 0.09181471914023542, -0.01673345744287065, 0.032393450096533105, -0.05513910944086011, -0.06313788384777269, 0.13598578788135263, -0.1337956918014346, -0.17371171773773852, -0.061217942190902506, -0.010680789086667126, 0.1377238857992983, 0.07310859114072686, 0.04006571613484294, -0.10409258270090008, -0.025178071136929426, -0.15506620255908798, -0.11050887038274901, -0.0682293894733004, 0.1781917688680039, 0.172016695349021, -0.06672994913133237, -0.004128619352515748, 0.020164813510524338, 0.16287615013067258, -0.05074489211668846, -0.011408416263403817, 0.13512339510401356, 0.03415837166920113, -0.02992372806041003, 0.012042248918524812, -0.010863640940172566, 0.01243673697589805, -0.055018636772543025, 0.04954935661578948, 0.0430511235947433, 0.06790206160738846, -0.06993623157609735, -0.1254628276540634, 0.004197140971389559, -0.06690403231922677, 0.039555692027583786, -0.04740213160472646, 0.0077153585492032994, 0.030537235341105497, 0.033678577301307196, -0.029004738565916073, 0.030214553606755446, 0.010649176776348381, 0.13706486695895112, -0.3836170396672197, -0.06078454407542337, -0.014028820819480939, 0.009427539299013405, -0.06642098417974387, 0.0678264439741617, 0.03858463024068967, -0.1801901127570564, 0.08528492731285828, 0.01197733168905103, -0.05375974682826955, 0.007172698447845426, 0.019747737595084844, -0.0039603102317458914, 0.15970649600930545, -0.05383135716445592, 0.15883841639549814, -0.0062661938868849906, 0.12464450389956819, 0.06556641722505757, 0.016736439357339507, 0.025456455550529934, 0.03393207907313913, -0.11559979579791654, -0.18074056827202747, -0.05445029798693473, -0.03532992827286786, 0.0018769516905646482, 0.04579121088897451, 0.17690893369490449, 0.015446636796776854, -0.002703206451806229, -0.16948559771604255, -0.1001861576307076, 0.15479433802119977, 0.018188977839445383, 0.06916475902192161, 0.00171241956687162, -0.08559278840548495, -0.012592768176327696, 0.06485400439041374, -0.07006167249990555, -0.036336661036739125, -0.2915937929322987, -0.15132038125221126, -0.03594441675179799, -0.16047878925136994, -0.02532790129583512, 0.039960104974028236, 0.02229100723935703, 0.031089344996359375, -0.1261453693709886, -0.013625241277529258, 0.2634135882497303, 0.02169601632551241, 0.04695350189466615, -0.11715818881964099, 0.03509820281781654, -0.02651131487658306, -0.03893378268147784, -0.039724341321057746, 0.26252589395771636, -0.016922607835968974, -0.04400063649250435, -0.03471445573335483, -0.09088469303521551, 0.16338325452746683, -0.042559875426151, -0.23701822186059165, 0.1702766593797222, 0.009683224606731015, 0.06972007289322052, 0.06591496151850458, -0.020350883192709727, -0.009563284627351972, 0.04028334336067519, 0.0796681193098519, 0.0380715063226495, 0.11833121745221614, 0.15478372037320381, -0.07190488233696056, -0.09289950026413166, 0.21401104692559128, -0.00045547462917204973, 0.13870883883697524, 0.13940329574473143, 0.04725392999095188, 0.04055095876750447, 0.07477076873326413, -0.07707295245273367, -0.1128280486805457, 0.02380258852940384, -0.018325428711972264, 0.1281824135280669, 0.040147441329727376, 0.25228844262150213, -0.10593631807842668, -0.0222396311648551, 0.002940018902452003, 0.06027140768475983, 0.03519816042524103, -0.015058697606657378, 0.12779694812409115, 0.13446119777461993, 0.0013353704734431033, -0.15122147685921783, -0.05563845386805795, -0.11821010244782648, -0.09673799828729927, 0.14133372536779784, 0.01632692746796509, 0.06682926531485762, 0.11232663279411179, 0.0002303364387421255, -0.03184541079247797, -0.06563484642226419, -0.026974079017024862, 0.021814924767502235, 0.06035579037674816, 0.13521448676242376, 0.004477093153854834, -0.1547396722444952, 0.032567547352143476, -0.10818180862420004, -0.03239189555927903, -0.25253557499810364, -0.1461983524792857, 0.15093586727427902, -0.0001871425467071968, 0.07311030475870947, 0.016313273364270128, -0.001769174876847772, -0.043981648398793725, 0.047895010187961364, 0.018836678078096972, -0.10784037724840612, -0.19642975562885737, -0.030862950105915943, 0.020784133220276984, -0.05720587637242023, -0.006147886592717472, -0.04651565730570041, 0.02872985784070786, -0.18312157324772982, -0.11614854586892133, 0.013156646620805599, 0.08066553170415726, -0.11920384451490414, 0.025024161366553485, 0.036396212717321345, 0.04669049529403823, 0.03638421380264995, -0.1446179494919538, -0.055519819234586584, 0.00825548553725286, -0.10745788597689471, 0.03294153405134258, -0.08124460404370742, -0.13923181567488116, -0.22003371167000696, 0.035933781563208374, -0.02799039613800091, -0.03519487983028893, -0.04701691506271131, 0.0435816984008306, 0.10575985990030434, -0.052190252811803714, 0.008065703260079653, 0.04901021428711113, -0.04944301790643321, 0.14730909559315725, -0.021028830977295715, -0.07572198211092349, -0.018016746420433556, 0.030463965396963843, -0.10362660992450556, 0.1707321254036352, -0.03292092770878964, 0.09130194835710909, 0.025010698870581438, 0.011356955446828904, 0.041511349645571446, 0.03775256966901666, -0.16453997675538476, 0.07362182836388405, -0.02260855293273938, 0.09308564199108181, 0.2077987439518128, 0.1678610774084273, 0.01643558289157409, 0.020921259332149462, -0.3116616273790657, 0.04217373347711698, 0.019998198565164105, -0.010242865456256568, 0.026094729461455527, 0.1717431242717164, -0.03878697218504184, -0.08521231325871927, -0.04786651518730275, -0.07056081820362148, 0.10506793923992995, -0.019133826022832912, -0.04980617583393693, 0.08560703290075611, 0.12070156191647534, 0.01782602694156271, -0.11631800778240349, -0.0770117120611917, -0.010888438294207554, -0.16419150582541941, 0.11591243193975498, -0.039576140755260586, 0.007108129411267957, -0.06893794637584075, 0.09783845370690987, 0.010369364106618175, -0.028561970718276163, 0.19438390891653684, -0.03176435253463715, -0.08641203460458427, 0.06895398439936884, 0.01981522201465915, 0.19393884484870236, -0.2709808963378972, -0.03004536546577878, 0.0276754110916636, 0.122413019682667, 0.2317352709484312, 0.0016226834748717795, -0.049802469506182084, -0.0165259839376479, -0.04222575760809218, -0.13578874395317705, 0.25805926229654147, -0.010327409604542457, -0.08293564838961523, -0.057512262332064995, -0.09279494706788231, 0.037957042966145896, 0.1190865410449954, -0.03867941578580647, -0.16768633254251447, 0.05483809730525501, -0.11233704681524295, 0.10571219117676145, -0.011094640553949008, -0.12843272524607233, 0.03077110121953172, 0.20319458774082574, -0.03627871153109218, -0.03315908600754882, -0.04334178733871585, 0.01071555945339797, 0.017228403144149414, -0.06619091817096359, 0.022305021985690916, 0.027984263781040252, 0.04133579928953109, 0.035421565957754114, 0.015929588564729515, 0.05457870865515447, -0.04082012203358786, 0.03415941855583329, 0.029356342225863467, 0.01904370368625217, -0.11208459935662607, 0.033014363127381534, -0.0314075127001857, 0.008236509570347727, -0.023892403831971252, 0.0003112763161696384, -0.09129460281740871, -0.09637785806267557, -0.1150567236328279, 0.008196232979126437, 0.27568389145674677, -0.0753174095096392, 0.03332436547125435, 0.04801828528575765, -0.06268914464015443, -0.07995139782963741, 0.07049984648474768, -0.06558929390245337, -0.11968832211300792, 0.046224736139748715, 0.06670714637939856, -0.04912937728243726, -0.2179683245446524, -0.04898952308617522, 0.031893118247938126, -0.0974350889689182, 0.054041067254039586, 0.0385970212661293, 0.013079858485680954, -0.04056687738256721, -0.01984248431450207, -0.07165953292147247, 0.022056932737947614, 0.014628631751683344, 0.059176266205751546, -0.019162414888894783, 0.27383572214456825, -0.04294880407090059, 0.007841905533520576, 0.009114896599129663, 0.10422509480589179, -0.12512015305521804, 0.16051090328216808, -0.0777307597462695, 0.08165313628111279, 0.02269725006371913, 0.02516304542472891, 0.08849913282921042, -0.024498662609116775, 0.09398868906705009, 0.1907676806162051, 0.10868306973363358, -0.08230297485023494, 0.039749785704175034, 0.022426966917330034, -0.047862741521467383, -0.06853570852788714, -0.0213780156061484, 0.04246019122706778, 0.1386001044761008, -0.026142927509065324, 0.020194782762432847, -0.007631523246291521, 0.22276753554702924, 0.012014244643697504, 0.010098682365841919, 0.0017794001042443226, -0.02943990030510212, -0.10405164148379895, -0.14040392988325792, 0.15376532325147935, 0.17187123750255784, 0.09607514687245941, -0.09295242877500104, 0.07504702281024782, -0.06347544106185456, -0.21141305675269015, -0.013494310354189545, 0.03300176042705959, -0.1062271111349633, -0.03176466897337622, 0.09814510813629458, 0.0009175168592680968, 0.02852239663509813, 0.03593670309397971, -0.05009863234383704, -0.008656930674286413, 0.03484537652101195, 0.11265180636226947, 0.0339870368338407, -0.007417514070571883, -0.02786491162732765, -0.07843320653624372, -0.012878073721885912, 0.1746052056707345, -0.07790403678916896, 0.03592015147871247, 0.027557512842820913, 0.1298887827031105, 0.09727681140188643, 0.00532082277746964, -0.07715145091330605, -0.011611786967769384, 0.08915841295263703, 0.049739300228798605, -0.08168765072769703, 0.02259914876997418, -0.0150999676503977, -0.005857742871094879, 0.022373216077376527, 0.01735729230356224, 0.11068211019138019, -0.05911320949467016, -0.1385223479042801, 0.19712910416306126, 0.09603812841778872, -0.025415888899522943, 0.03362955602341106, -0.01507989137161849, -0.0313234915965907, -0.07599050903997044, -0.062332980546860935, -0.14085420270897267, -0.04866331062959093, -0.0027717775699293906, 0.11593786488461975, -0.11233146177190276, -0.12764575483625243, 0.16967192370424436, -0.023219420496272847, -0.02255843199006862, -0.12112152213730731, 0.19854112130978332, 0.04104964217052717, -0.11902298044537886, -0.08188692321859992, -0.12922901482790652, -0.05954896736995568, 0.19093261140786794, 0.032006003063955915, 0.19626734668965554, 0.3083355774375187, -0.025834408049221803, 0.1962873198670757, -0.0006350040449589463, 0.19231434781929707, 0.029517303513250524, -0.07045944218248938, -0.005657770986852453, -0.006953256076917413, 0.018244621800217742, 0.1469293955142345, -0.043215747594827866, -0.008629095375871502, 0.0195176848944297, -0.07716127816648248, -0.04569062640511301, -0.222375045395846, -0.09489052658409856, 0.1328926388363396, 0.003316055754909077, -0.04869586155070224, -0.05040278113256108, 0.020237841789284467, 0.018307105229093294, -0.17757490826259803, 0.053073300936372726, 0.14526285432764416, -0.012348388868778898, 0.017256542160799127, -0.050692981858418265, 0.20898108012097705, -0.08828802257251381, -0.09142771049484973, 0.023621775141742272, -0.030333946547543258, 0.18689326672703882, 0.02780917675753416, 0.08694241437331167, -0.05365597190290131, 0.002745272451115263, -0.004153642812518902, -0.1862225298442545, -0.027185731795534233, -0.04533581216408987, -0.09643000676414673, 0.16968790882326373, 0.03324775344461616, 0.003506103505011436, -0.05875328628158669, 0.019100976129270968, 0.2709449502770544, -0.026789352695672067, 0.030074471430211053, 0.03659319441100033, 0.04571211572247935, 0.11126962412353342, 0.00595926088264354, 0.01896635424042893, -0.015767398042762473, 0.21697844692077958, -0.1472298467080354, -0.022741096389176076, -0.04267141604929398, 0.026180776839412778, 0.2094131359980933, -0.0799682379637318, -0.07192918419569641, -0.10358488433499281, 0.014777792771893053, -0.06759714282687695, 0.1537125387278781, -0.030582474638288823, 0.05699965280537907, -0.0599679887377641, -0.06596757069033325, -0.11123572410430345, 0.026063163948013644, 0.00846840785959316, 0.08830410658446942, -0.05954122900037187, 0.00666588183428227, -0.011521438347370913, 0.123422928024659, 0.19917660088706546, 0.07901123990154303, -0.05108098298843454, -0.10551536172905653, -0.07906826199265932, -0.09831093718836288, -0.024979268951373996, 0.12700890020838124, -0.048537085905109655, 0.04378812018526407, 0.036810715181630656, -0.15434361134474384, -0.03715771162922924, 0.06878924884785344, 0.0401328479907952, 0.0645156624311361, 0.004711779962745865, -0.0051535216789793635, -0.15269886532404348, -0.06101426859414696, -0.052637757492119186, -0.026194681328836338, 0.028611044879047358, -0.15409480538494974, -0.005499311492634257, -0.005494385379585597, 0.15384236110346441, 0.1922991868489217, -0.00699153091525242, 0.17383079050574887, -0.01836397610651519, -0.025376914502411967, -0.022351068739215604, -0.03388159270030174, 0.1023097680388529, -0.06429453990031342, 0.1879193964863853, -0.10158371671873802, -0.045557642052898494, 0.011997738624505316, -0.009781330746045105, -0.050303761963245916, -0.12942239271482178, 0.006648682146962719, -0.012483098249534044, 0.007096552678871525, -0.006982228523986558, 0.009070951333503876, -0.07257786292600857, -0.11914116466346679, 0.20967928276566414, 0.12805144063194032, 0.034394957506834356, -0.07855441698842218, 0.11491270784451305, 0.08374029056532664, 0.03386639739457378, -0.04196123249240248, 0.0001591193886613386, -0.06442084742626675, -0.14612536141234142, 0.0221873324950835, -0.2952944854294408, 0.21608255140206503, 0.1700770832880764, 0.011827132996561232, 0.1298801354977063, 0.004598015902112147, -0.04560204920385845, -0.05897352369967419, 0.0015092692571937823, -0.06964959725598362, 0.08031670163149175, 0.1792812626862398, -0.03448255959781131, -0.039220067901008314, 0.0988426994603609, -0.07268061418102528, 0.010459366592189013, -0.12782214689759022, 0.1846707243738751, 0.0010365480865858953, 0.0060071441293791895, 0.09284727687339406, -0.036691417004523004, -0.07186071943065023, -0.027983830696742436, 0.011373885150768322, -0.06751464255752429, 0.010725848170715641, -0.07116455280180885, -0.04425063445866016, -0.1864637094243127, -0.049130524139446095, 0.023613282115845576, -0.18455532018663326, 0.0650468193452169, 0.056045223888506114, 0.05150199685548269, -0.12690003695552854, 0.03151976148865085, -0.036519254583744803, -0.038241739605014834, 0.11261034858428151, -0.07319265131580312, 0.009249987709413779, 0.02317687851050791, 0.08481217193191326, 0.02232412981057584, -0.016924879142735834, 0.2163214897932512, 0.043340367660255175, 0.021350443748178682, -0.03281784091741651, 0.12428970267088882, -0.07536909775583973, 0.04528025387183298, 0.10928149113152075, -0.09137320447998647, -0.09825008054242554, -0.12348986830872584, -0.03655091987202007, -0.031076793752785714, -0.048378729985547764, -0.04301367039272088, 0.035919073300431904, -0.06421414973837956, -0.07922842051952425, 0.20619511880889077, -0.3924789448338552, 0.003976363244891833, -0.034574801741793504, -0.08654009253364617, -0.11572029284297704, -0.0033885448911717777, -0.01668079714200327, 0.08125378934173821, -0.031522115284446345, 0.03462162563694376, 0.13420573900584534, 0.11150106219746571, 0.004408641094878444, -0.03918704189243808, 0.1944264601222543, 0.01726019338616711, 0.033844663298192953, -0.0619951688274352, -0.2527499892646602, -0.03907064750235482, -0.00926331357720789, 0.08525147052957609, -0.05036622538421053, 0.3801020547750082, -0.021901903361060286, -0.12220477219046247, -0.044690074571724296, 0.1755251264876719, 0.05036270696100526, 0.13067680976991808, 0.12607648388865572, -0.09294636003563846, 0.11978370777947513, 0.1648621902919748, -0.17095779509386674, -0.07173379605287947, -0.07375379402619074, 0.013329350940532132, 0.13130842308008253, 0.04544945539465468, 0.034618050747976005, 0.03184025252712365, -0.03807748721122108, 0.14357285404035616, -0.020170218425784956, -0.07175222418569874, 0.04659460290929708, -0.07698796268195324, -0.18023927347402954, -0.07696459152080576, -0.025022209897681138, -0.04399706410353928, 0.0713025130893111, 0.10962038137268645, 0.042253329363554915, 0.04925769812805386, -0.08580531632518719, 0.09860766362990085, 0.03290634425331339, -0.1588976593741442, 0.005132964557727689, -0.06928775304710594, -0.08192555048980493, -0.016421240925789442, 0.02221938292756786, -0.19541155637310978, 0.010307084080012156, 0.01050055584381491, 0.057188326097754955, 0.15111487473804816, -0.06094558870627703, -0.08384748270408947, -0.1838457670557021, 0.13784134286351604, 0.059231771527203415, 0.18169252653782647, 0.005954033575682443, -0.052614056033854606, 0.12131067816989725, 0.020528344991844694, -0.04529884944842846, 0.06785586701223491, 0.008163843353732405, -0.06353637836058841, -0.004646899083181492, 0.1408383418002714, 0.00047189421763666086, 0.12378019776782544, -0.007218623069141125, 0.011507896429149157, -0.08018111410204272, -0.05392751973172021, -0.011166056844819642, -0.23715447060587516, -0.12285982983531436, 0.05704066335649967, -0.0201569585162594, 0.23699532273351637, -0.020729821089117, -0.006452300023967468, -0.04908540714506753, 0.18410217035701998, 0.14132708765136023, -0.0007659947372024807, -0.22023437685567213, -0.04164851838184037, 0.006131039616358818, -0.09778399584640374, -0.2088381034243654, 0.04028078573118414, -0.11997394209948688, -0.03674182882102158, -0.006590172774240778, -0.06067510697948267, 0.20375501738214788, -0.11699892883034274, 0.04402598594626466, 0.09311385453081508, 0.020722365660960907, -0.00837904793648332, 0.05507297554032687, 0.008068097235733434, 0.009855681252961598, 0.026090762888843152, -0.2027082193599833, 0.19459440082488952, 0.0112347899030881, -0.04786881967386035, 0.01479279417338533, -0.0497313929506261, 0.20887723453701684, 0.28078250260544135, -0.09023539953651553, 0.004055250856659237, 0.0016589275318651976, 0.04130980266488728, 0.09209823402748023, -0.03157303288517043, -0.03069206641170874, -0.08232437771251581, -0.037058259171781, 0.12568976881096885, 0.011622929235854937, 0.1776740912024082, -0.020829161766834824, -0.03221349542491436, 0.1312658544109306, -0.07414174639835645, -0.04117908087885735, -0.0662100141064954, 0.038119276877517275, 0.025249401822833254, 0.13240183334754557, 0.2066073523361466, -0.02148307114592243, 0.04341795633867716, 0.006256510587493802, -0.09989804584008002, -0.017482366422459407, -0.11058287080372453, 0.03257042977099986, -0.08028128390663804, 0.13860447735329504, -0.07508487695808039, -0.0497992921879638, -0.11227748928307194, -0.036194624222018926, 0.16200144809879347, -0.11754998145459392, 0.25143074546203986, -0.0032173708415899806, -0.05039350537355872, -0.2505587583708132, 0.14102775887995492, 0.023389214137578756, -0.025525828699897266, -0.03655589580659657, 0.04667739930053429, -0.043452331221029636, -0.09730832759361725, -0.1510827539144741, -0.0008564261475669161, 0.1805370129842679, 0.16201175249386734, -0.19737149430107093, 0.1707224888783109, -0.08669641935428869, -0.11498631028360182, -0.04131533035050612, -0.06042827504809416, 0.14795714305344818, 0.018744714566395895, -0.08140554865086404, -0.02226503319469476, 0.014278635963292978, -0.17098714588296077, -0.08842046729013071, -0.07143662910977111, -0.13658161239926428, -0.016854383882991656, -0.008453976169512774, -0.2056353434432676, 0.0244952111368982, 0.028109721958696387, -0.08312827592451429, -0.06458416218107844, -0.0017298775017124588, 0.005639081346835565, -0.00475421118946415, -0.058749466186367276, 0.16305618484177622, 0.03833821404103863, -0.00848806501429399, 0.047903883139870845, -0.07415849927763081, -0.24936014378945776, 0.05473864794575506, -0.08183737817138573, -0.12539929740923522, 0.10224786750367584, -0.005672113834964393, 0.03514053460323652, 0.015361180278777897, 0.008965265349122706, 0.04556815209612321, -0.020385933303423833, 0.1028712081093801, 0.0026536717857498803, -0.10181754342624638, 0.06311615233513501, 0.02928799827319611, -0.00037092675973590564, 0.09549226872770472, -0.0019406841074882818, -0.011230013339937768, -0.03562672817305954, 0.10851127172101718, 0.020123760514680917, 0.19944032381835416, 0.025825665451353983, 0.03948891901831529, -0.3682012209763449, -0.1361577592456659, -0.05547101369427276, 0.03100837956552516, -0.24499590635533428, -0.03848592604421279, 0.25916044079704637, 0.021725156209163567, -0.051943485920307016, 0.29577604677954256, -0.344729313800759, 0.02391825768706449, -0.14214023172801746, 0.10164725101777705, 0.0755673312513572, 0.0347349331905608, -0.05471959425196944, -0.1317881551573969, -0.09688130403502863, 0.049194020626673336, 0.008242314353122093, -0.16540593436032688, -0.11848888682497094, -0.07173788304142574, -0.03171963565985111, 0.21870973697787133, 0.04799632883781187, 0.10791406090113945, 0.031880279174594385, 0.19037945393398337, 0.20857310990245548, -0.18840042295457093, 0.08743234905601276, 0.10442670372207631, 0.12635176866811396, 0.004722065550230131, -0.012514455764689096, -9.421003105937932e-05, 0.02117644542315197, -0.08782850307264987, 0.061249459762080974, 0.004944810507745959, -0.04842807697894935, -0.03176326333797806, 0.04785485882019722, -0.12193998124683383, -0.019955974573599097, -0.027694215738500266, -0.08724791789793493, -0.03263480986934572, 0.02699050815984983, -0.05525454602262452, 0.03809347096562364, -0.16273072314906215, -0.2361665227206991, 0.21006050328646378, 0.10777388179265894, 0.03558759655596704, 0.037992344334624444, 0.030671985334907812, 0.016480482155063574, 0.006859283393130865, -0.024669351172872646, -0.07030052447057435, 0.03739308535619553, 0.14550112772592266, -0.276528602780762, 0.05158252079825514, -0.10774369450766873, 0.004806965523656555, 0.10006479140730175, -0.006148721801382595, 0.03203957087906747, -0.22329809741312476, 0.08134109786375704, -0.020403109007620792, 0.09296724883295848, 0.0604125871867093, -0.19765608992997172, -0.048748030931068845, -0.2591421159297279, -0.10864742598822175, -0.06673634041988406, 0.1396201615742119, -0.06098997543916577, 0.1988708600687622, -0.17241895902218932, 0.019684239371120852, -0.09509584575159725, -0.0583050362797372, -0.11119732357941002, 0.006407387333184428, -0.09189423866126573, 0.1590405837441268, -0.11684296087599287, -0.006956049094644623, 0.02027387128698334, -0.016289664415430494, -0.08786543417809065, 0.10317382302123171, -0.02409696491647597, 0.10881734534084483, 0.1552820011254844, 0.05673715538195781, -0.12888109155840258, 0.11390794763988671, -0.06207950595558278, -0.025371861536730603, 0.20768071399667862, -0.20204417193090268, -0.00028969596839126623, 0.027261465240306943, 0.01536118027492688, 0.024001993972547007, 0.21772063316589704, -0.0287695206817145, -0.07455279963114611, 0.02225414598758372, -0.01949839050260649, -0.027235921422533834, 0.03511592286544347, -0.019552381762478754, 0.10288543033723356, -0.04460180239197035, 0.21896884438770917, 0.05945991858960245, 0.026945786335714716, -0.07693275843671968, 0.034618050747976005, -0.06826689198660695, -0.17441636657059614, -0.04365190825145517, 0.09623660598245327, 0.12427924040076599, -0.029652009594207388, 0.06453879844686786, 0.03526029198002233, 0.0576012319654172, -0.005967798734602499, 0.0026135936842740584, -0.020228225641168277, 0.21290148055816635, -0.024165724021012854, -0.20236363198840754, 0.08373503168113151, 0.03584957157506773, -0.07188015908165156, 0.26868828088146546, -0.08855219233217787, 0.026790696064048707, -0.14660306480843666, -0.1231190772937874, -0.18512995238106347, 0.034843500748461355, 0.01617472988531693, -0.0027506297515660937, 0.04292930885024759, -0.0036222047605986704, -0.02250470341037808, -0.042557619372430806, 0.03989480629636714, 0.1348719905260353, 0.0017641086734458708, 0.019110025346256998, 0.004682858765940688, -0.06916762804802307, 0.20713964191577044, 0.09225373733656038, -0.06481084592721544, 0.0934437578579929, -0.009128232919143288, -0.0023627210928588837, 0.08634915241089494, 0.10162522177366819, -0.15200040779358986, -0.033406949303062046, 0.12912372925933813, 0.03530083952767648, -0.06006233121872133, 0.015380767042094104, -0.17375764026828722, -0.013210347228516132, -0.05868544261734489, -0.05223124086707953, -0.04074202895202882, 0.004642265073691639, 0.009611866148666423, -0.008358382031992673, -0.06512997338192987, 0.0129792639669773, -0.027866655131163424, -0.012833033185597805, -0.1059343626070505, -0.18976638983230806, -0.05371326024499826, 0.022312628555716627, 0.1737123931034156, 0.0192958007848213, 0.008129116374488819, 0.0007140643067864463, 0.18031078388658395, 0.004862066012405158, -0.021228355115988078, -0.07148103663643746, 0.01927432533023018, 0.027080770199070205, 0.11187555841400072, -0.1933253302607448, -0.029451938488320874, -0.05271402174093455, -0.012600451187052265, 0.03309748227406985, 0.10914715393108058, -0.07420685030272263, -0.18725046997855777, -0.02081178963539552, -0.011283497356329926, -0.03498095318297547, -0.04512941247968864, 0.1256219392603182, 0.014988600377206906, 0.004381892279383131, -0.08491604521576541, 0.015551119202866778, 0.14081363698639124, -0.028207351773721967, -0.13567227765127723, -0.11608752054446583, -0.020535243285813985, 0.09069271423956843, -0.2670635098118872, -0.06489797498001004, -0.1976182448849573, -0.007028370835311746, -0.053523479521440376, 0.024624098475759824, -0.035941589467816705, 0.036119216507094515, -0.082569939882493, -0.20071614797383078, -0.1286722150141037, 0.004191210275109616, 0.10261103121967356, 0.011742676139136172, 0.023869261784265863, 0.08512697450149234, 0.018653229125181924, -0.026124018461100174, 0.0017525292665933525, -0.03278652796736013, 0.0324756228843255, -0.11677333880001631, -0.09270380044333876, 0.018225521498801113, 0.18978794235050567, 0.08598302897169417, 0.14188424607888883, 0.0167658196228886, -0.19160628108691002, 0.018692233967799125, -0.11852854045645181, 0.15626837372602076, -0.05576872680090197, 0.027467068684038535, -0.24738154307993862, 0.03772053626564646, -0.030495349322712473, -0.011906639858683826, 0.01906131084401295, 0.04055851065087695, 0.14742061331883402, 0.13295896442337726, -0.07282850300779321, 0.034135744278848364, 0.177467563338559, -0.18110395904281656, -0.15977556065714824, 0.15676713822964897, -0.061997473405700304, 0.017819706516404882, 0.04344903225068216, 0.012946006642687672, -0.2419810522181841, 0.004361620864496572, 0.03359507790299764, 0.003362896846875587, -0.1379140379815789, -0.07621528688408982, 0.24383431887291804, -0.003629366087906654, 0.014191824606431444, 0.0395993520740711, 0.4262395099718178, 0.15744668932617284, 0.03355846359641637, -0.0012812565617066748, -0.02446023685744695, 0.10037524870132648, 0.0365934785788628, 0.07440973555060093, -0.2146529701418949, 0.011460232281428357, -0.028114662496194546, 0.012523011576156378, 0.019700890662364048, 0.02195260598835676, 0.11350249320177891, 0.002515271590740463, -0.05938070321652315, 0.04141405271255163, 0.13065643180010292, -0.09452117167551878, -0.08295090304836826, 0.08027980482377996, 0.023714775575822664, -0.13436041773295274, -0.022509461912967568, -0.013595635405495247, 0.04083450245804251, -0.04652656385987308, -0.07759658275719236, -0.018059978091165597, -0.006848059852196636, 0.03379166313588186, -0.006280462914642793, -0.03983598246331096, -0.048646313793230894, -0.1817118903166194, -0.02627505726039203, -0.0015637106668969865, 0.07740332392010174, 0.01835453937724974, 0.01564861559286261, 0.025725651854103686, -0.1630250820712897, 0.005082364590549357, -0.16842017910310098, 0.09164595566771615, -0.0007009558828316463, -0.029719999604780212, 0.0012629127672968128, -0.0365627789801699, -0.06257329405735347, -0.014709704164439834, -0.3291812423271987, -0.0647533942392482, -0.049405903591811774, 0.018045435690304715, 0.1702570113935651, -0.1867997892350392, -0.10249225696919399, -0.03596196454883204, 0.010578967444586094, -0.09062805198002463, -0.028205163278014516, 0.034276104550102215, 0.012345906375056306, 0.006332435067860644, -0.045952323672541616, -0.08400071803830847, -0.0634360844645315, 0.23719324083415763, -0.049724141728955604, -0.04199509851986486, -0.20764553467062827, 0.1127398678283308, -0.03053427239927891, -0.0586460607048781, -0.014203123667548067, -0.02438077167045169, -0.01540338778197635, 0.005924554619080171, 0.023039236828645834, 0.01654265624592559, 0.028640673886405683, -0.0748878229602887, 0.1974101124809933, -0.1582049289015203, 0.017808949197354103, 0.03846152026708096, -0.0663592786752252, 0.0295942190717589, 0.18366009634865346, -0.015655499339157736, 0.04953135833494877, -0.13273528114036465, 0.15224970350978287, -0.039996043447457594, 0.10031808085791388, 0.009279933398008616, 0.0037334205785505828, -0.07598540811275149, -0.14057783174905647, -0.0883340329761929, -0.008485607548863596, -0.16371564135187494, 0.03440437496347575, -0.011984187986227236, 0.0763929880931404, 0.14651378567819204, -0.01843769527558687, 0.04272661748189635, -0.01946216129374889, 0.062168148952056045, 0.16473486919193028, 0.02015512706900223, -0.020555543667800606, 0.10237975997293307, 0.17828889518988777, 0.13129250237846493, -0.006785833677404697, 0.12021489398309586, 0.00972676082346361, 0.0640184735978714, -0.03626162378828119, 0.027369571461947878, -0.023107166937211297, -0.09989936598916663, 0.012453450403643108, 0.23495474611789893, -0.04707690586907735, 0.07529999031071903, 0.32893030373636606, -0.025756829158571345, 0.005110262121709034, -0.034536730615531634, -0.05005413646796369, 0.0313617005677373, 0.002426390528066206, -0.165914666171892, 0.021110968309547708, -0.017490095625784595, 0.13311813978200263, 0.10589728589281343, 0.04840604341864208, -0.08620327239407692, 0.03282394318539544, 0.03668193822251226, 0.1713266652084765, 0.1299065295432377, 0.03466085128298162, 0.11506586157564293, -0.1163356364829695, 0.12203620008192831, -0.03610138940733078, -0.1307162020274519, -0.11164530180993387, -0.05936660830509858, -0.0003201652915664331, 0.04131080563798057, -0.031040372413512145, 0.02660850265265216, -0.12185968904811455, 0.035797537267659184, -0.1527304632142332, 0.18069716733666752, -0.020867759452008944, 0.005550493078611969, 0.17804865206750903, 0.2258829661552876, 0.040801122091180315, -0.03345986124612378, -0.027622821385265492, -0.0775392132908151, 0.03456178466542507, -0.05113136065845055, 0.0842588311241029, 0.14309001924430487, 0.1622566763187886, 0.190233491403355, -0.06142297670395603, 0.09116864256955644, -0.042469827990182125, -0.1190996817885511, -0.011826086052104392, 0.08271597566683606, 0.23756559031302937, -0.03777480699184537, -0.09160203333979464, 0.09316026804416432, 0.16266381411263073, -0.014616077092606143, -0.04083861507209537, 0.16177278365586067, -0.14372102501039158, -0.09177261956952562, 0.16135252022056246, -0.09144086113005105, -0.07831452978420511, 0.16498954125836107, -0.0012567029310124032, 0.035800082189704624, 0.009494643065778963, -0.08375404474946604, -0.015146966514112306, -0.008892504678791175, -0.0917107622496301, -0.10480835829018323, -0.024979268951373996, 0.12226727738089509, 0.14865305869318257, -0.12118677395870282, -0.0004362272856239175, -0.018898491182286717, 0.036921031960754094, 0.11422602476807309, 0.10812501717351818, -0.002585230460039882, -0.14574728166599146, -0.006883177196293772, 0.022209805710305067, 0.0080015658528166, -0.038852958094603644, 0.14282787868340002, 0.08640867842801857, 0.03663164928971525, 0.023496075522605596, 0.08093246863401271, -0.010069412406958274, 0.14729249665192276, -0.12002155135656074, 0.005074278497943749, 0.17008062160104565, -0.050916015200634344, -0.04305820792570529, -0.013687799751512223, -0.045042386390732166, 0.15176261562539883, 0.052391975202027236, -0.045432236771958574, -0.004727117315851111, 0.054683615294507305, -0.037887227024024, 0.010759783108317909, -0.019600843267406087, 0.004173895232693762, 0.14969010964154364, 0.1281468017307794, -0.06557488036128004, -0.025404814690689864, -0.2247837914275637, 0.0025714910668887796, -0.052765878982356205, 0.13939820935931868, -0.0181824999997824, 0.03498821351761551, -0.12382382350468046, 0.13117934634105977, 0.07401402627094468, 0.16515868106312456, 0.013041231324488093, -0.007576769337295384, -0.06896589667779868, 0.016947315225102382, 0.009828712229992473, -0.04141515001038354, -0.1253752491740807, 0.046164540004565886, 0.05647627445641182, 0.01061353288534563, -0.024979268951373996, -0.10026001705005139, -0.21094668503322514, 0.008769590592789955, -0.1045307416388653, 0.029350486303492306, 0.031328109223663844, -0.10338882646523755, 0.020969328464630985, -0.1370503001179541, -0.0463497375448653, -0.11709372904185557, 0.019307448153771948, -0.06265205957947398, 0.020027248301372723, 0.015948504359604453, 0.017116967431878582, 0.03517156793652139, 0.00012705050949374708, 0.19192993746979414, 0.04234076154510384, -0.019614082754116535, -0.1588909788997742, 0.03764438111239787, 0.13227397445288308, -0.0883572593291221, 0.11321437208849292, 0.016414843788884012, 0.007772906590154491, -0.002836897367191843, -0.08531974929153721, 0.040909519610033876, 0.022861129172024822, 0.13279120646755643, -0.04806401848185831, -0.0677376203289556, -0.14921673071553498, -0.05617537970963465, -0.05476699502857202, -0.09694871692054056, -0.051858438599508394, 0.04515433656284412, -0.045887768081477216, -0.18049208624325228, -0.11244308858095789, 0.014213093965284267, -0.0265371875577323, -0.020193032529363098, -0.0008349396917567677, 0.0371294847711313, -0.08794997667158784, 0.04350889963392444, -0.09005247425288973, 0.008626925460692556, 0.13940329574473143, 0.01257314878317474, 0.03578263263384074, -0.09648099635027939, -0.023301679525417673, 0.03771753214629484, -0.07499737219862831, -0.004585718160972344, 0.2533999758651619, 0.1710780484795014, 0.03320720972920184, -0.0003748288484440123, -0.226798919817053, 0.035808222494740756, 0.009276918263172479, 0.1545387672203533, 0.10795125821159843, -0.10429548428878131, 0.17252524363167418, 0.2160242767224051, 0.06100354716272202, -0.042709888047130565, -0.023592073525241914, -0.0047662983823064595, -0.003909135727076463, -0.028304088320732317, 0.058916930264767686, 0.044040315290091996, 0.01739865258721537, 0.036800323248272475, 0.03947420668832647, -0.03876240814690782, -0.033423776583877594, 0.17606164324785278, -0.060509435111033995, 0.043158980309661615, -0.033267207820001796, -0.03412097117064778, -0.04172390401313564, -0.053938179437156446, 0.1046489587065888, -0.0534454315934074, -0.00509024178901851, -0.01825302037143675, -0.09507992550825048, -0.07587416937409047, 0.02784526076673136, 0.03332970610605081, 0.03291789017253876, -0.015347748708056861, -0.02493125512033893, 0.23317123587169447, -0.05883591238903598, -0.01534949461310773, 0.015444544992374893, -0.09370398184548118, -0.1265235763976385, -0.01721403001541283, -0.07391818159176831, -0.022260832040264415, -0.017928612632988325, 0.012130710638742905, -0.04160855810716519, -0.0027837475851366276, -0.05028480070893761, 0.007299194070454611, -0.004034839122083056, -0.05040426938278314, 0.018365823238388854, 0.016569048230165175, 0.05927288935207945, -0.17008498652979054, 0.03572177437216382, 0.10703338710867233, -0.08039145343489265, -0.1069339348875345, 0.017562337073058615, 0.18452067347881954, 0.04600872576120874, 0.1363595601843602, -0.0586609014119257, 0.009013220539227056, -0.05685832028047046, 0.036635487922180646, -0.0770397806722537, 0.13311599038215313, 0.00827324684004969, -0.06875921064312547, -0.1068004279109683, -0.020524361751972937, 0.14130473669708118, 0.02822273597668578, 0.01217980579275933, 0.01003901151190124, -0.11966577979384549, -0.13772141915158814, 0.11581100132818316, 0.01819386714629315, -0.039860064808863185, -0.0023850747972572845, 0.022623722810438236, 0.009192377264890056, -0.06247746478392003, -0.052714564117215375, 0.022850452650315652, 0.010503928472175463, -0.05126783983501303, -0.03726984549949443, 0.13719783826107318, 0.08607633336604653, 0.003814261201239576, -0.03814503525058431, -0.01212209163448997, -0.11996147609521286, -0.11944955725667453, 0.014355602000208674, -0.003536297753240719, -0.0943990778139752, -0.04482285258618996, 0.025376742865524948, 0.050537002993425124, -0.06806266406476447, 0.17059964047831797, 0.029419344927061993, -0.027618342352939754, -0.2261018492042249, -0.13248460363382283, 0.05709599378447115, -0.04789729001706203, -0.0010042300987942291, 0.02498804816041554, 0.0014084335704033682, -0.055386965203491775, -0.05017756992447616, -0.03285959751156431, 0.021447751479614648, -0.004092650680337849, 0.0923530258260691, 0.05686607788218095, 0.0827675729287506, 0.11783894757411563, 0.013408946512001698, -0.12966246961726516, -0.15170239770370092, 0.15494363662878596, 0.16923539742852423, 0.029845493204037753, -0.0011201188873281353, -0.020313125746765567, -0.0006384534716931077, 0.1417627745704792, 0.218580476671398, -0.07891495905657125, 0.024220196800779425, -0.013211972518817016, 0.11442409925357208, 0.14013751699427918, -0.11948681911711291, -0.06401937042155384, 0.17658610817643355, -0.09256706625620746, -0.06868319396018019, 0.04731683685370832, 0.10977189048605089, 0.12639573116748182, 0.03213184469029533, 0.06506430176962279, 0.2354220047979703, 0.004029461960417142, 0.17836620380581641, 0.013393335113127692, -0.022835735983299632, 0.13544412465300337, 0.11761354927517811, -0.02554830761299371, 0.1832214399465504, 0.025146555047083286, 0.04965785445990388, 0.07584355408282721, -0.06132620121476026, 0.18689989190413187, -0.029254867875185785, 0.06760582156785266, -0.18493107154099792, 0.03187111432914156, 0.061319819005533585, -0.19953657857083323, -0.0440782542125741, -0.12142894999251817, 0.21401374740398943, -0.06279096821807408, 0.009907104820814102, 0.006030915310321881, 0.11895752818325296, -0.06702812727237624, -0.07968742906836081, -0.031294027139407425, 0.03438249378447815, -0.03802286769183873, -0.062151920238311335, 0.02443067596566773, 0.1381547395908228, -0.04914788635813593, 0.028485221163101038, 0.13372116125278236, 0.10305997605944664, -0.16652227968784694, 0.032104591251408264, 0.00010800021674105754, 0.15138424182039092, -0.06392831860501777, 0.022806789265073735, 0.010641437700143579, 0.036983707517243364, 0.17718504043403777, -0.08902371906591046, 0.06274827055117432, -0.03340915858143766, 0.12683734040627948, 0.0006910822109593748, 0.034074618914717095, 0.033145113389676455, -0.09857531794183726, 0.04700399440535748, -0.005542381978905215, 0.056436537322881127, 0.12874498874899795, 0.035529837020585216, -0.007093700927155849, -0.0875199632596841, 0.00777319009919227, -0.040425439180390425, 0.024296285153436323, -0.012833164203046447, -0.007870396632094602, -0.04326312777907261, -0.0005106384173838002, -0.03310292546039563, 0.001202209984912398, -0.02401417786030808, -0.07829654274789974, 0.024333849283505352, 0.020099553959620882, -0.0012715585308822317, -0.010444869542164992, 0.13329892345879069, 0.026484870972886033, -0.08589194119346479, -0.2926628482793179, 0.0213684618768811, -0.09859898294852344, 0.09343354566193615, 0.1948289681705488, -0.03764428631870476, -0.13452267413200183, -0.10744212633958514, 0.0299246124389447, 0.024679243579338903, -0.06416677327470716, -0.0015181130439211967, -0.057874761847726504, -0.03244061767102491, -0.251746710546183, -0.08301112317570067, -0.07218155506831521, 0.03549719967070978, 0.03453727812924313, 0.03967851100256492, 0.014591889925952688, 0.3218886380298996, 0.14285409193727405, -0.005512691326333094, -0.0494889204593074, -0.0011671123385148426, 0.029078067786903757, -0.00721174598181106, -0.10091622164063996, -0.003576476338569953, 0.003880560503213234, -0.13019341869856613, 0.01346300995975156, 0.10574333574289083, -0.11186239670303613, 0.04539792020087322, 0.15215447057667647, -0.12923068135259472, 0.027616475056668404, 0.035124028618565666, 0.09423903127998605, -0.09017546056445265, -0.055847991331214715, -0.026730023618794736, -0.09430420529231365, 0.2015146301806372, 0.07405430025767945, 0.03965799971788131, -0.017570227702631204, -0.042203702322648244, 0.22062856747392706, 0.17031477244290436, -0.02403113283406898, -0.028842860167766302, -0.017594875587825867, 0.18588188910512787, 0.08411965212593997, -0.0008788130265355938, 0.0071858212195227275, -0.018849086766487767, -0.0011121853951311447, -0.04989247739128623, 0.001517788797944971, -0.061745190754531123, 0.03521027135734276, 0.09357100626684996, 0.044136905075937045, 0.23454615123918607, 0.1734081164193096, 0.037472419855963106, 0.04289378192691768, -0.005547802492991935, 0.08980265015311804, -0.02538447447938384, 0.12016823882367778, -0.08740020640504811, 0.033224389809177135, -0.040709324778508915, -0.028409316586805858, -0.12346566936494471, -0.07439126860986706, -0.09988485187150253, 0.04096614965264706, 0.10626977034123837, 0.2826258868620077, 0.013533771664864818, -0.01144475888512788, 0.031387754197605575, -0.05011484830497694, 0.27244903226693484, 0.018022337039171547, 0.005829515256216457, 0.12724547620290175, 0.018662975856415243, 0.19390915657861396, -0.043443138125704606, -0.22523950668258966, 0.015558900906056262, -0.06667600056725534, 0.12584957408440023, -0.04429567145829598, 0.008496079410063763, 0.01900729198164092, -0.017547551648408696, 0.008892487640163283, 0.17261204148919615, 0.004123740303847573, 0.04337324693840561, 0.013828762502129518, 0.10429344995365965, 0.02897933453722198, -0.07894669845311647, -0.04982439304299314, -0.06500638268932313, 0.1434016760822423, -0.0731239812304056, 0.1088720603208379, 0.008383491272837894, -0.012499419225464558, 0.013494202689809318, -0.20201231722479224, -0.09239851147607042, -0.019410051662444593, -0.08345816188540622, 0.016822147295064285, -0.08809031152088992, 0.021303845973494075, -0.1320858550146645, 0.10929205398135416, -0.1991247916158023, 0.02616566510662594, -0.020911544072944363, -0.1364481809493271, 0.039970854737505904, 0.21950010230227968, 0.03682273797440711, -0.1506998790640468, -0.027306615455749517, 0.0013360358803470942, 0.033864509522842784, -0.05117330229408232, 0.23770331023059552, 0.0004074686848063843, -0.0222490470850891, 0.11203391939339855, 0.00018446909292538788, 0.0749892959624858, -0.09355405978609346, -0.11224886870199621, 0.015315736770340862, -0.17122892565427789, -0.08377266392586394, -0.18489656756882433, 0.18957686446396868, 0.026107274701877837, 0.023872419915598617, -0.1129119186566958, 0.08761141681580312, -0.015442738727143619, 0.03030189025167717, -0.13159792668640394, 0.13183711824335917, 0.2133220503538212, -0.007804421053332367, -0.06907908148002126, -0.05506678866410009, -0.08867099785320635, -0.03182879348944365, -0.0039787131098265325, -0.06451930740572066, 0.09223897430863094, 0.02688882962633704, 0.033186923198448966, 0.02172947089493156, -0.0530261160373234, -0.16868500806221282, -0.011620978687602661, 0.1185929640159717, 0.03620711027748687, -0.079765362252275, 0.02374213202142806, 0.016739680400423902, -0.029728235533699146, 0.19870155487329105, -0.037439078342726556, -0.14351149737107138, 0.058465429202060375, -0.08003715178648355, 0.0034121485554625365, -0.1494024037600297, 0.1468624664404918, -0.030214613728919214, -0.03097743988687772, 0.03679654941042794, -0.0018398652573340625, 0.18207470756478333, -0.0028007719841175757, 0.03165933625757661, 0.06170882903741407, -0.2120540154239479, -0.0710910924878939, 0.06711102668941857, -0.07682927713493734, 0.17705265208707602, -0.19175081020959367, 0.019144766652632098, 0.2622586195596852, 0.10212133639948923, -0.18767214006615168, 0.15541377618093566, 0.07733736480521067, 0.018763500952348928, -0.09274318840147279, -0.002734447791356221, 0.09376833923307032, 0.09566985342631468, -0.2539901056711881, -0.1042336815048513, 0.0789366881542106, -0.009223188914425228, 0.19216025374754125, -0.03881011352184551, 0.1027733496263848, 0.19826673848027002, 0.11837129777053361, -0.054027714048739406, -0.33978526024634104, -0.0034688361409697284, 0.021972948051703186, 0.042881493265666946, 0.046656117905809646, 0.012895509731762312, 0.013729236208575465, -0.1094400941194234, 0.055266029470743896, 0.0008342450475084598, 0.003542045140917243, -0.032353862015413556, 0.02975486235121985, 0.0544530360988219, 0.006449245596349848, -0.007016867538580657, 0.05698270404271927, 0.043524293556970656, -0.12879045324876165, -0.14427103953146816, -0.09995281984340484, 0.03659947669021753, 0.06117558865398517, -0.0625540231632177, -0.06230468435701685, -0.12337840673536428, -0.22055574939831837, -0.03551006562570334, 0.03786853749918029, 0.03210035853086404, -0.139717803136682, -0.05616466378328746, 0.03901658185053856, 0.15336672266586088, 0.07610868121167078, -0.16935005517814314, 0.06698416563122135, -0.0463606373830928, 0.03661487754524752, 0.03543111262038591, 0.03405980166057225, -0.034421876708484754, -0.08562414446529507, -0.15713607380276048, 0.021050580471819, 0.029562164509866946, 0.017859317827993947, 0.16903052137058966, 0.03055353405667789, -0.07521271132393162, 0.03568598596719077, -0.02368890971649179, 0.031481830419895954, -0.007948845284033921, -0.14620959481474363, 0.09582173697553109, -0.010960710656377317, -0.14202830934102043, -0.023998146267633722, 0.0236950111730261, 0.11004476045818229, -0.02835440227816226, 0.1084126266892824, 0.11908163666382933, 0.2057331487999052, -0.11931056951468505, -0.10035264681274768, -0.10697943635574934, 0.013877858526287238, -0.004713400446335333, 0.07648581887233337, -0.09410031568942986, 0.029800723438139113, -0.09659397545163273, -0.07094633304908315, -0.015118336120944972, 0.015238749705491293, -0.04595328478963642, -0.10378373446976781, -0.08570752374431019, 0.11215763118653663, 0.008789320419224651, 0.03881251485170444, -0.013755850843706988, -0.04880871028034682, -0.10310946402827789, -0.11488952338750355, -0.029297421620404548, -0.05129419354512205, 0.03225069064688258, 0.02228417014955741, 0.20207016482298873, 0.06388548297137668, -0.06915638558763529, -0.03193943006884384, 0.02581306715982451, 0.20999664439641397, 0.14858622288044215, -0.03253206793897377, 0.035521249841924986, -0.09939485605767977, -0.21506001317434917, 0.11592913343895256, -0.0651957500429665, 0.11685322520841786, -0.11886574341372486, -0.08475882028466056, 0.012558350516327858, 0.1386312884792367, -0.1877955720205469, -0.008958129708762107, 0.07778300592698924, -0.06292316498685643, 0.144624516002903, -0.07003050039261687, -0.03295193772997742, 0.015718531228471156, 0.010697123213431439, 0.003722739847650294, 0.03285827333121971, -0.11618077981369368, 0.04472707588888705, 0.23890138580420203, 0.02000067560433477, -0.14461552018923507, -0.13701135785348023, 0.04103268166310397, -0.01242001886403278, 0.03200570871439148, -0.03723658791077141, 0.15468646971543368, 0.0377928629870319, -0.10405709216029403, -0.10825982512304565, -0.015631757097452202, -0.11564471859486876, -0.09267792393256415, -0.1037269308454205, -0.17304911794469324, -0.046134762046737984, -0.10777351736341559, 0.03823677256445008, -0.050231044153848735, -0.1883109771011233, -0.0235910386521631, 0.03421827406885499, 0.12071621845703308, 0.008965323184628362, 0.10077517781567644, -0.09517878314914918, 0.007120262789380797, 0.02608586845819591, -0.07132261183225888, -0.02665773847791602, -0.012982314955147298, 0.14801436152004707, -0.017716561837513314, -0.037449123007427025, 0.012125200022336842, -0.05994256419520013, -0.020768853267853167, 0.06234703988340265, 0.03237037648319781, 0.20210866458246493, 0.08901070587902807, 0.026270480764605017, 0.1831800954084044, -0.1059841166362018, -0.2056115282887398, -0.07384271529786988, 0.04523753714183157, -0.009720635650396353, -0.1571090616563235, -0.008126632947913433, 0.028085518500718037, 0.11582662705537326, -0.10500308428907434, 0.023695076325522032, -0.16125703779366063, -0.08576898392539616, 0.00762997509831471, -0.05812000143352315, 0.012771579543360717, -0.01545403044483796, 0.0902440021121652, 0.03690845896533874, 0.0063668876993391785, 0.04242925785735833, -0.014473468108368965, 0.0005603159652677378, -0.06510351121326671, 0.11218180801836315, -0.1396237153172973, 0.35446495454066107, 0.12959762862043794, 0.005159153885115916, 0.006445384744284217, -0.06618109778739896, -0.029143644487543183, 0.08690411233489329, 0.045908303255619114, 0.07458834055488367, -0.00015065927011729353, -0.07275698382737512, -0.09031774574561614, 0.08468477427962012, 0.02369315284392566, 0.08332773492346779, 0.13288955135703936, 0.12196244032367792, 0.06973553918702202, -0.06914391741194226, -0.08568662477827309, -0.02638478028211524, 0.24403071748995483, 0.08771313729050005, 0.022706852686987297, 0.12534795796874038, -0.08248912809251924, 0.026048568237711393, -0.06901941520855759, 0.006956077146348132, 0.02210152935713811, 0.0022773419701372965, -0.006628388007864335, 0.0922093558844854, 0.022083406365978494, -0.03778157107931752, -0.1947879806591209, 0.16768244318027076, -0.03505626852533662, 0.03212661416275049, -0.012485431250457202, -0.07143284870458488, -0.016793261883271543, 0.0015296331439749952, -0.019999102531509837, -0.13978771542397203, -0.07238590119129334, 0.1777065253611587, -0.2882380411916062, 0.03148958138111567, 0.022646742516126285, -0.06604329827202199, 0.03314925039906536, -0.028056216087066738, 0.27097999569838954, 0.0950088233042071, -0.08788119418585381, -0.010241834772204826, -0.11378026265426669, -0.14485932386457429, -0.04569777441334687, 0.013406617121111932, 0.024206354772735802, -0.17250045519791005, -0.01815006349907869, 0.23393047857796975, 0.05841989177395227, -0.005080665240811649, 0.011103117241408975, -0.003545593833277653, 0.04558039221409085, -0.0003632390628050065, 0.056530719705659045, -0.06510009395953727, 0.030673956137406713, 0.03622527138617594, 0.1665105437873735, -0.07037769573685777, 0.10644786721153167, 0.13281599989478413, 0.03932816078259811, -0.09288272409997873, -0.01672303809354549, 0.021911086815429564, -0.012885273670000901, -0.19880309180915365, -0.05409483581677306, 0.0685874628937752, -0.21243810856984804, 0.07294984572368911, 0.012983464208808266, -0.006770295700166382, -0.05731757906476618, 0.01647848031470523, -0.04309816400396243, 0.03322732307219786, 0.31819875025547767, 0.057254773516790675, -0.015889979096570802, 0.028906610457667143, -0.027254079212537915, -0.14748585301585293, 0.02474503361323892, 0.033400763989624285, -0.015715352396649937, -0.001780757648849538, -0.002058111517278431, 0.23891950308371476, 0.2015013157686069, -0.0762339380934808, 0.12751131260239384, 0.10056783348152278, 0.19411488001569846, -0.13385206262090146, 0.13593727200028388, 0.036530235384982956, -0.0994246918669878, 0.012629861416631999, -0.060342493423437164, 0.09147343960339452, 0.010972470643750619, 0.029765568114683963, 0.034081673833904765, 0.02171805718743039, -0.0386820138082457, 0.0840614161758871, 0.033886576870652906, -0.019117179740775453, 0.03881251485170444, 0.007094695634184038, 0.00908411319885964, 0.22826475145074687, -0.08570376382486662, 0.044004478422693276, -0.10523932630206344, -0.059465297492458956, -0.07428664683295169, -0.04039026891436173, 0.0322003924544565, -0.00982958177205677, 0.19987936121329283, -0.1659280714675891, -0.09526543877785476, -0.05383603983670622, -0.06126732648171641, -0.07763517384276666, -0.01157945226954467, 0.07337407876693919, 0.1665270122884362, -0.004742733760485172, -0.07797791823800854, 0.02952677154981719, 0.06086672736284896, -0.05638521336702402, -0.0744603485740359, -0.2164055382135406, -0.11747179413509688, 0.11647552003746736, -0.10496144798502298, -0.17428936832555164, 0.01988819976697407, -0.06652499675993409, 0.1462856983778794, 0.042707302866865506, 0.09156989005997207, -0.013027660335812577, -0.14441725731549362, -0.05800000626175011, 0.030847278531052867, 0.09429340591991636, -0.118780130660792, -0.078676255281776, 0.09149435451991071, -0.043124067273376906, -0.05501966164238222, -0.006556855827629904, 0.06439924763411008, -0.14167365562456422, -0.1121506840780561, -0.08094446815458038, -0.16388046449138513, -0.057916056844962614, -0.03054867726432812, 0.019693407651824583, 0.12532782971839226, -0.04656045544822263, 0.04923339296932528, 0.011694034052127187, 0.012433146427706054, -0.06693046226080607, 0.03935269174874857, 0.12885931821314162, 0.002134359737611461, -0.11963425007210635, 0.0037133889110121093, 0.0622236949745824, -0.08737628495003764, -0.026563929254127944, -0.06449113117711831, 0.17743236758105907, -0.021355445197316743, -0.026805707364122463, -0.05793072938815026, 0.007189502478872706, 0.09795515057036466, -0.01606645153726641, -0.04108532324743422, -0.0900379619620039, -0.02123442151318105, -0.00837565037900639, 0.03500018028367718, -0.03857750927248092, 0.1295626222979442, -0.008533764219280012, -0.037715170495690384, -0.114148015004747, -0.039634197226928916, 0.06679705876040296, -0.05615176380880223, -0.09488034689916988, -0.06174173346025478, 0.08059666345353575, -0.16248986337862054, 0.3258093473860714, 0.08689203580005797, -0.0667729046433097, -0.09616715307567761, -0.07482741338202206, -0.10446435927847028, 0.008717247499815655, -0.03654681613417934, -0.06276997580268272, 0.20592916959721594, -0.011329275326101184, -0.10420885475773198, -0.23498789545019053, 0.002547768354901008, 0.0561682594311996, -0.02970600833991989, -0.22376324287178823, -0.029625143365893888, -0.09798853763633351, 0.0109438761447971, -0.10549979335802298, -0.041663900067460714, 0.03252958699584069, 0.0190204962584804, 0.11291393412536382, 0.03073505820222575, 0.024138758631101077, 0.0975119981721022, 0.08862511582688852, -0.06511850636668094, 0.01917072384052786, 0.18740936919399095, -0.14454106712480697, -0.02712251030416956, 0.0763254610224814, 0.018096142617179293, 0.031045189553166503, 0.02023492479404149, -0.08796218247544063, 0.07636574034916972, -0.08764347420209163, -0.07031160304446407, -0.025587213720557375, -0.009342329195666226, 0.07114933661796181, -0.06322930825080837, 0.012920775708870734, -0.09695160709531218, -0.11476321694377219, -0.05608606830149318, -0.025508571798829722, 0.01067653146226061, -0.09692718502847014, -0.07866032110850953, 0.10927729331815657, 0.028906607126425947, 0.030851866021240498, -0.035319557497339074, 0.018370773403510638, -0.1374124068586969, -0.00744385603482789, 0.12363008974705095, -0.047942405268594925, -0.0643515805112239, -0.18758374211991646, -0.09837405639166352, 0.27972212881974634, 0.20751045333373308, 0.06746792030585196, -0.04318518089511005, -0.024667313386579307, -0.08533815141036682, -0.10424218620726379, 0.21918837411926925, -0.060100673948005084, 0.04546053447725662, 0.016851227673672538, -0.2361906643408316, 0.026110253003383654, -0.043762131486075176, 0.1662110162928944, 0.07393287203131213, -0.030835223263392306, -0.045432549842175186, -0.15128837375279097, 0.10476460816286266, 0.023721182210128706, 0.011241781446461973, -0.011406214256684869, 0.08871244890128181, 0.027358216842819527, -0.09159638442125462, -0.08809435736151604, 0.1232732129784999, 0.027255257098284663, 0.043963118234919774, 0.028214209636092613, 0.09844321374651631, -0.13858460870375067, -0.05322193693485743, -0.007983101476525447, 0.0038013498191004683, -0.05375256120876702, 0.18636406139993814, -0.044090816977730714, -0.12698255395126434, 0.027399260043100136, -0.0862132404933174, -0.048708555972048324, 0.10874350912239082, 0.18597337344239048, 0.24652288236597256, -0.15106936180581257, -0.028994947466976815, 0.03335214476901735, -0.07820514115492778, -0.07073837308321475, 0.03967131793360985, 0.00432542516599797, -0.025175429551987618, -0.035421404440836066, 0.019860696780449393, -0.0067375487673803506, -0.013207194286452844, -0.03674418782108952, 0.09042556788023887, -0.21101498139602287, -0.04050120994284819, 0.06220258677042791, 0.13020780893159828, 0.004161789408930174, -0.008134641872632792, -0.025202090302389468, -0.03791837964017723, -0.07591805742956317, 0.01714946905147655, -0.036413109691541636, 0.08112240172340215, -0.0925592330567954, 0.06396487104520678, 0.0023618907380075402, -0.08015122236945693, -0.00937221433011332, -0.007402514318995991, -0.01730816011659245, 0.029359774138223056, 0.2051521564499056, 0.07162680131445177, -0.03582094524769978, 0.1081921708244029, 0.03710593779664265, 0.03790687347564915, 0.0198259982593005, 0.0406519604527374, 0.008475787280591902, 0.19770393589157337, 0.10587245442062908, -0.029233017189426808, 0.052561153615971246, -0.25227356223904657, 0.03432668200336679, -0.13917310096176594, -0.008266586782087268, -0.0825494963118366, -0.0545882943353694, -0.013133458749117902, 0.002096969477120021, -0.10860616369643694, -0.13836082881857092, -0.01728579380367004, -0.005826528703071769, 0.04334785114710743, 0.015335139427754456, 0.11552761633525344, -0.19928065575311615, -0.05227754485414704, 0.05327549003441064, 0.20493391121753785, -0.04681852246136868, 0.02287722877437571, -0.058942309496559826, 0.023685455593267103, -0.04786900622540974, -0.003083085710117635, 0.1168889606233466, -0.1020375294703301, -0.01801842765100272, 0.00860958650242872, -0.1580372069681429, -0.014538499204789902, 0.25988549517236587, -0.05196636533629681, -0.07443880114906623, 0.002511361588462403, -0.031748140318319615, 0.03298478166455999, 0.05040324518750422, -0.011137133932999881, 0.024756168831725177, 0.0033815314656427675, -0.08537117969570733, 0.03819196529491415, -0.0060143600405537425, -0.017568447115431225, 0.051388998564234074, 0.015924339564782546, -0.1984371015201807, -0.025415903755522388, -0.11454099481649624, -0.08746733149933848, 0.22504571674579482, 0.0405503232064805, 0.06145635452654063, 0.046372215253816365, 0.05008050091347838, -0.017832645738940226, 0.00930252353023712, 0.24415588770103872, 0.003632791024079416, -0.008376025404360481, 0.0012709329521533555, -0.0021204066611617113, -0.018172955248697113, 0.016242078336731882, -0.0181824999997824, 0.04487261685400697, -0.030961682506658338, -0.10340813129084928, 0.07527877606308822, 0.07466754889935072, 0.021069542922644492, -0.10827658913604282, 0.02563841761828219, 0.007117276390767599, -0.05068721594024576, -0.026106731696543747, -0.1882161527387183, -0.1245000049270717, -0.03801143605511674, -0.046088118657721895, -0.0731740013112686, -0.1089621877154583, 0.10478003542994503, 0.10331424529149841, 0.13789608318744562, 0.09172469089085723, 0.005026995081834031, -0.09563129086710205, -0.07529487557525238, -0.06254973483958218, -0.17630438268592147, -0.09302638864112865, 0.18433564275319883, -0.18100143960141404, 0.008256326646029264, 0.028540136305693117, -0.0010484985601948832, 0.015363442109172346, 0.05632867723872621, -0.06760525349649761, -0.07068929186034006, 0.0025401168136436007, 0.12923776594312902, 0.14061042613030558, 0.15672089013133111, 0.011596315660132792, 0.038324242858107774, 0.03638150233223395, -0.2026891963791708, 0.03627834002096017, -0.05139275243279579, 0.22177852034994633, -0.01276405353794704, -0.03690346543197565, -0.1955626821890915, -0.05128207657916194, 0.0545393174311928, -0.05777721551268336, 0.09815542275750597, -0.0021870781676811956, 0.02114497123491257, -0.045701478531434185, -0.02158405454553765, 0.004888598929026556, -0.007740904625952128, -0.020643128248585604, -0.023579012802725532, 0.05478771388764137, 0.03009369046796935, 0.04252039016751202, -0.05541043684147965, 0.0248101508882974, 0.03713609897709964, -0.006841898360870657, 0.021281712475729124, -0.12647133449205178, -0.015974128286205452, -0.043112158928743774, 0.027127685215207864, -0.05647614905816557, -0.21767081172554756, 0.031147376561773972, 0.027059276545095358, 0.10983247749163726, -0.021223408197110622, -0.012034185459042572, 0.1400450717282833, 0.012801220302015654, 0.19559865945336466, -0.06403491165576733, -0.07891186243290473, -0.01647179032224842, 0.04076407717647689, 0.02346234403914534, -0.022270313707350237, -0.04039279724030745, 0.08077090433238489, -0.0051165277494048065, 0.02825167652884086, 0.057793674832002914, 0.023950702009711555, -0.04828263101054743, 0.02244152817825631, -0.05380340376841421, -0.05309481746000615, 0.10313127411139839, -0.07913542803295726, -0.06867720591322093, -0.08046885079700491, 0.028128884050751268, 0.010688135684299783, -0.007916013785553525, -0.296029667925713, -0.26666101226013744, 0.03843031140343822, -0.10472196967215704, -0.01478749110513729, -0.03411972983739035, -0.026680075003653977, 0.03585753112560542, 0.0034796255862382633, -0.003719255916210053, 0.060939201901471125, -0.007139924379822882, -0.09096057463575825, -0.01806606638097275, 0.12117200406879125, -0.06046475351757849, 0.09211833464760087, -0.024979268951373996, -0.05708423700499897, 0.10369850898028976, 0.10392070294120119, -0.015144140643040266, 0.10622446749937425, -0.00019344258044784018, -0.05951775862225949, -0.04288977575972457, -0.04367631123130299, 0.08302026155827148, 0.05169935137531072, -0.045900997101077264, -0.0076129589360563075, -0.027196996959682845, 0.11728083818140216, -0.11807063127666283, -0.017392397965633875, 0.12342375531683329, -0.08828369601463937, -0.08529103315614346, 0.1716269295902779, -0.11841751748078314, 0.11864856092714483, -0.10403931308423704, 0.0018043370060269613, 0.015068235693124463, 0.0011990982293718997, -0.04875210733966546, -0.017985364261288633, 0.007056307052625793, 0.0024767728370020344, -0.30186203832224145, 0.014389881934295117, -0.04542061587874696, -0.018247230288673734, -0.04450662738204091, 0.03187521504224927, 0.019866133788706764, -0.07579906947865146, -0.19815705766523764, 0.007498253538218627, 0.009954750769654758, -0.04847610255124684, -0.05486741328410615, -0.14212954581150578, 0.005246189936277428, -0.06364502314807353, -0.0047395583589603645, 0.1765964213151864, 0.025308076123256487, 0.16345713261419328, 0.01791344624903737, 0.10566776656553563, 0.1660583131738434, 0.18442587186051596, 0.16704968342358595, -0.18109549164370048, -0.030168396068595248, 0.06659391641316514, 0.05628910709578948, -0.029123422566937614, 0.0937139501495521, -0.14457542479340085, -0.11010864778995094, 0.20416652301669355, -0.35656855133329607, -0.14280557759622164, 0.11092772555581047, 0.038576001314809365, -0.06417424191867872, 0.044110457781154244, -0.0096139440247861, -0.009367043689282331, 0.07420826041698313, -0.15504870625681316, 0.027294779274844107, -0.1044842282195864, 0.08461868516445024, -0.18830593461167341, 0.029448206351448382, 0.059860532977597136, -0.13437355088805195, -0.0415321507813009, -0.1767249443132375, -0.0074285678440721955, -0.041413942657017984, 0.156757709504527, 0.1784000917776198, 0.17578764429621577, 0.03712377925095089, 0.20212376922481667, 0.03881251485170444, 0.017299248949246775, -0.05353587263335218, 0.30263204545871436, 0.014658944039652374, 0.1901487760478537, -0.062277599937482414, -0.007200176110855393, -0.004470969180776416, 0.03881251485170444, 0.16634473569669392, 0.05983829689939807, 0.011671694642670403, -0.10804600772629569, -0.07540282832184109, 0.2857538504201073, 0.15279402273971762, -0.024664823656102074, 0.12731166403579336, 0.1947120810348472, -0.028307145716484277, 0.06888516748837092, 0.17783748048896905, 0.026120001840697724, -0.015204306777417189, 0.008471900971207374, -0.026206111003459567, 0.010775863914004735, 0.19487202985059635, -0.016507916099094796, -0.055557459685007075, -0.057209739649210466, -0.11218193722061576, 0.029805313523122313, -0.1065863272232113, -0.08330955305608016, 0.03316243489443691, -0.019325558442307794, -0.04837661706970361, 0.09643267466413558, 0.00786117893876163, -0.058144775662189195, 0.11348304952979528, -0.0032680047004895124, -0.1745777214391043, 0.011427447961819959, -0.1519733282661166, -0.01888799708568301, -0.07323580122486052, 0.01441505373991657, 0.04518202826695152, -0.025466758357396355, 0.04803925111285688, -0.051941137401247575, -0.020892819999108126, 0.14653473889803664, 0.04455734540373321, -0.1675771165929485, 0.053110850481084944, -0.024463548161114342, 0.18210637498574445, 0.003971802261605996, -0.009903969773904104, -0.11570853674643927, 0.020817541088176423, 0.01948573246350404, 0.19150853963363634, -0.08199684528868648, 0.12321671376340591, 0.018059020151462536, 0.21634370436813824, -0.03742844609898743, -0.05751499880981004, -0.22417807744477877, 0.002472851798058729, -0.086946444155597, 0.03224874819164571, -0.06051749249184176, -0.13714793522365673, -0.06010185759970297, 0.05764780370165544, 0.03395588419210551, -0.11177203552155988, -0.16776254499806662, -0.0035001986892784624, 0.02351230331493369, -0.10407635100654794, -0.08871224253359641, -0.13540186498561801, -0.08283433604173839, -0.1953939320175951, 0.0047677133815967054, 0.012073235829446733, -0.04680098224634797, 0.1477376290885287, 0.15507609362924185, 0.03648282547667811, 0.05897755921126966, -0.0638836508129583, -0.08614131909200733, -0.11672646375349023, 0.010102653197668173, -0.27033339677552776, -0.12588057195847388, 0.014628228052378968, -0.025848504456287013, 0.05173065078661172, 0.0030593351364941136, 0.20395992573417987, -0.061167729422237006, 0.00678727279993166, 0.0752314738697321, -0.07245492869785705, -0.03970064164374175, 0.12568610951851814, -0.014539254203334795, 0.010823371319549753, 0.08653453514660125, 0.015923351857072988, -0.09346803627961409, -0.16224733265368882, 0.04059583307993064, -0.09467244820854402, -0.07718502186741089, 0.018263918413359625, -0.026577754234597697, -0.15235047945904062, -0.17511669117729925, -0.06676397433188795, 0.032475974586266804, 0.07719031356578883, 0.21639379137471243, 0.1964777240690893, -0.03660965128393554, 0.08516225066263002, -0.06405030509175019, 0.03216046098567071, 0.03802058297297608, -0.0508475398282934, -0.3455433882056439, 0.1026043455817292, -0.08328327450151615, -0.11935861781529905, -0.1548875082688949, -0.0014651654132553798, 0.18931297135635453, -0.01344922098564219, -0.002777386898526376, -0.018112600507214017, -0.001158348524542727, 0.04619579744783754, -0.09410422843823313, -0.04007906875816796, 0.011276594985549627, 0.04549580612294122, 0.061993556125453, -0.02833853958397904, 0.024460422696424747, -0.0387410608666618, 0.1369208873035577, -0.0555406183732259, -0.006198377392476996, -0.09014641922540888, 0.060409740454332075, -0.007161188512480392, -0.2298592201706897, 0.007073481151145442, 0.03947656395739011, 0.029331804651466373, 0.0365990782476425, 0.03948080705278996, -0.11643682423400942, -0.08968654401136447, -0.04496188411014935, 0.14663337515857347, -0.06436055511023203, -0.16471077276639873, -0.048544993353824666, 0.012790054616845507, 0.23174908248205867, -0.05254456409308958, -0.23866481113633353, -0.1432566910132217, 0.022149101883268265, 0.000905369448290899, 0.11755575733660877, -0.03931517013024135, -0.24385057672045923, -0.004431477567829326, 0.13742726668868763, 0.09465284348328051, -0.03204892152272463, 0.0036756600820333513, -0.07351175396346585, -0.07578099314801752, -0.03437202065777709, -0.042742364459853535, 0.13529306944297811, -0.11433352303273016, -0.11290417966994434, 0.14620524428740891, 0.06327473147875175, -0.16750846506774, -0.07856862062387512, 0.14903041214079696, -0.044864298428578815, 0.00025429523252865993, 0.1833268885881132, -0.005344686392154049, 0.22589869685174785, -0.05347595124916879, 0.18658966022279375, -0.17094993802865976, -0.02684637871580328, 0.0425569995735274, 0.007519715447475951, 0.028838947981474537, 0.008964848885918388, 0.024905477916721364, -0.11683263599791402, 0.024371951547166357, -0.03442167315830652, -0.025407860057062314, 0.1372102760519073, -0.02428032709697692, -0.16293344412902155, -0.02424987279056649, 0.015665425061181012, -0.10558182325341978, -0.061154151581893246, -0.19341164473191694, -0.04455722880304948, -0.14379692679556652, 0.15966221689561402, 0.01029351613083851, -0.0004829013451848932, 0.03701202079707525, -0.15768673042733558, -0.015176680339178846, -0.043531336586499526, -0.0033128077441815225, -0.07938333360425416, -0.0017285322494284812, 0.02614167026384495, -0.06496714893482716, 0.15334790217249347, -0.025729005076505388, 0.040875192491164516, -0.011795561073262897, -0.029649441605550372, 0.035729099511696866, -0.03580554990190198, -0.017374241582903505, -0.027412100824162185, 0.02282827899288766, 0.12670198143015032, 0.243820122584994, 0.18088524873831413, 0.005209586828293851, 0.034339218911661175, 0.01418809323980932, -0.10024999032402043, -0.0685556476982118, 0.057707914518509804, -0.07542188530643758, 0.03578213420739907, -0.13330335557103573, -0.15584172949312677, 0.013359157278231667, -0.05037433741039347, 0.03404028710354513, 0.028550897685514023, -0.07841376428753756, 0.10040308911060017, 0.07585981623069939, -0.005840414889659092, 0.10905302088009562, -0.0722303007781312, -0.03243356262416488, -0.19085958676659542, 0.10566207447240861, -0.021054736907939645, -0.013948786261425348, 0.16371394125764618, -0.08570027962679784, 0.032642091313840514, 0.1394106060597023, 0.033824567365787345, 0.1317560984969776, -0.06204340707836662, -0.2420654324266105, -0.060942645601705446, -0.11463643162747683, -0.04989058057221298, -0.0793552698534253, 0.18326994602575403, -0.026703866895472595, 0.019101842863749628, 0.06282938068922142, 0.11893233011845412, -0.03419621358481851, -0.07011459677538985, 0.0872158205359374, 0.001807211368506603, -0.011708505613155193, 0.028567181005102544, -0.12530214508671506, -0.02617213908991918, -0.061464613864846813, 0.14376457563922904, 0.0016054695988704883, -0.25160129573285334, -0.0065386552877766966, -0.007052952925927401, -0.00879678560170639, -0.08132110091987255, -0.09640993873120773, 0.050964613851024916, -0.08296074676982702, -0.16298637265849725, -0.05357923577104808, 0.028789075944045023, -0.00940787385026004, -0.06208045908715296, -0.08760765908065726, 0.11660195742192898, -0.016874323759460904, 0.0019492808926547477, -0.059873466588988414, 0.04928145626672552, 0.008579056671952063, -0.14245267054606356, 0.07012587453151618, 0.022615639062357057, 0.1595597727066141, 0.07911862001008896, 0.009386933023747194, 0.04228655628283491, -0.09750598886282204, 0.02884710931484287, -0.0555474000428475, -0.10038626246881835, 0.028265327593270523, 0.0062770702650800266, 0.23698744844796815, 0.05059365351539855, -0.06594978365992547, -0.023713301374848377, 0.0047698723542307095, 0.016091381802215317, 0.01646597207878527, 0.1310626331057727, -0.040694458230193395, -0.02568351384820229, 0.14560865210695828, -0.006843099737799093, -0.009079147830569007, -0.1275298620167284, 0.03733034172578586, 0.17101530675210141, 0.14438462841104596, -0.0006643652988845768, 0.1029719388988532, 0.0031448596865835386, -0.03202409485503402, 0.07273849338926973, 0.1575511959776825, -0.07190798756437483, -0.04299879437570668, -0.05320513382886358, 0.013330120259597756, -0.14318731644462268, 0.05408250522980519, 0.008695578295890278, 0.20451079958687385, 0.04834692891208126, -0.05076101857665506, 0.052460669436182474, 0.11222675707774675, -0.09535286205585831, -0.003383128200356728, -0.093787642083726, 0.020744423610779706, 0.011664114814221535, -0.0725435315564923, 0.058087548370589326, 0.23470815042129894, -0.10130377404922225, -0.046426533456370454, -0.06547634149773003, 0.13556885642835062, 0.010839120409357807, 0.029677627867807088, -0.0627344123474153, 0.006116378161608407, 0.004971277497760856, 0.03338313711503637, 0.037196236112285695, 0.047824625792326346, 0.026047553102552648, -0.032771680006345213, 0.03305245942726851, 0.1449449535921959, -0.014836488269314043, -0.1853316113989809, -0.17660451809015384, -0.08292615099779062, -0.06999767772082095, -0.07952378650671796, -0.06705587320968086, 0.20818197002016606, 0.08469699507892252, 0.14675351717383706, -0.03268419976424619, -0.09277229452683862, 0.019401430512088914, -0.03148741092964721, -0.17010441941754065, -0.1527304632142332, 0.03767716289102875, 0.004173782884341022, -0.022922425650551085, 0.017086543820955667, 0.07875090756733294, 0.025908355006714612, 0.023137415163794573, 0.20821283333054968, -0.03128180582773973, 0.005077646298970699, 0.01096099834830665, -0.024059944451297464, 0.040882233652988764, 0.168137309205766, -0.13216418927399726, -0.016890753484450485, 0.06923507505203483, -0.12840959651192763, 0.021591179318900464, 0.011918712027575296, -0.19469149985283166, -0.031237671382472184, 0.06216212433692215, 0.01566151177999425, -0.11211926376399292, 0.16723901555182735, 0.01443392353215847, -0.06025515743402791, -0.01374627279334167, 0.03896840172123074, 0.0350753678084592, 0.17301543485083232, -0.053935778400766014, -0.09039893092022491, 0.05140229733062117, 0.13693529204210597, 0.0005356111074688652, -0.022351068739215604, 0.0392582308113438, -0.0641665150818051, 0.22233182426298118, 0.033418497354158895, -0.18915292443017379, -0.03455022038079516, -0.06870314076678306, -0.011482950707571155, 0.035271505645140956, -0.05674311843483915, -0.05312229930629163, -0.09212842361475117, 0.17815461942621708, -0.08806946237185118, 0.01680052666914898, -0.05112833815215247, -0.006293809222502238, -0.07241013795610658, -0.04967137411680025, -0.06940417088746785, 0.08549447815655667, -0.04959586069721056, -0.01718705994297403, -0.01650357536508402, -0.26602976897636504, 0.1771580011782472, 0.032689792314835486, -0.034623744825355136, 0.17690066671782223, -0.08983910622021743, -0.27503580295682245, 0.019657381210261622, 0.016116788184409977, -0.006214934687524506, -0.07161734253422446, 0.04431812283114894, 0.0984465730056957, -0.023434143323147864, 0.13105587384067774, 0.03709455755856962, -0.026157382485369366, 0.1567011683805306, 0.030458080574691244, 0.014988988791188141, -0.13574396198501684, -0.23969445564959962, -0.09866049145542301, 0.02589653166510197, -0.21059894641424765, -0.03380148753280656, 0.049285311813085694, -0.02662491194955105, 0.03134008226640742, -0.0781266641246317, -0.03952203394035628, 0.11969788357156984, 0.015002469365075351, 0.011091908091683136, -0.006646633500605785, 0.12312995505166921, 0.015187327772332684, -0.052465393883169474, -0.01419890244836437, 0.018557336379763104, 0.012219973523434948, -0.054770797515679974, -0.1288704238589212, -0.024824031922095254, -0.018463854645319622, 0.02803671703050375, 0.07131227664603051, -0.06041143195984905, -0.02317816140109172, 0.16811772661290925, 0.13632038275639505, 0.14817524099248638, -0.09053287633403041, -0.02874959255534776, 0.10172018722994994, 0.03802855083424715, -0.12109656439206119, 0.007920649918670216, -0.09022421705537144, 0.014463857596576684, -0.021743635877304372, 0.025618114923462644, -0.0674249179232146, -0.008976175042017401, 0.03474654975033881, -0.015253826512660008, 0.09741217301484853, -0.2600829124926817, 0.06491629123839851, -0.009237763446928313, 0.03692935806902428, 0.02121335508897271, 0.031693394662349196, 0.09525351732389942, -0.0335562894098584, -0.10661277152213468, -0.08274147505653753, -0.23322159841341264, 0.011476574544202365, -0.0146009661014556, 0.01249993451380248, 0.007275884548562811, 0.03300339174071623, -0.025792520579677486, 0.021332675542419053, -0.03094463281345774, -0.015757689048329155, -0.07295564053072232, 0.04755739558814541, 0.011688387127955568, -0.1476529937006964, 0.017158716161693753, -0.1720489173047968, 0.061688796604196204, -0.07369375900863889, -0.17308760124306044, 0.1052343390182927, 0.09922805397429926, -0.0899457872127109, 0.11047423558950192, -0.0766899028529632, -0.017236825816223354, -0.014858343108302157, 0.04619432750374564, -0.104890488005848, 0.035786113871595956, -0.09090111120914876, -0.10715874085830737, 0.029019835264622755, 0.17052074070254847, -0.03794492767156051, -0.038484257780252225, -0.01836689240292592, 0.17935619272220107, -0.015209576786303537, -0.05701885132633687, -0.08719867289663191, 0.004983532622959111, -0.022150873834496175, -0.09074857000087079, 0.14140100387977672, -0.00788588851149213, -0.09915780278154024, -0.009997002193983005, -0.01152747866127027, 0.2002817998015115, -0.050847652931993184, -0.04392798612237046, -0.037427206543397734, 0.03794615423199551, -0.022917358007127003, 0.09162909265560423, -0.1977351046409984, -0.0033816650756694636, -0.14658268709054703, -0.11957506212734365, -0.16957014112816637, -0.018786756386627083, 0.04453904148327922, 0.04388112093477036, 0.0428861094195463, -0.06286792656536076], \"yaxis\": \"y\"}],\n",
              "                        {\"coloraxis\": {\"colorbar\": {\"title\": {\"text\": \"label\"}}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"legend\": {\"tracegroupgap\": 0}, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}, \"title\": {\"text\": \"\\u5546\\u54c1\\u8a55\\u8ad6\\u8a13\\u7df4\\u96c6PCA\"}, \"xaxis\": {\"anchor\": \"y\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"x\"}}, \"yaxis\": {\"anchor\": \"x\", \"domain\": [0.0, 1.0], \"title\": {\"text\": \"y\"}}},\n",
              "                        {\"responsive\": true}\n",
              "                    ).then(function(){\n",
              "                            \n",
              "var gd = document.getElementById('e9c5ad4e-e3a2-41a4-8074-fabcf5b7c6ec');\n",
              "var x = new MutationObserver(function (mutations, observer) {{\n",
              "        var display = window.getComputedStyle(gd).display;\n",
              "        if (!display || display === 'none') {{\n",
              "            console.log([gd, 'removed!']);\n",
              "            Plotly.purge(gd);\n",
              "            observer.disconnect();\n",
              "        }}\n",
              "}});\n",
              "\n",
              "// Listen for the removal of the full notebook cells\n",
              "var notebookContainer = gd.closest('#notebook-container');\n",
              "if (notebookContainer) {{\n",
              "    x.observe(notebookContainer, {childList: true});\n",
              "}}\n",
              "\n",
              "// Listen for the clearing of the current output cell\n",
              "var outputEl = gd.closest('.output');\n",
              "if (outputEl) {{\n",
              "    x.observe(outputEl, {childList: true});\n",
              "}}\n",
              "\n",
              "                        })\n",
              "                };\n",
              "                \n",
              "            </script>\n",
              "        </div>\n",
              "</body>\n",
              "</html>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "hero.scatterplot(train, col='pca_from_tfidf', color='label', title=\"商品評論訓練集PCA\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B8ZU0ELGM132",
        "outputId": "812d13fd-0e96-4555-f908-1b36cb7f4d2d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "cat    \n",
              "平板   個     244\n",
              "     買     207\n",
              "     不錯    181\n",
              "     還     153\n",
              "     這     140\n",
              "     京東    132\n",
              "     沒     131\n",
              "     說     129\n",
              "     手機    121\n",
              "     給     111\n",
              "水果   個     403\n",
              "     蘋果    334\n",
              "     買     265\n",
              "     次     222\n",
              "     京東    210\n",
              "     這     183\n",
              "     還     178\n",
              "     不錯    144\n",
              "     吃     111\n",
              "     味道    101\n",
              "洗髮水  買     264\n",
              "     還     209\n",
              "     京東    181\n",
              "     這     177\n",
              "     個     170\n",
              "     次     154\n",
              "     洗髮    103\n",
              "     說      95\n",
              "     不錯     85\n",
              "     沒      84\n",
              "衣服   質量    246\n",
              "     不錯    219\n",
              "     褲子    211\n",
              "     次     150\n",
              "     還     149\n",
              "     買     144\n",
              "     這     136\n",
              "     舒服    119\n",
              "     個     114\n",
              "     衣服     89\n",
              "酒店   酒店    844\n",
              "     房間    582\n",
              "     個     400\n",
              "     服務    351\n",
              "     還     298\n",
              "     這     277\n",
              "     不錯    268\n",
              "     沒有    266\n",
              "     說     258\n",
              "     入住    213\n",
              "Name: tokens, dtype: int64"
            ]
          },
          "execution_count": 184,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "NUM_TOP_WORDS = 10\n",
        "train.groupby('cat')['tokens'].apply(lambda x: hero.top_words(x)[:NUM_TOP_WORDS])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8xImkr2Rpzoa"
      },
      "source": [
        "### word2vec"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9VrjxDV1Ounm"
      },
      "source": [
        "分佈式語意基本假設\n",
        "\n",
        "![](https://slideplayer.com/slide/12147948/71/images/10/John+Rupert+Firth+You+shall+know+a+word+by+the+company+it+keeps.jpg)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T452ZOBJTyy-"
      },
      "source": [
        "- 藍色目標詞\n",
        "- 紅色語境詞\n",
        "- 自監督學習\n",
        "\n",
        "![](https://twice22.github.io/images/word2vec/cbow-example.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "HX783PuLmahX",
        "outputId": "5aee0091-5c8e-4496-e3f8-4eed2b97d431"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: spacy in /usr/local/lib/python3.7/dist-packages (2.3.7)\n",
            "Collecting spacy\n",
            "  Using cached spacy-3.2.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.0 MB)\n",
            "Collecting thinc<8.1.0,>=8.0.12\n",
            "  Using cached thinc-8.0.13-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (628 kB)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy) (59.5.0)\n",
            "Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy) (3.10.0.2)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (2.23.0)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy) (0.6.1)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (0.4.1)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy) (2.0.6)\n",
            "Requirement already satisfied: typer<0.5.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (0.4.0)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (1.0.6)\n",
            "Collecting catalogue<2.1.0,>=2.0.6\n",
            "  Using cached catalogue-2.0.6-py3-none-any.whl (17 kB)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from spacy) (0.8.2)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (1.0.1)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (1.19.5)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy) (3.0.6)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy) (2.11.3)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (21.3)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (4.62.3)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy) (1.8.2)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /usr/local/lib/python3.7/dist-packages (from spacy) (3.0.8)\n",
            "Collecting srsly<3.0.0,>=2.4.1\n",
            "  Using cached srsly-2.4.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (451 kB)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy) (3.3.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy) (3.6.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->spacy) (3.0.6)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy) (5.2.1)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy) (2.10)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy) (1.24.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy) (2021.10.8)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy) (3.0.4)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer<0.5.0,>=0.3.0->spacy) (7.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy) (2.0.1)\n",
            "Installing collected packages: catalogue, srsly, thinc, spacy\n",
            "  Attempting uninstall: catalogue\n",
            "    Found existing installation: catalogue 1.0.0\n",
            "    Uninstalling catalogue-1.0.0:\n",
            "      Successfully uninstalled catalogue-1.0.0\n",
            "  Attempting uninstall: srsly\n",
            "    Found existing installation: srsly 1.0.5\n",
            "    Uninstalling srsly-1.0.5:\n",
            "      Successfully uninstalled srsly-1.0.5\n",
            "  Attempting uninstall: thinc\n",
            "    Found existing installation: thinc 7.4.5\n",
            "    Uninstalling thinc-7.4.5:\n",
            "      Successfully uninstalled thinc-7.4.5\n",
            "  Attempting uninstall: spacy\n",
            "    Found existing installation: spacy 2.3.7\n",
            "    Uninstalling spacy-2.3.7:\n",
            "      Successfully uninstalled spacy-2.3.7\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "texthero 1.1.0 requires spacy<3.0.0, but you have spacy 3.2.0 which is incompatible.\u001b[0m\n",
            "Successfully installed catalogue-2.0.6 spacy-3.2.0 srsly-2.4.2 thinc-8.0.13\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n"
          ]
        },
        {
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "catalogue",
                  "spacy",
                  "srsly",
                  "thinc"
                ]
              }
            }
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting zh-core-web-md==3.2.0\n",
            "  Downloading https://github.com/explosion/spacy-models/releases/download/zh_core_web_md-3.2.0/zh_core_web_md-3.2.0-py3-none-any.whl (79.0 MB)\n",
            "     |████████████████████████████████| 79.0 MB 70 kB/s             \n",
            "\u001b[?25hRequirement already satisfied: spacy-pkuseg<0.1.0,>=0.0.27 in /usr/local/lib/python3.7/dist-packages (from zh-core-web-md==3.2.0) (0.0.28)\n",
            "Requirement already satisfied: spacy<3.3.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from zh-core-web-md==3.2.0) (3.2.0)\n",
            "Requirement already satisfied: thinc<8.1.0,>=8.0.12 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (8.0.13)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (0.4.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.11.3)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (21.3)\n",
            "Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.10.0.2)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.0.1)\n",
            "Requirement already satisfied: typer<0.5.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (0.4.0)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.23.0)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (4.62.3)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.0.8)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.0.6)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.0.6)\n",
            "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.0.6)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.0.6)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (0.6.1)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.8.2)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.19.5)\n",
            "Requirement already satisfied: srsly<3.0.0,>=2.4.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.4.2)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (0.8.2)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.3.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (59.5.0)\n",
            "Requirement already satisfied: cython>=0.25 in /usr/local/lib/python3.7/dist-packages (from spacy-pkuseg<0.1.0,>=0.0.27->zh-core-web-md==3.2.0) (0.29.24)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.6.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.0.6)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (5.2.1)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (3.0.4)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2021.10.8)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer<0.5.0,>=0.3.0->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (7.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy<3.3.0,>=3.2.0->zh-core-web-md==3.2.0) (2.0.1)\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
            "\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
            "You can now load the package via spacy.load('zh_core_web_md')\n"
          ]
        }
      ],
      "source": [
        "!pip install -U spacy\n",
        "!python -m spacy download zh_core_web_md"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gGhwQvr-WDQB",
        "outputId": "366b8fd8-ea68-4b6d-d523-4e097ea7aa22"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Name: spacy\n",
            "Version: 3.2.0\n",
            "Summary: Industrial-strength Natural Language Processing (NLP) in Python\n",
            "Home-page: https://spacy.io\n",
            "Author: Explosion\n",
            "Author-email: contact@explosion.ai\n",
            "License: MIT\n",
            "Location: /usr/local/lib/python3.7/dist-packages\n",
            "Requires: blis, catalogue, cymem, jinja2, langcodes, murmurhash, numpy, packaging, pathy, preshed, pydantic, requests, setuptools, spacy-legacy, spacy-loggers, srsly, thinc, tqdm, typer, typing-extensions, wasabi\n",
            "Required-by: en-core-web-sm, fastai, texthero, zh-core-web-md\n"
          ]
        }
      ],
      "source": [
        "!pip show spacy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "52OiKCh1bVxI",
        "outputId": "6b561f97-45ec-4577-f429-19279188f19c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "!pip install -q whatlies"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4Frv8hiGbexz"
      },
      "outputs": [],
      "source": [
        "from whatlies import EmbeddingSet\n",
        "from whatlies.language import SpacyLanguage\n",
        "\n",
        "lang = SpacyLanguage('zh_core_web_md')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 385
        },
        "id": "9-Jadg3tcIwk",
        "outputId": "485940f0-d0b1-415d-973d-de27891ba1d3"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "<div id=\"altair-viz-a6554402d3c6431fb9ce919b7519d796\"></div>\n",
              "<script type=\"text/javascript\">\n",
              "  (function(spec, embedOpt){\n",
              "    let outputDiv = document.currentScript.previousElementSibling;\n",
              "    if (outputDiv.id !== \"altair-viz-a6554402d3c6431fb9ce919b7519d796\") {\n",
              "      outputDiv = document.getElementById(\"altair-viz-a6554402d3c6431fb9ce919b7519d796\");\n",
              "    }\n",
              "    const paths = {\n",
              "      \"vega\": \"https://cdn.jsdelivr.net/npm//vega@5?noext\",\n",
              "      \"vega-lib\": \"https://cdn.jsdelivr.net/npm//vega-lib?noext\",\n",
              "      \"vega-lite\": \"https://cdn.jsdelivr.net/npm//vega-lite@4.8.1?noext\",\n",
              "      \"vega-embed\": \"https://cdn.jsdelivr.net/npm//vega-embed@6?noext\",\n",
              "    };\n",
              "\n",
              "    function loadScript(lib) {\n",
              "      return new Promise(function(resolve, reject) {\n",
              "        var s = document.createElement('script');\n",
              "        s.src = paths[lib];\n",
              "        s.async = true;\n",
              "        s.onload = () => resolve(paths[lib]);\n",
              "        s.onerror = () => reject(`Error loading script: ${paths[lib]}`);\n",
              "        document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
              "      });\n",
              "    }\n",
              "\n",
              "    function showError(err) {\n",
              "      outputDiv.innerHTML = `<div class=\"error\" style=\"color:red;\">${err}</div>`;\n",
              "      throw err;\n",
              "    }\n",
              "\n",
              "    function displayChart(vegaEmbed) {\n",
              "      vegaEmbed(outputDiv, spec, embedOpt)\n",
              "        .catch(err => showError(`Javascript Error: ${err.message}<br>This usually means there's a typo in your chart specification. See the javascript console for the full traceback.`));\n",
              "    }\n",
              "\n",
              "    if(typeof define === \"function\" && define.amd) {\n",
              "      requirejs.config({paths});\n",
              "      require([\"vega-embed\"], displayChart, err => showError(`Error loading script: ${err.message}`));\n",
              "    } else if (typeof vegaEmbed === \"function\") {\n",
              "      displayChart(vegaEmbed);\n",
              "    } else {\n",
              "      loadScript(\"vega\")\n",
              "        .then(() => loadScript(\"vega-lite\"))\n",
              "        .then(() => loadScript(\"vega-embed\"))\n",
              "        .catch(showError)\n",
              "        .then(() => displayChart(vegaEmbed));\n",
              "    }\n",
              "  })({\"config\": {\"view\": {\"continuousWidth\": 400, \"continuousHeight\": 300}}, \"layer\": [{\"mark\": {\"type\": \"circle\", \"size\": 60}, \"encoding\": {\"color\": {\"type\": \"nominal\", \"field\": \"\", \"legend\": null}, \"tooltip\": [{\"type\": \"nominal\", \"field\": \"name\"}, {\"type\": \"nominal\", \"field\": \"original\"}], \"x\": {\"type\": \"quantitative\", \"axis\": {\"title\": \"\\u91ab\\u9662\"}, \"field\": \"x_axis\"}, \"y\": {\"type\": \"quantitative\", \"axis\": {\"title\": \"\\u5b78\\u6821\"}, \"field\": \"y_axis\"}}, \"selection\": {\"selector002\": {\"type\": \"interval\", \"bind\": \"scales\", \"encodings\": [\"x\", \"y\"]}}, \"title\": \"\\u91ab\\u9662 vs. \\u5b78\\u6821\"}, {\"mark\": {\"type\": \"text\", \"color\": \"black\", \"dx\": -15, \"dy\": 3}, \"encoding\": {\"text\": {\"type\": \"nominal\", \"field\": \"original\"}, \"x\": {\"type\": \"quantitative\", \"field\": \"x_axis\"}, \"y\": {\"type\": \"quantitative\", \"field\": \"y_axis\"}}}], \"data\": {\"name\": \"data-246c64c09bfdb097904cca7f82c1cf1f\"}, \"$schema\": \"https://vega.github.io/schema/vega-lite/v4.8.1.json\", \"datasets\": {\"data-246c64c09bfdb097904cca7f82c1cf1f\": [{\"x_axis\": 0.5985222458839417, \"y_axis\": 0.41838929057121277, \"name\": \"\\u91ab\\u751f\", \"original\": \"\\u91ab\\u751f\"}, {\"x_axis\": 0.43434667587280273, \"y_axis\": 0.35816940665245056, \"name\": \"\\u8b77\\u58eb\", \"original\": \"\\u8b77\\u58eb\"}, {\"x_axis\": 1.0, \"y_axis\": 0.5983630418777466, \"name\": \"\\u91ab\\u9662\", \"original\": \"\\u91ab\\u9662\"}, {\"x_axis\": 0.1430959850549698, \"y_axis\": 0.12564389407634735, \"name\": \"\\u6559\\u6388\", \"original\": \"\\u6559\\u6388\"}, {\"x_axis\": 0.4145154654979706, \"y_axis\": 0.7847005128860474, \"name\": \"\\u5b78\\u751f\", \"original\": \"\\u5b78\\u751f\"}, {\"x_axis\": 0.5500433444976807, \"y_axis\": 1.0, \"name\": \"\\u5b78\\u6821\", \"original\": \"\\u5b78\\u6821\"}]}}, {\"mode\": \"vega-lite\"});\n",
              "</script>"
            ],
            "text/plain": [
              "alt.LayerChart(...)"
            ]
          },
          "execution_count": 189,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "words = ['醫生', '護士', '醫院', '教授', '學生', '學校']\n",
        "emb = lang[words]\n",
        "emb.plot_interactive(x_axis='醫院', y_axis='學校')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oOVU5VCCq1f1"
      },
      "source": [
        "餘弦相似性\n",
        "\n",
        "![](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQHFjinj2_lYY-tQ9rTzOiKpwovrfzJ60QXPg&usqp=CAU)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9ZzgHzxmL2Ha"
      },
      "source": [
        "[向量投影視覺化](https://projector.tensorflow.org/)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mYI4Y01bOksK"
      },
      "source": [
        "### fasttext\n",
        "- 字元等級Ngram\n",
        "- 合成未登錄詞(OOV)的向量\n",
        "\n",
        "![](https://amitness.com/images/fasttext-center-word-embedding.png)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z5r-i8MNp8hH",
        "outputId": "c5403928-b6a2-4700-9530-4f56281a8f5b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "!pip install -q fasttext"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VuRX2Bi5oojr"
      },
      "outputs": [],
      "source": [
        "import fasttext"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DIYMQ89SqCMk"
      },
      "outputs": [],
      "source": [
        "texts = downsized_df['tokens'].to_list()\n",
        "\n",
        "with open('corpus.txt', 'w') as f:\n",
        "    for item in texts:\n",
        "        f.write(f\"{item}\\n\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bFgscYFEqGJM",
        "outputId": "b6822e15-570a-4a6a-fb1d-e448e0ad4e84"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時4.876421213150024秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "model = fasttext.train_unsupervised('corpus.txt')\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YcidT-VqqJnF",
        "outputId": "fbf6484a-bae5-46ff-b4c2-4123313d66e0"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "2253"
            ]
          },
          "execution_count": 194,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "len(model.words)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ptBdgvpnqOFD",
        "outputId": "5d6e9696-0c0f-427e-d000-3813142f2655"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['</s>',\n",
              " '個',\n",
              " '還',\n",
              " '不錯',\n",
              " '這',\n",
              " '買',\n",
              " '酒店',\n",
              " '次',\n",
              " '說',\n",
              " '沒有',\n",
              " '房間',\n",
              " '京東',\n",
              " '沒',\n",
              " '服務',\n",
              " '差',\n",
              " '太',\n",
              " '給',\n",
              " '蘋果',\n",
              " '感覺',\n",
              " '質量']"
            ]
          },
          "execution_count": 195,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.words[:20]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5tNi1mOSqTTo",
        "outputId": "c4772673-0255-4d88-cee7-b5ff36e92184"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[(0.9984833598136902, '購物'),\n",
              " (0.9978212714195251, '值得'),\n",
              " (0.9977539777755737, '信賴'),\n",
              " (0.9965359568595886, '購買'),\n",
              " (0.9963666200637817, '賣家'),\n",
              " (0.9957764744758606, '實惠'),\n",
              " (0.995775580406189, '贊'),\n",
              " (0.9956874251365662, '描述'),\n",
              " (0.9956346154212952, '速度'),\n",
              " (0.9950734972953796, '手感')]"
            ]
          },
          "execution_count": 196,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.get_nearest_neighbors('好評')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZQy9_8J-qbcZ",
        "outputId": "60ac455b-7eaa-4c5d-c856-58b88f810a24"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[(0.997538149356842, '怎麼'),\n",
              " (0.994895339012146, '買到'),\n",
              " (0.99481201171875, '商品'),\n",
              " (0.9948036074638367, '貨'),\n",
              " (0.994503378868103, '爛'),\n",
              " (0.9943825006484985, '商家'),\n",
              " (0.9942765235900879, '用了'),\n",
              " (0.9937253594398499, '這麼'),\n",
              " (0.9934755563735962, '活動'),\n",
              " (0.9928011298179626, '下單')]"
            ]
          },
          "execution_count": 197,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.get_nearest_neighbors('差評')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GSMkhmy0O7sD"
      },
      "source": [
        "### transformer\n",
        "\n",
        "- GPT\n",
        "- BERT\n",
        "\n",
        "![](https://raw.githubusercontent.com/jessevig/bertviz/master/images/head-view.gif)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2WQmsVkltTfc"
      },
      "source": [
        "# 遷移學習\n",
        "\n",
        "![](https://ruder.io/content/images/2019/08/pretraining_adaptation.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qzTKcH1z1R6D"
      },
      "source": [
        "## 遷移學習的類別\n",
        "\n",
        "![](https://ruder.io/content/images/2019/08/transfer_learning_taxonomy.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Cj8uYd4JqEjS"
      },
      "source": [
        "## spaCy的遷移學習架構\n",
        "\n",
        "![](https://spacy.io/tok2vec-listener-8c4d53807708b270c07a085f4a2da75f.svg)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iPEOKDWxlfK8"
      },
      "source": [
        "# 文本表徵的具體應用"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KnPkbsmIOWtx"
      },
      "source": [
        "## 文本相似性"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GXcHHAGOgp-A"
      },
      "outputs": [],
      "source": [
        "nlp = spacy.load(\"zh_core_web_md\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "__vUOwVVf9D2",
        "outputId": "4b2e1263-a6ef-4d3c-b2b1-104d7d1fbd05"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0.6026955369516264\n",
            "0.35929124359235026\n"
          ]
        }
      ],
      "source": [
        "doc01 = nlp(\"防Omicron威脅 一至三類今起可打第3劑\")\n",
        "doc02 = nlp(\"第3劑開打…賣場設站隨到隨打 首波4.8萬人符資格\")\n",
        "doc03 = nlp(\"西半部低溫特報 全台平地最冷在基隆僅8度\")\n",
        "\n",
        "sim01 = doc01.similarity(doc02)\n",
        "sim02 = doc01.similarity(doc03)\n",
        "\n",
        "print(sim01)\n",
        "print(sim02)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AQrx7oKAMPon",
        "outputId": "2b966a7c-3d90-4707-abcc-3ca7c78120e1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "!pip install -q faiss-cpu"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_eVx5XTvMSJZ"
      },
      "outputs": [],
      "source": [
        "import faiss"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KKFoYtZBM5Tp"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "\n",
        "def create_index_embeddings(doc_vectors, index_series):\n",
        "    # Step 1: Change data type\n",
        "    embeddings = np.array([embedding for embedding in doc_vectors]).astype(\"float32\")\n",
        "\n",
        "    # Step 2: Instantiate the index\n",
        "    index = faiss.IndexFlatL2(embeddings.shape[1]) \n",
        "\n",
        "    # Step 3: Pass the index to IndexIDMap\n",
        "    index = faiss.IndexIDMap(index)\n",
        "\n",
        "    # Step 4: Add vectors and their IDs\n",
        "    index.add_with_ids(embeddings, index_series)\n",
        "\n",
        "    return index, embeddings"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4jWRvSNtMjnn"
      },
      "outputs": [],
      "source": [
        "text_list = downsized_df['review'].to_list()\n",
        "index_list = downsized_df.index.values"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dYv6bsmAMY_v",
        "outputId": "fc0a67f3-061f-4b11-88fa-dff6ff5cc505"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時28.20578145980835秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "doc_vectors = []\n",
        "for doc in nlp.pipe(text_list):\n",
        "    doc_vectors.append(doc.vector)\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v61M0Il801ok",
        "outputId": "94603fdc-9aef-4127-c592-191076e490bf"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時0.014593839645385742秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "index, embeddings = create_index_embeddings(doc_vectors, index_list)\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xagAoKlNEzXi",
        "outputId": "46c50e23-4129-4062-bbf9-b279317cde0f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['標準間的衛生間居然熱水都上不來，後來叫來了服務生和經理折騰了半個小時才弄好。。。 第一次住這麼差的房間！！',\n",
              " '寶貝收到了，質量很好，摸起來很舒服，很不錯的一條褲子哦，值得購買!',\n",
              " '褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡']"
            ]
          },
          "execution_count": 206,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "text_list[-3:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z9Kpa49Y0pvp",
        "outputId": "1b022d3f-f306-43c3-a2d2-d2dadfb38275"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array([4997, 4998, 4999])"
            ]
          },
          "execution_count": 207,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "index_list[-3:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xZ6lD1QFQGYG"
      },
      "outputs": [],
      "source": [
        "D, I = index.search(np.array([embeddings[4999]]), k=5)\n",
        "SNs = I.flatten().tolist()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "2nBfqs1MQiD9",
        "outputId": "eca3b14f-9286-42f5-c04e-1379a294996e"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>review</th>\n",
              "      <th>cat</th>\n",
              "      <th>label</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>4999</th>\n",
              "      <td>褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡</td>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3740</th>\n",
              "      <td>質量很好，做工紮實，大小合適，顏色跟圖片描述一樣沒有色差，穿起來舒服，很滿意。</td>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1837</th>\n",
              "      <td>質量不錯，版型很好，尺碼合適，顏色和店鋪照片上一樣完全不存在色差 穿著很喜歡，發貨迅速，贊一個</td>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1950</th>\n",
              "      <td>褲子很好看，特別喜歡這個顏色，沒有色差，和圖片上一模一樣，客服的態度也很滿意，很愉快的一次購...</td>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2005</th>\n",
              "      <td>褲子很合適，跟圖片描述的一模一樣，質量也非常好，穿起來很舒服。</td>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                                 review cat  label\n",
              "4999                             褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡  衣服      1\n",
              "3740            質量很好，做工紮實，大小合適，顏色跟圖片描述一樣沒有色差，穿起來舒服，很滿意。  衣服      1\n",
              "1837    質量不錯，版型很好，尺碼合適，顏色和店鋪照片上一樣完全不存在色差 穿著很喜歡，發貨迅速，贊一個  衣服      1\n",
              "1950  褲子很好看，特別喜歡這個顏色，沒有色差，和圖片上一模一樣，客服的態度也很滿意，很愉快的一次購...  衣服      1\n",
              "2005                    褲子很合適，跟圖片描述的一模一樣，質量也非常好，穿起來很舒服。  衣服      1"
            ]
          },
          "execution_count": 209,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "downsized_df.loc[SNs, [\"review\", \"cat\", \"label\"]]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-HD7iSDvjK2f"
      },
      "source": [
        "## 文本分類"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sja1AIgm104E"
      },
      "source": [
        "### 使用TF-IDF"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 293
        },
        "id": "bOSQmqz094w7",
        "outputId": "4efca8cf-6b20-4b0e-e185-520a935e3ff7"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>cat</th>\n",
              "      <th>label</th>\n",
              "      <th>review</th>\n",
              "      <th>tokens</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>4995</th>\n",
              "      <td>平板</td>\n",
              "      <td>0</td>\n",
              "      <td>不好，什麼都沒有，裸機，而且總出故障！差差差！</td>\n",
              "      <td>什麼 沒有 裸機 總出 故障 差差 差</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4996</th>\n",
              "      <td>平板</td>\n",
              "      <td>1</td>\n",
              "      <td>很喜歡，主要是不卡，拿出門也很方便</td>\n",
              "      <td>喜歡 卡 出門</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4997</th>\n",
              "      <td>酒店</td>\n",
              "      <td>0</td>\n",
              "      <td>標準間的衛生間居然熱水都上不來，後來叫來了服務生和經理折騰了半個小時才弄好。。。 第一次住這...</td>\n",
              "      <td>標準 間 衛生 間 熱水 來 後來 叫來 服務生 經理 折騰 個 小時 弄好 次 住 這麼 ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4998</th>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "      <td>寶貝收到了，質量很好，摸起來很舒服，很不錯的一條褲子哦，值得購買!</td>\n",
              "      <td>寶貝 收到 質量 摸起來 舒服 不錯 條 褲子 值得 購買</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4999</th>\n",
              "      <td>衣服</td>\n",
              "      <td>1</td>\n",
              "      <td>褲子顏色和圖片一樣，質量很好，尺碼準，很喜歡</td>\n",
              "      <td>褲子 顏色 圖片 一樣 質量 尺碼 準 喜歡</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "     cat  ...                                             tokens\n",
              "4995  平板  ...                                什麼 沒有 裸機 總出 故障 差差 差\n",
              "4996  平板  ...                                            喜歡 卡 出門\n",
              "4997  酒店  ...  標準 間 衛生 間 熱水 來 後來 叫來 服務生 經理 折騰 個 小時 弄好 次 住 這麼 ...\n",
              "4998  衣服  ...                      寶貝 收到 質量 摸起來 舒服 不錯 條 褲子 值得 購買\n",
              "4999  衣服  ...                             褲子 顏色 圖片 一樣 質量 尺碼 準 喜歡\n",
              "\n",
              "[5 rows x 4 columns]"
            ]
          },
          "execution_count": 70,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "downsized_df.tail()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9dtULrFQ92Cd"
      },
      "outputs": [],
      "source": [
        "from sklearn import feature_extraction, model_selection, naive_bayes, pipeline"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5TchZgMy93rN"
      },
      "outputs": [],
      "source": [
        "train_df, test_df = model_selection.train_test_split(downsized_df, test_size=0.3, random_state=100)\n",
        "train_y = train_df[\"cat\"].values\n",
        "test_y = test_df[\"cat\"].values"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "92ySI6uf-sAv"
      },
      "outputs": [],
      "source": [
        "vectorizer = feature_extraction.text.TfidfVectorizer(max_features=10000, ngram_range=(1,2))\n",
        "corpus = train_df[\"tokens\"]\n",
        "vectorizer.fit(corpus)\n",
        "train_X = vectorizer.transform(corpus)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Q_WLJpwqBOvs"
      },
      "outputs": [],
      "source": [
        "classifier = naive_bayes.MultinomialNB()\n",
        "model = pipeline.Pipeline([(\"vectorizer\", vectorizer),  \n",
        "                           (\"classifier\", classifier)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N92oJ_ZuYhaF",
        "outputId": "c87af2c6-cc56-461b-9d5d-5b18dc56246f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時0.048974037170410156秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "model[\"classifier\"].fit(train_X, train_y)\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lTj2-RXSBfIq",
        "outputId": "0790ccef-c3f9-45f9-a14f-82d672616b49"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時0.05185198783874512秒\n"
          ]
        }
      ],
      "source": [
        "test_X = test_df[\"tokens\"].values\n",
        "start = time.time()\n",
        "predicted = model.predict(test_X)\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Nqfs2AdIEt0c"
      },
      "outputs": [],
      "source": [
        "from sklearn import metrics"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "vYRPpKwFBuH_",
        "outputId": "87cda613-6c29-4dff-f99b-6ab43d640d91"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>precision</th>\n",
              "      <th>recall</th>\n",
              "      <th>f1-score</th>\n",
              "      <th>support</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>平板</th>\n",
              "      <td>0.764151</td>\n",
              "      <td>0.801980</td>\n",
              "      <td>0.782609</td>\n",
              "      <td>303.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>水果</th>\n",
              "      <td>0.826087</td>\n",
              "      <td>0.863636</td>\n",
              "      <td>0.844444</td>\n",
              "      <td>308.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>洗髮水</th>\n",
              "      <td>0.841155</td>\n",
              "      <td>0.795222</td>\n",
              "      <td>0.817544</td>\n",
              "      <td>293.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>衣服</th>\n",
              "      <td>0.864583</td>\n",
              "      <td>0.816393</td>\n",
              "      <td>0.839798</td>\n",
              "      <td>305.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>酒店</th>\n",
              "      <td>0.952542</td>\n",
              "      <td>0.965636</td>\n",
              "      <td>0.959044</td>\n",
              "      <td>291.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>accuracy</th>\n",
              "      <td>0.848000</td>\n",
              "      <td>0.848000</td>\n",
              "      <td>0.848000</td>\n",
              "      <td>0.848</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>macro avg</th>\n",
              "      <td>0.849704</td>\n",
              "      <td>0.848574</td>\n",
              "      <td>0.848688</td>\n",
              "      <td>1500.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>weighted avg</th>\n",
              "      <td>0.848879</td>\n",
              "      <td>0.848000</td>\n",
              "      <td>0.847987</td>\n",
              "      <td>1500.000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "              precision    recall  f1-score   support\n",
              "平板             0.764151  0.801980  0.782609   303.000\n",
              "水果             0.826087  0.863636  0.844444   308.000\n",
              "洗髮水            0.841155  0.795222  0.817544   293.000\n",
              "衣服             0.864583  0.816393  0.839798   305.000\n",
              "酒店             0.952542  0.965636  0.959044   291.000\n",
              "accuracy       0.848000  0.848000  0.848000     0.848\n",
              "macro avg      0.849704  0.848574  0.848688  1500.000\n",
              "weighted avg   0.848879  0.848000  0.847987  1500.000"
            ]
          },
          "execution_count": 78,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "class_report = metrics.classification_report(test_y, predicted, output_dict=True)\n",
        "report_df = pd.DataFrame(class_report).transpose()\n",
        "report_df"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Wy_cDfEXbR_a"
      },
      "outputs": [],
      "source": [
        "from sklearn.svm import SVC \n",
        "\n",
        "svc_clf = SVC(C=1, gamma=\"auto\", kernel='linear',probability=False)\n",
        "model = pipeline.Pipeline([(\"vectorizer\", vectorizer),  \n",
        "                           (\"svm\", svc_clf)])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5tyKdUKVas-8",
        "outputId": "e5d0bd94-983a-4b83-970e-fdfb47899954"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時1.3198256492614746秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "model[\"svm\"].fit(train_X, train_y)\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\") "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kp1e-cXGcCoM",
        "outputId": "c6b78413-f860-495f-e505-21307bde387c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時0.4450039863586426秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "predicted = model.predict(test_X)\n",
        "end = time.time()\n",
        "\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "5oZYxDHAceyZ",
        "outputId": "449fd6b8-a047-4c77-c138-021247613a11"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>precision</th>\n",
              "      <th>recall</th>\n",
              "      <th>f1-score</th>\n",
              "      <th>support</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>平板</th>\n",
              "      <td>0.704348</td>\n",
              "      <td>0.801980</td>\n",
              "      <td>0.750000</td>\n",
              "      <td>303.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>水果</th>\n",
              "      <td>0.880000</td>\n",
              "      <td>0.785714</td>\n",
              "      <td>0.830189</td>\n",
              "      <td>308.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>洗髮水</th>\n",
              "      <td>0.764901</td>\n",
              "      <td>0.788396</td>\n",
              "      <td>0.776471</td>\n",
              "      <td>293.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>衣服</th>\n",
              "      <td>0.832215</td>\n",
              "      <td>0.813115</td>\n",
              "      <td>0.822554</td>\n",
              "      <td>305.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>酒店</th>\n",
              "      <td>0.989286</td>\n",
              "      <td>0.951890</td>\n",
              "      <td>0.970228</td>\n",
              "      <td>291.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>accuracy</th>\n",
              "      <td>0.827333</td>\n",
              "      <td>0.827333</td>\n",
              "      <td>0.827333</td>\n",
              "      <td>0.827333</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>macro avg</th>\n",
              "      <td>0.834150</td>\n",
              "      <td>0.828219</td>\n",
              "      <td>0.829888</td>\n",
              "      <td>1500.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>weighted avg</th>\n",
              "      <td>0.833521</td>\n",
              "      <td>0.827333</td>\n",
              "      <td>0.829113</td>\n",
              "      <td>1500.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "              precision    recall  f1-score      support\n",
              "平板             0.704348  0.801980  0.750000   303.000000\n",
              "水果             0.880000  0.785714  0.830189   308.000000\n",
              "洗髮水            0.764901  0.788396  0.776471   293.000000\n",
              "衣服             0.832215  0.813115  0.822554   305.000000\n",
              "酒店             0.989286  0.951890  0.970228   291.000000\n",
              "accuracy       0.827333  0.827333  0.827333     0.827333\n",
              "macro avg      0.834150  0.828219  0.829888  1500.000000\n",
              "weighted avg   0.833521  0.827333  0.829113  1500.000000"
            ]
          },
          "execution_count": 82,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "class_report = metrics.classification_report(test_y, predicted, output_dict=True)\n",
        "report_df = pd.DataFrame(class_report).transpose()\n",
        "report_df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wZtjuVRw17S-"
      },
      "source": [
        "### 使用fasttext"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NbRP2S8YIZvY",
        "outputId": "b7777ea4-1d8b-4632-89a8-cd62f3205358"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['__label__水果 甜 包裝 好看 速度 盒子 送禮 剛剛好 支援 京東',\n",
              " '__label__洗髮水 東西 快遞 實惠 好評',\n",
              " '__label__水果 購買 好多 次 價格 小貴 口感 不錯']"
            ]
          },
          "execution_count": 83,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "all_texts = train_df['tokens'].tolist()\n",
        "all_labels = train_df['cat'].tolist()\n",
        "prep_datapoints=[]\n",
        "for i in range(len(all_texts)):\n",
        "    sample = '__label__'+ str(all_labels[i]) + ' '+ all_texts[i]\n",
        "    prep_datapoints.append(sample)\n",
        "prep_datapoints[:3]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f2Wtm3gXJJ9I"
      },
      "outputs": [],
      "source": [
        "with open('train_fasttext.txt','w') as f:\n",
        "    for datapoint in prep_datapoints:\n",
        "        f.write(datapoint)\n",
        "        f.write('\\n')\n",
        "    f.close()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gK5XpySYJQWA",
        "outputId": "81335c10-931a-4a72-e9fb-2e89770c11c7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "總耗時0.20587754249572754秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "model = fasttext.train_supervised('train_fasttext.txt')\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "izxql8OOKgkX",
        "outputId": "e983cde4-13e5-4517-efe6-9d76475836ad"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'彈力 不錯 穿著 舒服 質量 過關 好評'"
            ]
          },
          "execution_count": 86,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sample = test_df.iloc[33]['tokens']\n",
        "sample"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "CwwWQNWHKuQi",
        "outputId": "baad77ac-d88d-429c-e66d-f83b66da031d"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'衣服'"
            ]
          },
          "execution_count": 87,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "answer = test_df.iloc[33]['cat']\n",
        "answer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "36P5PKPwK0US",
        "outputId": "eb7cd61c-9e64-4778-b3b3-1f8ed7b2e813"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(('__label__衣服',), array([0.98002046]))"
            ]
          },
          "execution_count": 88,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "predicted = model.predict(sample)\n",
        "predicted"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HWPV4zPLJ7Fm",
        "outputId": "e7d43792-3dce-49dc-a875-38e7a9b530d2"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['__label__洗髮水 確定 假貨 家 沙宣 東西 京東 搞 活動 趕緊 買 到貨 後 放 想 著 剩下 用完 拆 新 媳婦 拆開 用了 說 合適 晚上 洗澡 時候 為了 驗證 洗 時候 感覺 味道 對 頭髮 幹了 沒有 家裡 沙宣 洗出來 順滑 還 乾 糙 第二天 上午 立馬 頭屑 挺癢 中午 到家 全 扔 想 佔便宜 親們 想想 買',\n",
              " '__label__酒店 前臺 服務 不錯 中 餐廳 服務 差些 健身房 早上 開放 比較 晚',\n",
              " '__label__衣服 差差 差差 差差 差差 差差 差差 差差']"
            ]
          },
          "execution_count": 89,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "all_texts = test_df['tokens'].tolist()\n",
        "all_labels = test_df['cat'].tolist()\n",
        "prep_datapoints=[]\n",
        "for i in range(len(all_texts)):\n",
        "    sample = '__label__'+ str(all_labels[i]) + ' '+ all_texts[i]\n",
        "    prep_datapoints.append(sample)\n",
        "prep_datapoints[:3]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "u2usR6PbKKXi"
      },
      "outputs": [],
      "source": [
        "with open('test_fasttext.txt','w') as f:\n",
        "    for datapoint in prep_datapoints:\n",
        "        f.write(datapoint)\n",
        "        f.write('\\n')\n",
        "    f.close()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "o45eN8nNKR53",
        "outputId": "ffc8f657-8256-47e5-bfc6-9d0d08be1e58"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(1500, 0.818, 0.818)"
            ]
          },
          "execution_count": 91,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "model.test(\"test_fasttext.txt\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kFL_VI5gfeDD",
        "outputId": "c85e7332-e427-4a1a-bac2-200f9d6a0aaa"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['平板', '平板', '水果']"
            ]
          },
          "execution_count": 92,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "predicted = [model.predict(text)[0][0].split(\"__\")[-1] for text in test_X] \n",
        "predicted[-3:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ais8AZ7JgdHd",
        "outputId": "66c253a6-33dc-44f9-f607-8097bfdffb6f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array(['送給 兒子 禮物 寶貝 喜歡 外觀 漂亮 好用 買 幾部 華為 手機 經得 住 摔 這 部 不會 讓 失望 繼續 支援 華為 繼續 支援 京東',\n",
              "       '解析度 螢幕 靈敏度 高 快遞員 給力 送 地方 還 取 那你 說 快遞 幹什麼 還是 沒 給 快遞員 給 摔 真心 傷心 噁心 京東 網購',\n",
              "       '寶貝 太 物超所值 好吃 要不要 經濟 實惠 愉快 次 購物 買 水果 選 家'], dtype=object)"
            ]
          },
          "execution_count": 93,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "test_X[-3:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "XmutUcpUeFlN",
        "outputId": "dabefea9-e06d-4058-a627-7204b97f0d48"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>precision</th>\n",
              "      <th>recall</th>\n",
              "      <th>f1-score</th>\n",
              "      <th>support</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>平板</th>\n",
              "      <td>0.660221</td>\n",
              "      <td>0.788779</td>\n",
              "      <td>0.718797</td>\n",
              "      <td>303.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>水果</th>\n",
              "      <td>0.904059</td>\n",
              "      <td>0.795455</td>\n",
              "      <td>0.846287</td>\n",
              "      <td>308.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>洗髮水</th>\n",
              "      <td>0.725753</td>\n",
              "      <td>0.740614</td>\n",
              "      <td>0.733108</td>\n",
              "      <td>293.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>衣服</th>\n",
              "      <td>0.868512</td>\n",
              "      <td>0.822951</td>\n",
              "      <td>0.845118</td>\n",
              "      <td>305.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>酒店</th>\n",
              "      <td>0.985663</td>\n",
              "      <td>0.945017</td>\n",
              "      <td>0.964912</td>\n",
              "      <td>291.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>accuracy</th>\n",
              "      <td>0.818000</td>\n",
              "      <td>0.818000</td>\n",
              "      <td>0.818000</td>\n",
              "      <td>0.818</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>macro avg</th>\n",
              "      <td>0.828842</td>\n",
              "      <td>0.818563</td>\n",
              "      <td>0.821644</td>\n",
              "      <td>1500.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>weighted avg</th>\n",
              "      <td>0.828578</td>\n",
              "      <td>0.818000</td>\n",
              "      <td>0.821202</td>\n",
              "      <td>1500.000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "              precision    recall  f1-score   support\n",
              "平板             0.660221  0.788779  0.718797   303.000\n",
              "水果             0.904059  0.795455  0.846287   308.000\n",
              "洗髮水            0.725753  0.740614  0.733108   293.000\n",
              "衣服             0.868512  0.822951  0.845118   305.000\n",
              "酒店             0.985663  0.945017  0.964912   291.000\n",
              "accuracy       0.818000  0.818000  0.818000     0.818\n",
              "macro avg      0.828842  0.818563  0.821644  1500.000\n",
              "weighted avg   0.828578  0.818000  0.821202  1500.000"
            ]
          },
          "execution_count": 94,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "class_report = metrics.classification_report(test_y, predicted, output_dict=True)\n",
        "report_df = pd.DataFrame(class_report).transpose()\n",
        "report_df"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3V9F5-Zr2AG1"
      },
      "source": [
        "### 使用transformer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Lf4BpyZZl87h",
        "outputId": "edbcb014-11b5-4db0-e302-28a32104473c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "nvcc: NVIDIA (R) Cuda compiler driver\n",
            "Copyright (c) 2005-2020 NVIDIA Corporation\n",
            "Built on Mon_Oct_12_20:09:46_PDT_2020\n",
            "Cuda compilation tools, release 11.1, V11.1.105\n",
            "Build cuda_11.1.TC455_06.29190527_0\n"
          ]
        }
      ],
      "source": [
        "!nvcc --version"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nk2zMpIGlr2u",
        "outputId": "23b785d4-ca19-4b34-e8ac-e392b5f791b6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: pip in /usr/local/lib/python3.7/dist-packages (21.3.1)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (59.4.0)\n",
            "Requirement already satisfied: wheel in /usr/local/lib/python3.7/dist-packages (0.37.0)\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
            "Requirement already satisfied: spacy[cuda111,transformers] in /usr/local/lib/python3.7/dist-packages (3.2.0)\n",
            "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (2.0.6)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (2.0.6)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (59.4.0)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (4.62.3)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (2.23.0)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (0.6.1)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (1.0.6)\n",
            "Requirement already satisfied: thinc<8.1.0,>=8.0.12 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (8.0.13)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (1.19.5)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (3.0.6)\n",
            "Requirement already satisfied: typer<0.5.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (0.4.0)\n",
            "Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (3.10.0.2)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (0.8.2)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (3.3.0)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (3.0.8)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (1.0.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (2.11.3)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (21.3)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (0.4.1)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (1.8.2)\n",
            "Requirement already satisfied: srsly<3.0.0,>=2.4.1 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (2.4.2)\n",
            "Requirement already satisfied: cupy-cuda111<10.0.0,>=5.0.0b4 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (9.4.0)\n",
            "Requirement already satisfied: spacy-transformers<1.2.0,>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from spacy[cuda111,transformers]) (1.1.2)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy[cuda111,transformers]) (3.6.0)\n",
            "Requirement already satisfied: fastrlock>=0.5 in /usr/local/lib/python3.7/dist-packages (from cupy-cuda111<10.0.0,>=5.0.0b4->spacy[cuda111,transformers]) (0.8)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->spacy[cuda111,transformers]) (3.0.6)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy[cuda111,transformers]) (5.2.1)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy[cuda111,transformers]) (2021.10.8)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy[cuda111,transformers]) (3.0.4)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy[cuda111,transformers]) (2.10)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy[cuda111,transformers]) (1.24.3)\n",
            "Requirement already satisfied: transformers<4.12.0,>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (4.11.3)\n",
            "Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (1.10.0+cu111)\n",
            "Requirement already satisfied: spacy-alignments<1.0.0,>=0.7.2 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (0.8.4)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer<0.5.0,>=0.3.0->spacy[cuda111,transformers]) (7.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy[cuda111,transformers]) (2.0.1)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (4.8.2)\n",
            "Requirement already satisfied: huggingface-hub>=0.0.17 in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (0.2.1)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (6.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (3.4.0)\n",
            "Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (0.0.46)\n",
            "Requirement already satisfied: tokenizers<0.11,>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (0.10.3)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (2019.12.20)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (1.1.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->spacy[cuda111,transformers]) (1.15.0)\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "!pip install -U pip setuptools wheel\n",
        "!pip install -U spacy[cuda111,transformers]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OZnUHGTkj_NG",
        "outputId": "da647db3-7b22-4da0-a789-aaf8ac108bd8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Collecting zh-core-web-trf==3.2.0\n",
            "  Downloading https://github.com/explosion/spacy-models/releases/download/zh_core_web_trf-3.2.0/zh_core_web_trf-3.2.0-py3-none-any.whl (417.4 MB)\n",
            "     |████████████████████████████████| 417.4 MB 20 kB/s              \n",
            "\u001b[?25hRequirement already satisfied: spacy-transformers<1.2.0,>=1.1.2 in /usr/local/lib/python3.7/dist-packages (from zh-core-web-trf==3.2.0) (1.1.2)\n",
            "Requirement already satisfied: spacy<3.3.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from zh-core-web-trf==3.2.0) (3.2.0)\n",
            "Requirement already satisfied: spacy-pkuseg<0.1.0,>=0.0.27 in /usr/local/lib/python3.7/dist-packages (from zh-core-web-trf==3.2.0) (0.0.28)\n",
            "Requirement already satisfied: numpy>=1.15.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (1.19.5)\n",
            "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (2.0.6)\n",
            "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (3.0.6)\n",
            "Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (0.4.1)\n",
            "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (2.0.6)\n",
            "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (1.0.1)\n",
            "Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (0.6.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (21.3)\n",
            "Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (3.3.0)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.13.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (2.23.0)\n",
            "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (4.62.3)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (59.4.0)\n",
            "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (1.8.2)\n",
            "Requirement already satisfied: wasabi<1.1.0,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (0.8.2)\n",
            "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.8 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (3.0.8)\n",
            "Requirement already satisfied: srsly<3.0.0,>=2.4.1 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (2.4.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (2.11.3)\n",
            "Requirement already satisfied: typing-extensions<4.0.0.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (3.10.0.2)\n",
            "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (1.0.6)\n",
            "Requirement already satisfied: thinc<8.1.0,>=8.0.12 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (8.0.13)\n",
            "Requirement already satisfied: typer<0.5.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (0.4.0)\n",
            "Requirement already satisfied: cython>=0.25 in /usr/local/lib/python3.7/dist-packages (from spacy-pkuseg<0.1.0,>=0.0.27->zh-core-web-trf==3.2.0) (0.29.24)\n",
            "Requirement already satisfied: spacy-alignments<1.0.0,>=0.7.2 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (0.8.4)\n",
            "Requirement already satisfied: torch>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (1.10.0+cu111)\n",
            "Requirement already satisfied: transformers<4.12.0,>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (4.11.3)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (3.6.0)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (3.0.6)\n",
            "Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (5.2.1)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (3.0.4)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (2021.10.8)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.13.0->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (2.10)\n",
            "Requirement already satisfied: tokenizers<0.11,>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (0.10.3)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (2019.12.20)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (4.8.2)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (6.0)\n",
            "Requirement already satisfied: huggingface-hub>=0.0.17 in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (0.2.1)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (3.4.0)\n",
            "Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (0.0.46)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer<0.5.0,>=0.3.0->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (7.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy<3.3.0,>=3.2.0->zh-core-web-trf==3.2.0) (2.0.1)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (1.15.0)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers<4.12.0,>=3.4.0->spacy-transformers<1.2.0,>=1.1.2->zh-core-web-trf==3.2.0) (1.1.0)\n",
            "\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\n",
            "\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
            "You can now load the package via spacy.load('zh_core_web_trf')\n"
          ]
        }
      ],
      "source": [
        "!python -m spacy download zh_core_web_trf"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cQXwZrhRkEwa"
      },
      "outputs": [],
      "source": [
        "import spacy\n",
        "\n",
        "spacy.prefer_gpu()\n",
        "nlp = spacy.load(\"zh_core_web_trf\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4QJIEEYerDAS",
        "outputId": "9902160c-e2ec-469e-f607-36e3fb9adf1a"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['transformer', 'tagger', 'parser', 'attribute_ruler', 'ner']"
            ]
          },
          "execution_count": 53,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "nlp.pipe_names"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ibj9mtCDn6nS",
        "outputId": "eb61f0f8-4e5f-4428-b030-54a2923999c6"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "WordpieceBatch(strings=[['[CLS]', '送', '人', '的', '，', '不', '知', '道', '好', '不', '好', '。', '聽', '說', '紅', '心', '火', '龍', '果', '還', '是', '比', '較', '甜', '的', '。', '[SEP]']], input_ids=array([[ 101, 6843,  782, 4638, 8024,  679, 4761, 6887, 1962,  679, 1962,\n",
              "         511, 5481, 6303, 5148, 2552, 4125, 7983, 3362, 6917, 3221, 3683,\n",
              "        6733, 4494, 4638,  511,  102]], dtype=int32), attention_mask=array([[1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,\n",
              "        1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.]], dtype=float32), lengths=[27], token_type_ids=array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
              "        0, 0, 0, 0, 0]], dtype=int32))"
            ]
          },
          "execution_count": 54,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "text = \"送人的，不知道好不好。聽說紅心火龍果還是比較甜的。\"\n",
        "doc = nlp(text)\n",
        "doc._.trf_data.wordpieces"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zMu2zjbGoucy",
        "outputId": "47f05931-670c-428e-bf27-4324425658d8"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(1, 27, 768)"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "doc._.trf_data.tensors[0].shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vTeI9wK_sCZc",
        "outputId": "a88aaf19-755c-46ec-8992-84e0d01c7ec4"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['洗髮水', '酒店', '衣服', '平板', '水果']"
            ]
          },
          "execution_count": 47,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "text_labels = train_df['cat'].unique().tolist()\n",
        "text_labels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "80ITsjJJo_AE",
        "outputId": "3a92a912-f755-4a71-ad84-1aeb2f773ed1"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:1: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  \"\"\"Entry point for launching an IPython kernel.\n",
            "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:3: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  This is separate from the ipykernel package so we can avoid doing imports until\n"
          ]
        }
      ],
      "source": [
        "train_df['tuples'] = train_df.apply(lambda row: (row['review'], row['cat']), axis=1)\n",
        "train_data = train_df['tuples'].tolist()\n",
        "test_df['tuples'] = test_df.apply(lambda row: (row['review'], row['cat']), axis=1)\n",
        "test_data = test_df['tuples'].tolist()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dS7r1bpyutNH",
        "outputId": "8e155e9d-f5c8-4ce0-b10f-2a2be29b429a"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "('總體感覺就是 做不出價效比高的平板來 就別做 也不能 忽悠人吧', '平板')"
            ]
          },
          "execution_count": 58,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train_data[100]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "iM7fhjwDu6bM"
      },
      "outputs": [],
      "source": [
        "from tqdm.auto import tqdm\n",
        "from spacy.tokens import DocBin\n",
        "\n",
        "def make_docs(data_list, unique_labels, dest_path):\n",
        "    docs = []\n",
        "    for doc, label in tqdm(nlp.pipe(data_list, as_tuples=True), total=len(data_list)):\n",
        "        label_dict = {label: False for label in unique_labels}\n",
        "        doc.cats = label_dict\n",
        "        doc.cats[label] = True\n",
        "        docs.append(doc)\n",
        "    doc_bin = DocBin(docs=docs)\n",
        "    doc_bin.to_disk(dest_path)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 81,
          "referenced_widgets": [
            "32fe9de1c45240d3bc3ca25e9bb7bb86",
            "8a375859a7a046e2821bc22916ac5246",
            "fc36d7b6c67b4642953d7e6d3bb70320",
            "a54510f248d74814a40624118e9e23e6",
            "670669e227c1424ea1967a11821196d6",
            "e01d27fe11764182884481768a4cb68d",
            "4c85e46561c940dfa1b5b6a8b0a95b8b",
            "13d8decc44f74da582bf1ce3521721f4",
            "a5baff0f9fd44801b244fbc2e01d0964",
            "ac15f2fff49846cba49cee0c41967ea3",
            "a9c73838a82e4ac78c61663607e2aa40",
            "afab9b71d9554783b5a0dcfe4f8d23d2",
            "24f68c720a4c43ceac159ea24506df93",
            "5342d84e474f41a79b7d9c5d3e886015",
            "5c08ba0fa182413aa1d11f0af7ec47ae",
            "c77b60a1d79e4e94bf4f230787362e81",
            "0eb1503563a245b6bd2d1360af2a612f",
            "2e80c65b32d6434fb1cd6d0628fade2a",
            "b8afde1754974cfe9ac3f3482b983a07",
            "c78c35f0e8444b3dbfa4511da23baaba",
            "d09dd27b15a146d4835dd25c24d09346",
            "c48faba8bcc64746a08b2a35a2af25c7"
          ]
        },
        "id": "kVDF0GYjwTNE",
        "outputId": "82ab9d57-6c53-4b05-affb-3fd848f7258c"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "32fe9de1c45240d3bc3ca25e9bb7bb86",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/3000 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "afab9b71d9554783b5a0dcfe4f8d23d2",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "  0%|          | 0/1000 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "make_docs(train_data, text_labels, \"train.spacy\")\n",
        "make_docs(test_data, text_labels, \"test.spacy\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4JObXZKa1EUL"
      },
      "source": [
        " 取得基本[設定檔案](https://spacy.io/usage/training)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ky4x-trNw-qi",
        "outputId": "d6cde89d-c00c-4bf5-a473-f29fc4b717ed"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Overwriting base_config.cfg\n"
          ]
        }
      ],
      "source": [
        "%%writefile base_config.cfg\n",
        "\n",
        "# This is an auto-generated partial config. To use it with 'spacy train'\n",
        "# you can run spacy init fill-config to auto-fill all default settings:\n",
        "# python -m spacy init fill-config ./base_config.cfg ./config.cfg\n",
        "[paths]\n",
        "train = null\n",
        "dev = null\n",
        "\n",
        "[system]\n",
        "gpu_allocator = \"pytorch\"\n",
        "\n",
        "[nlp]\n",
        "lang = \"zh\"\n",
        "pipeline = [\"transformer\",\"textcat\"]\n",
        "batch_size = 128\n",
        "\n",
        "[components]\n",
        "\n",
        "[components.transformer]\n",
        "factory = \"transformer\"\n",
        "\n",
        "[components.transformer.model]\n",
        "@architectures = \"spacy-transformers.TransformerModel.v3\"\n",
        "name = \"bert-base-chinese\"\n",
        "tokenizer_config = {\"use_fast\": true}\n",
        "\n",
        "[components.transformer.model.get_spans]\n",
        "@span_getters = \"spacy-transformers.strided_spans.v1\"\n",
        "window = 128\n",
        "stride = 96\n",
        "\n",
        "[components.textcat]\n",
        "factory = \"textcat\"\n",
        "\n",
        "[components.textcat.model]\n",
        "@architectures = \"spacy.TextCatEnsemble.v2\"\n",
        "nO = null\n",
        "\n",
        "[components.textcat.model.tok2vec]\n",
        "@architectures = \"spacy-transformers.TransformerListener.v1\"\n",
        "grad_factor = 1.0\n",
        "\n",
        "[components.textcat.model.tok2vec.pooling]\n",
        "@layers = \"reduce_mean.v1\"\n",
        "\n",
        "[components.textcat.model.linear_model]\n",
        "@architectures = \"spacy.TextCatBOW.v2\"\n",
        "exclusive_classes = true\n",
        "ngram_size = 1\n",
        "no_output_layer = false\n",
        "\n",
        "[corpora]\n",
        "\n",
        "[corpora.train]\n",
        "@readers = \"spacy.Corpus.v1\"\n",
        "path = ${paths.train}\n",
        "max_length = 0\n",
        "\n",
        "[corpora.dev]\n",
        "@readers = \"spacy.Corpus.v1\"\n",
        "path = ${paths.dev}\n",
        "max_length = 0\n",
        "\n",
        "[training]\n",
        "accumulate_gradient = 3\n",
        "dev_corpus = \"corpora.dev\"\n",
        "train_corpus = \"corpora.train\"\n",
        "\n",
        "[training.optimizer]\n",
        "@optimizers = \"Adam.v1\"\n",
        "\n",
        "[training.optimizer.learn_rate]\n",
        "@schedules = \"warmup_linear.v1\"\n",
        "warmup_steps = 250\n",
        "total_steps = 20000\n",
        "initial_rate = 5e-5\n",
        "\n",
        "[training.batcher]\n",
        "@batchers = \"spacy.batch_by_padded.v1\"\n",
        "discard_oversize = true\n",
        "size = 2000\n",
        "buffer = 256\n",
        "\n",
        "[initialize]\n",
        "vectors = ${paths.vectors}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R91kFJ8ax7di",
        "outputId": "bb8bcad3-c833-4b4d-c277-6be63e03815c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[38;5;2m✔ Auto-filled config with all values\u001b[0m\n",
            "\u001b[38;5;2m✔ Saved config\u001b[0m\n",
            "config.cfg\n",
            "You can now add your data and train your pipeline:\n",
            "python -m spacy train config.cfg --paths.train ./train.spacy --paths.dev ./dev.spacy\n"
          ]
        }
      ],
      "source": [
        "!python -m spacy init fill-config ./base_config.cfg ./config.cfg"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bb0OjTdSyODb"
      },
      "outputs": [],
      "source": [
        "!mkdir output"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "I3gw3LKpysfD"
      },
      "outputs": [],
      "source": [
        "import time"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q7xchz0yxj2u",
        "outputId": "b32f4981-d73b-45e4-80ec-2c8c970970d3"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[2021-12-05 04:16:22,835] [DEBUG] Config overrides from CLI: ['paths.train', 'paths.dev']\n",
            "\u001b[38;5;4mℹ Saving to output directory: /content/output\u001b[0m\n",
            "\u001b[38;5;4mℹ Using GPU: 0\u001b[0m\n",
            "\u001b[1m\n",
            "=========================== Initializing pipeline ===========================\u001b[0m\n",
            "[2021-12-05 04:16:25,808] [INFO] Set up nlp object from config\n",
            "[2021-12-05 04:16:25,822] [DEBUG] Loading corpus from path: /content/test.spacy\n",
            "[2021-12-05 04:16:25,824] [DEBUG] Loading corpus from path: /content/train.spacy\n",
            "[2021-12-05 04:16:25,824] [INFO] Pipeline: ['transformer', 'textcat']\n",
            "[2021-12-05 04:16:25,836] [INFO] Created vocabulary\n",
            "[2021-12-05 04:16:25,838] [INFO] Finished initializing nlp object\n",
            "Downloading: 100% 29.0/29.0 [00:00<00:00, 27.6kB/s]\n",
            "Downloading: 100% 624/624 [00:00<00:00, 564kB/s]\n",
            "Downloading: 100% 107k/107k [00:00<00:00, 202kB/s] \n",
            "Downloading: 100% 263k/263k [00:00<00:00, 372kB/s]\n",
            "Downloading: 100% 393M/393M [00:25<00:00, 16.1MB/s]\n",
            "Some weights of the model checkpoint at bert-base-chinese were not used when initializing BertModel: ['cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.dense.bias']\n",
            "- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "[2021-12-05 04:17:18,737] [INFO] Initialized pipeline components: ['transformer', 'textcat']\n",
            "\u001b[38;5;2m✔ Initialized pipeline\u001b[0m\n",
            "\u001b[1m\n",
            "============================= Training pipeline =============================\u001b[0m\n",
            "[2021-12-05 04:17:18,750] [DEBUG] Loading corpus from path: /content/test.spacy\n",
            "[2021-12-05 04:17:18,751] [DEBUG] Loading corpus from path: /content/train.spacy\n",
            "\u001b[38;5;4mℹ Pipeline: ['transformer', 'textcat']\u001b[0m\n",
            "\u001b[38;5;4mℹ Initial learn rate: 0.0\u001b[0m\n",
            "E    #       LOSS TRANS...  LOSS TEXTCAT  CATS_SCORE  SCORE \n",
            "---  ------  -------------  ------------  ----------  ------\n",
            "  0       0           0.00          2.40        0.00    0.00\n",
            "  1     200           0.04         80.77       86.20    0.86\n",
            "  3     400           0.38         15.07       87.36    0.87\n",
            "  4     600           0.52          5.67       88.84    0.89\n",
            "  6     800           0.18          3.00       88.36    0.88\n",
            "  8    1000           0.12          2.74       89.38    0.89\n",
            "  9    1200           0.12          0.32       88.59    0.89\n",
            " 11    1400           0.26          0.44       88.12    0.88\n",
            " 13    1600           0.02          0.00       88.34    0.88\n",
            " 14    1800           0.04          0.01       87.86    0.88\n",
            " 16    2000           0.03          0.02       88.96    0.89\n",
            " 18    2200           0.04          0.05       88.33    0.88\n",
            " 19    2400           0.08          0.20       87.23    0.87\n",
            " 21    2600           0.02          0.01       88.64    0.89\n",
            "\u001b[38;5;2m✔ Saved pipeline to output directory\u001b[0m\n",
            "/content/output/model-last\n",
            "總耗時3282.8532857894897秒\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "!python -m spacy train /content/config.cfg --output /content/output --paths.train /content/train.spacy --paths.dev /content/test.spacy --gpu-id 0 --verbose\n",
        "end = time.time()\n",
        "print(f\"總耗時{end-start}秒\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GHUvvCH4NxRH",
        "outputId": "15d4eb8f-058c-4d85-b454-e41e8ee35c6c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "如果不是飛機取消臨時住宿，怎麼都不會去住的 酒店\n"
          ]
        }
      ],
      "source": [
        "sample_text = test_df.loc[350]['review']\n",
        "sample_cat = test_df.loc[350]['cat']\n",
        "print(sample_text, sample_cat)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "o90PCYO60bKb",
        "outputId": "007aac8d-2448-47e0-e0ea-bcec498f4cd6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'水果': 1.4866126321066986e-06, '洗髮水': 1.8659034140000585e-06, '酒店': 0.9999920129776001, '衣服': 1.6179602653210168e-06, '平板': 3.0514656828017905e-06}\n"
          ]
        }
      ],
      "source": [
        "nlp = spacy.load(\"/content/output/model-last\")\n",
        "doc = nlp(sample_text)\n",
        "print(doc.cats)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "id": "-0Tn_U1IOty5",
        "outputId": "e808ecf5-d6d0-4465-8dc3-03d3fd0c3394"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'酒店'"
            ]
          },
          "execution_count": 74,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "max_cat = max(doc.cats, key=doc.cats.get)\n",
        "max_cat"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "S8Sj3gf8PPQy",
        "outputId": "a4d8b8e8-49f5-4174-e5b4-68017400128d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'cats_f_per_type': {'平板': {'f': 0.8431703204,\n",
              "   'p': 0.8620689655,\n",
              "   'r': 0.8250825083},\n",
              "  '水果': {'f': 0.9143780291, 'p': 0.9099678457, 'r': 0.9188311688},\n",
              "  '洗髮水': {'f': 0.8376068376, 'p': 0.8390410959, 'r': 0.8361774744},\n",
              "  '衣服': {'f': 0.8946515397, 'p': 0.8846153846, 'r': 0.9049180328},\n",
              "  '酒店': {'f': 0.9793103448, 'p': 0.9826989619, 'r': 0.9759450172}},\n",
              " 'cats_macro_auc': 0.9845796608,\n",
              " 'cats_macro_auc_per_type': 0.0,\n",
              " 'cats_macro_f': 0.8938234143,\n",
              " 'cats_macro_p': 0.8956784507,\n",
              " 'cats_macro_r': 0.8921908403,\n",
              " 'cats_micro_f': 0.8937875752,\n",
              " 'cats_micro_p': 0.8955823293,\n",
              " 'cats_micro_r': 0.892,\n",
              " 'cats_score': 0.8938234143,\n",
              " 'cats_score_desc': 'macro F',\n",
              " 'textcat_loss': 2.7358574572,\n",
              " 'transformer_loss': 0.1171762578}"
            ]
          },
          "execution_count": 75,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import json\n",
        "\n",
        "model_meta_path = \"/content/output/model-best/meta.json\"\n",
        "with open(model_meta_path) as json_file:\n",
        "    metrics = json.load(json_file)\n",
        "metrics['performance'] "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7NMeFWm5IU34"
      },
      "outputs": [],
      "source": [
        "import random\n",
        "\n",
        "def show_test():\n",
        "    idx = random.choice(test_df.index)\n",
        "    text = df.at[idx, 'review']\n",
        "    cat = df.at[idx, 'cat']\n",
        "    predicted_proba = nlp(text).cats\n",
        "    predicted_cat = max(predicted_proba, key=predicted_proba.get)\n",
        "    print(f\"編號:{idx}\")\n",
        "    print(f\"評論:{text}\")\n",
        "    print(f\"正確商品類別:{cat}\")\n",
        "    print(f\"類別機率分佈:{json.dumps(predicted_proba, indent=4, ensure_ascii=False)}\")\n",
        "    print(f\"模型判斷類別:{predicted_cat}\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UhMaxlewJ41G",
        "outputId": "1ed7e5af-5f56-4001-ffeb-2d61853d0052"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "編號:3337\n",
            "評論:已經收到了，東西非常的不錯，物流也特別快，用起來非常的流暢，一點兒也不卡，而且沒有任何破損，感覺非常好，價格來說京東也是比較優惠一些的，以後還會在京東繼續購物\n",
            "正確商品類別:平板\n",
            "類別機率分佈:{\n",
            "    \"水果\": 3.9765222936694045e-06,\n",
            "    \"洗髮水\": 5.964189404039644e-05,\n",
            "    \"酒店\": 3.3598655591049464e-06,\n",
            "    \"衣服\": 4.1509620132273994e-06,\n",
            "    \"平板\": 0.9999289512634277\n",
            "}\n",
            "模型判斷類別:平板\n"
          ]
        }
      ],
      "source": [
        "show_test()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GeFotoLXTDaH",
        "outputId": "b8757643-1287-4703-b291-5f85b43b381b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "編號:3000\n",
            "評論:真正好PAD，學英語不用愁啦\n",
            "正確商品類別:平板\n",
            "類別機率分佈:{\n",
            "    \"水果\": 0.007460991386324167,\n",
            "    \"洗髮水\": 0.8048735857009888,\n",
            "    \"酒店\": 0.0007097200723364949,\n",
            "    \"衣服\": 0.003813863033428788,\n",
            "    \"平板\": 0.18314185738563538\n",
            "}\n",
            "模型判斷類別:洗髮水\n"
          ]
        }
      ],
      "source": [
        "show_test()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a11-DzEqjaJf"
      },
      "source": [
        "# 相關連結\n",
        "\n",
        "- [Symbolic, Distributed, and Distributional Representations for Natural Language Processing in the Era of Deep Learning: A Survey](https://www.frontiersin.org/articles/10.3389/frobt.2019.00153/full)\n",
        "- [Transfer Learning for Natural Language Processing](https://www.manning.com/books/transfer-learning-for-natural-language-processing)\n",
        "- [Mastering spaCy](https://www.packtpub.com/product/mastering-spacy/9781800563353)"
      ]
    }
  ],
  "metadata": {
    "celltoolbar": "Slideshow",
    "colab": {
      "collapsed_sections": [
        "Us8FwcHEwJuX",
        "_GaxajzGlass"
      ],
      "name": "CJCU_talk.ipynb",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "jupytext": {
      "formats": "ipynb,auto:percent"
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.11"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "0eb1503563a245b6bd2d1360af2a612f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "13d8decc44f74da582bf1ce3521721f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "24f68c720a4c43ceac159ea24506df93": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2e80c65b32d6434fb1cd6d0628fade2a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "32fe9de1c45240d3bc3ca25e9bb7bb86": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_fc36d7b6c67b4642953d7e6d3bb70320",
              "IPY_MODEL_a54510f248d74814a40624118e9e23e6",
              "IPY_MODEL_670669e227c1424ea1967a11821196d6"
            ],
            "layout": "IPY_MODEL_8a375859a7a046e2821bc22916ac5246"
          }
        },
        "4c85e46561c940dfa1b5b6a8b0a95b8b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5342d84e474f41a79b7d9c5d3e886015": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2e80c65b32d6434fb1cd6d0628fade2a",
            "placeholder": "​",
            "style": "IPY_MODEL_0eb1503563a245b6bd2d1360af2a612f",
            "value": "100%"
          }
        },
        "5c08ba0fa182413aa1d11f0af7ec47ae": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c78c35f0e8444b3dbfa4511da23baaba",
            "max": 1000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b8afde1754974cfe9ac3f3482b983a07",
            "value": 1000
          }
        },
        "670669e227c1424ea1967a11821196d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a9c73838a82e4ac78c61663607e2aa40",
            "placeholder": "​",
            "style": "IPY_MODEL_ac15f2fff49846cba49cee0c41967ea3",
            "value": " 3000/3000 [01:11&lt;00:00, 43.90it/s]"
          }
        },
        "8a375859a7a046e2821bc22916ac5246": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a54510f248d74814a40624118e9e23e6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a5baff0f9fd44801b244fbc2e01d0964",
            "max": 3000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_13d8decc44f74da582bf1ce3521721f4",
            "value": 3000
          }
        },
        "a5baff0f9fd44801b244fbc2e01d0964": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a9c73838a82e4ac78c61663607e2aa40": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ac15f2fff49846cba49cee0c41967ea3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "afab9b71d9554783b5a0dcfe4f8d23d2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5342d84e474f41a79b7d9c5d3e886015",
              "IPY_MODEL_5c08ba0fa182413aa1d11f0af7ec47ae",
              "IPY_MODEL_c77b60a1d79e4e94bf4f230787362e81"
            ],
            "layout": "IPY_MODEL_24f68c720a4c43ceac159ea24506df93"
          }
        },
        "b8afde1754974cfe9ac3f3482b983a07": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c48faba8bcc64746a08b2a35a2af25c7": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c77b60a1d79e4e94bf4f230787362e81": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c48faba8bcc64746a08b2a35a2af25c7",
            "placeholder": "​",
            "style": "IPY_MODEL_d09dd27b15a146d4835dd25c24d09346",
            "value": " 1000/1000 [00:22&lt;00:00, 51.70it/s]"
          }
        },
        "c78c35f0e8444b3dbfa4511da23baaba": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d09dd27b15a146d4835dd25c24d09346": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e01d27fe11764182884481768a4cb68d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fc36d7b6c67b4642953d7e6d3bb70320": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4c85e46561c940dfa1b5b6a8b0a95b8b",
            "placeholder": "​",
            "style": "IPY_MODEL_e01d27fe11764182884481768a4cb68d",
            "value": "100%"
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}