APS_datamining.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# APC Scania Truck Failure Dataset\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Veri seti, Scania marka kamyonların günlük verilerinden oluşmaktadır. Frenleme ve vites değişimlerinde kullanılan Hava Basınç Sistemine (APS) ait veriler bulunmaktadır.  Pozitif olan örnekler APS ile ilgili sorunları Negatif olanlar ise APS ile alakasız sorunları işaret etmektedir."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gerekli kütüphaneleri import edelim."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt \n",
    "from scipy import stats\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "from sklearn.metrics import confusion_matrix, accuracy_score\n",
    "from sklearn.neighbors import LocalOutlierFactor\n",
    "from sklearn.feature_selection import SelectKBest, f_classif\n",
    "from sklearn.naive_bayes import ComplementNB\n",
    "from sklearn.cluster import KMeans\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.cluster import AgglomerativeClustering\n",
    "from scipy.cluster.hierarchy import dendrogram, linkage\n",
    "from sklearn.datasets import make_classification\n",
    "from sklearn.cluster import Birch\n",
    "from sklearn.metrics import precision_recall_curve\n",
    "from sklearn.metrics import plot_precision_recall_curve\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import precision_recall_fscore_support as score\n",
    "\n",
    "import seaborn as sn"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Veri setlerimizi dataframe olarak dosyadan alalım."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = pd.read_csv(\"aps_failure_training_set.csv\")\n",
    "df_test = pd.read_csv(\"aps_failure_test_set.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Veri setimizde rahat işlem yapabilmek için attributeların tiplerini ve olmayan değerlerin gösterimini düzenleyelim."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = df_train.replace(\"na\", np.nan)\n",
    "df_test = df_test.replace(\"na\", np.nan)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convertToNumeric(columns, df):\n",
    "    cols=[i for i in df.columns if i not in columns]\n",
    "    for col in cols:\n",
    "        df[col]=pd.to_numeric(df[col])\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = convertToNumeric([\"class\"], df_train)\n",
    "df_test = convertToNumeric([\"class\"], df_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Veri setimizin genel özelliklerine bakalım."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>class</th>\n",
       "      <th>aa_000</th>\n",
       "      <th>ab_000</th>\n",
       "      <th>ac_000</th>\n",
       "      <th>ad_000</th>\n",
       "      <th>ae_000</th>\n",
       "      <th>af_000</th>\n",
       "      <th>ag_000</th>\n",
       "      <th>ag_001</th>\n",
       "      <th>ag_002</th>\n",
       "      <th>...</th>\n",
       "      <th>ee_002</th>\n",
       "      <th>ee_003</th>\n",
       "      <th>ee_004</th>\n",
       "      <th>ee_005</th>\n",
       "      <th>ee_006</th>\n",
       "      <th>ee_007</th>\n",
       "      <th>ee_008</th>\n",
       "      <th>ee_009</th>\n",
       "      <th>ef_000</th>\n",
       "      <th>eg_000</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>neg</td>\n",
       "      <td>76698</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.130706e+09</td>\n",
       "      <td>280.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1240520.0</td>\n",
       "      <td>493384.0</td>\n",
       "      <td>721044.0</td>\n",
       "      <td>469792.0</td>\n",
       "      <td>339156.0</td>\n",
       "      <td>157956.0</td>\n",
       "      <td>73224.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>neg</td>\n",
       "      <td>33058</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>421400.0</td>\n",
       "      <td>178064.0</td>\n",
       "      <td>293306.0</td>\n",
       "      <td>245416.0</td>\n",
       "      <td>133654.0</td>\n",
       "      <td>81140.0</td>\n",
       "      <td>97576.0</td>\n",
       "      <td>1500.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>neg</td>\n",
       "      <td>41040</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.280000e+02</td>\n",
       "      <td>100.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>277378.0</td>\n",
       "      <td>159812.0</td>\n",
       "      <td>423992.0</td>\n",
       "      <td>409564.0</td>\n",
       "      <td>320746.0</td>\n",
       "      <td>158022.0</td>\n",
       "      <td>95128.0</td>\n",
       "      <td>514.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>neg</td>\n",
       "      <td>12</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.000000e+01</td>\n",
       "      <td>66.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>240.0</td>\n",
       "      <td>46.0</td>\n",
       "      <td>58.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>neg</td>\n",
       "      <td>60874</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.368000e+03</td>\n",
       "      <td>458.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>622012.0</td>\n",
       "      <td>229790.0</td>\n",
       "      <td>405298.0</td>\n",
       "      <td>347188.0</td>\n",
       "      <td>286954.0</td>\n",
       "      <td>311560.0</td>\n",
       "      <td>433954.0</td>\n",
       "      <td>1218.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 171 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  class  aa_000  ab_000        ac_000  ad_000  ae_000  af_000  ag_000  ag_001  \\\n",
       "0   neg   76698     NaN  2.130706e+09   280.0     0.0     0.0     0.0     0.0   \n",
       "1   neg   33058     NaN  0.000000e+00     NaN     0.0     0.0     0.0     0.0   \n",
       "2   neg   41040     NaN  2.280000e+02   100.0     0.0     0.0     0.0     0.0   \n",
       "3   neg      12     0.0  7.000000e+01    66.0     0.0    10.0     0.0     0.0   \n",
       "4   neg   60874     NaN  1.368000e+03   458.0     0.0     0.0     0.0     0.0   \n",
       "\n",
       "   ag_002  ...     ee_002    ee_003    ee_004    ee_005    ee_006    ee_007  \\\n",
       "0     0.0  ...  1240520.0  493384.0  721044.0  469792.0  339156.0  157956.0   \n",
       "1     0.0  ...   421400.0  178064.0  293306.0  245416.0  133654.0   81140.0   \n",
       "2     0.0  ...   277378.0  159812.0  423992.0  409564.0  320746.0  158022.0   \n",
       "3     0.0  ...      240.0      46.0      58.0      44.0      10.0       0.0   \n",
       "4     0.0  ...   622012.0  229790.0  405298.0  347188.0  286954.0  311560.0   \n",
       "\n",
       "     ee_008  ee_009  ef_000  eg_000  \n",
       "0   73224.0     0.0     0.0     0.0  \n",
       "1   97576.0  1500.0     0.0     0.0  \n",
       "2   95128.0   514.0     0.0     0.0  \n",
       "3       0.0     0.0     4.0    32.0  \n",
       "4  433954.0  1218.0     0.0     0.0  \n",
       "\n",
       "[5 rows x 171 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>aa_000</th>\n",
       "      <th>ab_000</th>\n",
       "      <th>ac_000</th>\n",
       "      <th>ad_000</th>\n",
       "      <th>ae_000</th>\n",
       "      <th>af_000</th>\n",
       "      <th>ag_000</th>\n",
       "      <th>ag_001</th>\n",
       "      <th>ag_002</th>\n",
       "      <th>ag_003</th>\n",
       "      <th>...</th>\n",
       "      <th>ee_002</th>\n",
       "      <th>ee_003</th>\n",
       "      <th>ee_004</th>\n",
       "      <th>ee_005</th>\n",
       "      <th>ee_006</th>\n",
       "      <th>ee_007</th>\n",
       "      <th>ee_008</th>\n",
       "      <th>ee_009</th>\n",
       "      <th>ef_000</th>\n",
       "      <th>eg_000</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>6.000000e+04</td>\n",
       "      <td>13671.000000</td>\n",
       "      <td>5.666500e+04</td>\n",
       "      <td>4.513900e+04</td>\n",
       "      <td>57500.000000</td>\n",
       "      <td>57500.000000</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>...</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>5.932900e+04</td>\n",
       "      <td>57276.000000</td>\n",
       "      <td>57277.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>5.933650e+04</td>\n",
       "      <td>0.713189</td>\n",
       "      <td>3.560143e+08</td>\n",
       "      <td>1.906206e+05</td>\n",
       "      <td>6.819130</td>\n",
       "      <td>11.006817</td>\n",
       "      <td>2.216364e+02</td>\n",
       "      <td>9.757223e+02</td>\n",
       "      <td>8.606015e+03</td>\n",
       "      <td>8.859128e+04</td>\n",
       "      <td>...</td>\n",
       "      <td>4.454897e+05</td>\n",
       "      <td>2.111264e+05</td>\n",
       "      <td>4.457343e+05</td>\n",
       "      <td>3.939462e+05</td>\n",
       "      <td>3.330582e+05</td>\n",
       "      <td>3.462714e+05</td>\n",
       "      <td>1.387300e+05</td>\n",
       "      <td>8.388915e+03</td>\n",
       "      <td>0.090579</td>\n",
       "      <td>0.212756</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.454301e+05</td>\n",
       "      <td>3.478962</td>\n",
       "      <td>7.948749e+08</td>\n",
       "      <td>4.040441e+07</td>\n",
       "      <td>161.543373</td>\n",
       "      <td>209.792592</td>\n",
       "      <td>2.047846e+04</td>\n",
       "      <td>3.420053e+04</td>\n",
       "      <td>1.503220e+05</td>\n",
       "      <td>7.617312e+05</td>\n",
       "      <td>...</td>\n",
       "      <td>1.155540e+06</td>\n",
       "      <td>5.433188e+05</td>\n",
       "      <td>1.168314e+06</td>\n",
       "      <td>1.121044e+06</td>\n",
       "      <td>1.069160e+06</td>\n",
       "      <td>1.728056e+06</td>\n",
       "      <td>4.495100e+05</td>\n",
       "      <td>4.747043e+04</td>\n",
       "      <td>4.368855</td>\n",
       "      <td>8.830641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>8.340000e+02</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.600000e+01</td>\n",
       "      <td>2.400000e+01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>...</td>\n",
       "      <td>2.936000e+03</td>\n",
       "      <td>1.166000e+03</td>\n",
       "      <td>2.700000e+03</td>\n",
       "      <td>3.584000e+03</td>\n",
       "      <td>5.120000e+02</td>\n",
       "      <td>1.100000e+02</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.077600e+04</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.520000e+02</td>\n",
       "      <td>1.260000e+02</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>...</td>\n",
       "      <td>2.337960e+05</td>\n",
       "      <td>1.120860e+05</td>\n",
       "      <td>2.215180e+05</td>\n",
       "      <td>1.899880e+05</td>\n",
       "      <td>9.243200e+04</td>\n",
       "      <td>4.109800e+04</td>\n",
       "      <td>3.812000e+03</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>4.866800e+04</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>9.640000e+02</td>\n",
       "      <td>4.300000e+02</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>...</td>\n",
       "      <td>4.383960e+05</td>\n",
       "      <td>2.182320e+05</td>\n",
       "      <td>4.666140e+05</td>\n",
       "      <td>4.032220e+05</td>\n",
       "      <td>2.750940e+05</td>\n",
       "      <td>1.678140e+05</td>\n",
       "      <td>1.397240e+05</td>\n",
       "      <td>2.028000e+03</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.746564e+06</td>\n",
       "      <td>204.000000</td>\n",
       "      <td>2.130707e+09</td>\n",
       "      <td>8.584298e+09</td>\n",
       "      <td>21050.000000</td>\n",
       "      <td>20070.000000</td>\n",
       "      <td>3.376892e+06</td>\n",
       "      <td>4.109372e+06</td>\n",
       "      <td>1.055286e+07</td>\n",
       "      <td>6.340207e+07</td>\n",
       "      <td>...</td>\n",
       "      <td>7.793393e+07</td>\n",
       "      <td>3.775839e+07</td>\n",
       "      <td>9.715238e+07</td>\n",
       "      <td>5.743524e+07</td>\n",
       "      <td>3.160781e+07</td>\n",
       "      <td>1.195801e+08</td>\n",
       "      <td>1.926740e+07</td>\n",
       "      <td>3.810078e+06</td>\n",
       "      <td>482.000000</td>\n",
       "      <td>1146.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 170 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             aa_000        ab_000        ac_000        ad_000        ae_000  \\\n",
       "count  6.000000e+04  13671.000000  5.666500e+04  4.513900e+04  57500.000000   \n",
       "mean   5.933650e+04      0.713189  3.560143e+08  1.906206e+05      6.819130   \n",
       "std    1.454301e+05      3.478962  7.948749e+08  4.040441e+07    161.543373   \n",
       "min    0.000000e+00      0.000000  0.000000e+00  0.000000e+00      0.000000   \n",
       "25%    8.340000e+02      0.000000  1.600000e+01  2.400000e+01      0.000000   \n",
       "50%    3.077600e+04      0.000000  1.520000e+02  1.260000e+02      0.000000   \n",
       "75%    4.866800e+04      0.000000  9.640000e+02  4.300000e+02      0.000000   \n",
       "max    2.746564e+06    204.000000  2.130707e+09  8.584298e+09  21050.000000   \n",
       "\n",
       "             af_000        ag_000        ag_001        ag_002        ag_003  \\\n",
       "count  57500.000000  5.932900e+04  5.932900e+04  5.932900e+04  5.932900e+04   \n",
       "mean      11.006817  2.216364e+02  9.757223e+02  8.606015e+03  8.859128e+04   \n",
       "std      209.792592  2.047846e+04  3.420053e+04  1.503220e+05  7.617312e+05   \n",
       "min        0.000000  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00   \n",
       "25%        0.000000  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00   \n",
       "50%        0.000000  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00   \n",
       "75%        0.000000  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00   \n",
       "max    20070.000000  3.376892e+06  4.109372e+06  1.055286e+07  6.340207e+07   \n",
       "\n",
       "       ...        ee_002        ee_003        ee_004        ee_005  \\\n",
       "count  ...  5.932900e+04  5.932900e+04  5.932900e+04  5.932900e+04   \n",
       "mean   ...  4.454897e+05  2.111264e+05  4.457343e+05  3.939462e+05   \n",
       "std    ...  1.155540e+06  5.433188e+05  1.168314e+06  1.121044e+06   \n",
       "min    ...  0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00   \n",
       "25%    ...  2.936000e+03  1.166000e+03  2.700000e+03  3.584000e+03   \n",
       "50%    ...  2.337960e+05  1.120860e+05  2.215180e+05  1.899880e+05   \n",
       "75%    ...  4.383960e+05  2.182320e+05  4.666140e+05  4.032220e+05   \n",
       "max    ...  7.793393e+07  3.775839e+07  9.715238e+07  5.743524e+07   \n",
       "\n",
       "             ee_006        ee_007        ee_008        ee_009        ef_000  \\\n",
       "count  5.932900e+04  5.932900e+04  5.932900e+04  5.932900e+04  57276.000000   \n",
       "mean   3.330582e+05  3.462714e+05  1.387300e+05  8.388915e+03      0.090579   \n",
       "std    1.069160e+06  1.728056e+06  4.495100e+05  4.747043e+04      4.368855   \n",
       "min    0.000000e+00  0.000000e+00  0.000000e+00  0.000000e+00      0.000000   \n",
       "25%    5.120000e+02  1.100000e+02  0.000000e+00  0.000000e+00      0.000000   \n",
       "50%    9.243200e+04  4.109800e+04  3.812000e+03  0.000000e+00      0.000000   \n",
       "75%    2.750940e+05  1.678140e+05  1.397240e+05  2.028000e+03      0.000000   \n",
       "max    3.160781e+07  1.195801e+08  1.926740e+07  3.810078e+06    482.000000   \n",
       "\n",
       "             eg_000  \n",
       "count  57277.000000  \n",
       "mean       0.212756  \n",
       "std        8.830641  \n",
       "min        0.000000  \n",
       "25%        0.000000  \n",
       "50%        0.000000  \n",
       "75%        0.000000  \n",
       "max     1146.000000  \n",
       "\n",
       "[8 rows x 170 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verideki sınıfların dağılımına bakalım."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "59000"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train[df_train['class'] == 'neg'].shape[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1000"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train[df_train['class'] == 'pos'].shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "59000 tane negatif sınıfa ait veri varken 1000 tane pozitif sınıfa ait veri var. Bu dengesiz dağılımı göz önünde bulundurmak lazım."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Veri setimizde eksik olan değerlerin miktarına bakalım."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "class         0\n",
       "aa_000        0\n",
       "ab_000    46329\n",
       "ac_000     3335\n",
       "ad_000    14861\n",
       "          ...  \n",
       "ee_007      671\n",
       "ee_008      671\n",
       "ee_009      671\n",
       "ef_000     2724\n",
       "eg_000     2723\n",
       "Length: 171, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_train.isnull().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD5CAYAAADFqlkBAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA68UlEQVR4nO29eZwkVZ3o+/1lZmVt3V297w3dQAPSrNICKu6K4MyI8646eHXgMl6ZueosPudzxXvfu87m++ideTrDfW7cEYHRERydGRvF6UFQQWRrpBtoaOiim96X6qWquroqqzIjzvsjTkRGZmVW5V7L+X0/n/pk5ImIzBNZJ+J3fusRYwyKoiiK2ySmugOKoijK1KPCQFEURVFhoCiKoqgwUBRFUVBhoCiKogCpqe5ArSxevNisXbt2qruhKIoyY3j66aePGWOWlNo3Y4XB2rVr2bJly1R3Q1EUZcYgInvK7VMzkaIoiqLCQFEURalQGIjIqyLynIhsFZEttm2hiDwgIjvt6wLbLiJym4j0isizIvLa2OfcZI/fKSI3xdovt5/fa8+VRl+ooiiKUp5qNIO3GWMuNcZstO9vBR40xqwHHrTvAa4D1tu/W4CvQSA8gM8BVwJXAJ8LBYg95mOx866t+YoURVGUqqnHTHQ9cJfdvgt4X6z9bhPwODBfRFYA7wYeMMacMMacBB4ArrX75hljHjdBoaS7Y5+lKIqitIBKhYEB/l1EnhaRW2zbMmPMIbt9GFhmt1cB+2Ln7rdtE7XvL9E+DhG5RUS2iMiWvr6+CruuKIqiTEaloaVXG2MOiMhS4AER2RHfaYwxItL08qfGmNuB2wE2btyo5VYVRVEaREWagTHmgH09CvwLgc3/iDXxYF+P2sMPAGtip6+2bRO1ry7RrijTgr5To/zqlWNT3Q1FaSqTCgMR6RaRueE2cA3wPLAJCCOCbgJ+aLc3ATfaqKKrgAFrTtoMXCMiC6zj+Bpgs903KCJX2SiiG2OfpShTyqlMlg///ePc/K2nproritJUKjETLQP+xUZ7poB/NMb8m4g8BXxPRD4K7AE+aI+/H3gP0AsMAzcDGGNOiMhfAuFd9RfGmBN2++PAnUAn8BP7pyhTiu8bPnXvVl4+MoQGOyuznUmFgTFmF3BJifbjwDtKtBvgE2U+6w7gjhLtW4ALK+ivorSMH/x6Pz998ShnLupiz/FhjDFoCowyW9EMZEUpge8bbn94F+cvn8tvXxYEt/kasqDMYlQYKEoJfvFyHzuPDnHLm88iabUBX9cLV2YxKgwUpQS3P7yL5fM6+K1LVpJIBMLAU9VAmcWoMFCUIl7pG+KxXcf5T29cS1syQcJqBqoYKLMZFQaKUsTm7YcBuP7SlQBYxQBPpYEyi1FhoChFbN5+hEtW97CipxOAZEJ9BsrsR4WBosQ4NDDCtn39XLNhedQWhpMaf6p6pSjNR4WBosT49+1HALj2wrwwUDOR4gJOC4NjQ6P8rwd3YvQmVyybtx/mnKVzOHvJnKjNRTPRlldPcN+2g1PdDaWFOC0MHtpxlP/3gZfZf3JkqruiTAOyns+WPSd5y7lLCtrFwTyDux/bw9/8+0tT3Q2lhTgtDHwbN67x4woEIaVjOZ+LVvUUtIdmIt8hn4FnjFPCT3FcGIQ2YB30CsD2A4MAbFg5r6DdxQxk3zdOCT/FcWEQKgQu3eRKebYfHKSjLcFZMX8BECWduTROfNUMnMNtYeCHmsEUd0SZFmw/OMBrVsyLHMYhYTkKl2bKnq/mU9dwWxgY9RkoAcYYXjg0OM5EBDGfgUMzZWOMTpIcw2lh4KkDWbHsOzHCqUyODSt7xu1z0UykDmT3cFoYhGNdx7yy/eAAMN55DDEzkUMDxTduXa/iuDCIzEQ66J3n+YMDJBPCucvmjtuXNxO1uFNTiO8b1Zgdw2lh4KnPQLG8fGSIsxZ309GWHLfPRTORb4xqzI7htDDIm4l01LvOkcEMK+d3ltwXCgOXJg2eagbO4bQwUAeyEnJkMMOyee0l94VmIpfmDEZ9Bs7htDBQn4ECwWTg2NAYS+d2lNzvoplIo4ncw21hYDUCHfNuc/z0KJ5vymoGSQfXQPY1z8A53BYGdrC7dJMr4zk6OArAkjKagWg0keIATgsDT81ECnD0VAZgAp+Be2YiX4MrnMNpYRDe3Drg3eaI1QyWziutGUSL2zg0U9bgCvdwWxhEA36KO6JMKZGZaE5pzcBJM1FU3n2KO6K0DLeFgfoMFODIqQyLutOkU6VvByfXM9C1PpzDaWHg+WomUuDoYIYlc0trBeBubaLg1Z1rdh2nhYFRB7ICHD01yrIy/gJwtzYRqNbsEk4LA61NpECQfbx0Is1A3HMgq8/APSoWBiKSFJFnRORH9v06EXlCRHpF5F4RSdv2dvu+1+5fG/uMz9r2l0Tk3bH2a21br4jc2sDrm5B8+FyrvlGZbni+oW9SzcA9M1G0PrhKA2eoRjP4Y+DF2PsvAl82xpwDnAQ+ats/Cpy07V+2xyEiFwA3ABuAa4GvWgGTBL4CXAdcAHzIHtt0jGoGznP89Ci+gaVlcgwgLgxa1aupJ1zi0yUB6DoVCQMRWQ38BvD39r0Abwe+bw+5C3if3b7evsfuf4c9/nrgHmPMqDFmN9ALXGH/eo0xu4wxY8A99timE8VS64B3ljCstFxdIoCEvUtcmjRo3S73qFQz+FvgvwJhRP4ioN8Yk7Pv9wOr7PYqYB+A3T9gj4/ai84p1950NMtSCbOPK9EMXBon+YTMKe6I0jImFQYi8pvAUWPM0y3oz2R9uUVEtojIlr6+vro/T5POlDD7uDKfQUu6NC0I7wmXtCHXqUQzeCPwXhF5lcCE83bg74D5IpKyx6wGDtjtA8AaALu/Bzgeby86p1z7OIwxtxtjNhpjNi5ZsqSCrk+MqsLKidNjACzqTpc9JhmaiRwaJ0aTzpxjUmFgjPmsMWa1MWYtgQP4IWPMh4GfAe+3h90E/NBub7LvsfsfMsHI2gTcYKON1gHrgSeBp4D1Njopbb9jU0OubhI8NRM5z8iYR0KgvUz2MYA4aCbKRxNNcUeUlpGa/JCyfAa4R0T+CngG+KZt/ybwDyLSC5wgeLhjjNkuIt8DXgBywCeMMR6AiHwS2AwkgTuMMdvr6FfF+BpN5DwjWY+OtmT0wC+Fi6GloQnVpWt2naqEgTHm58DP7fYugkig4mMywAfKnP954PMl2u8H7q+mL41AsyyVjBUGE5GM1kBuRY+mB1HdLhUGzuB0BrJGTCiZrE/nJMIgX7XUnYGi5d3dw2lhEEVM6IB3lkzWo71t4tsgLFTn0oPR00g753BaGGgGspLJenSk1ExUTCj3XNKGXMdpYeCpKuw8mZxHZ3piYZBw0EykRRzdw2lhkF/cZmr7oUwdmaxPR4VmIpeEgfrT3MNtYaDhc84zMja5mci1EtbGmEgIqD/NHdwWBppl6TyZnEdHxWaiFnRoGhC/Tr033EGFAWoXdZnRrD+5ZuCYmSh+P7iiDSmuC4OoZvvU9kOZOoIM5El8Bo5lIMevU+8Nd3BbGKiZyHkyWW/SpDP3zET5C1Wt2R2cFgYaPuc2xpiKylEkojwDN8ZJ/DI17NodnBYG4aBXzcBNxjwf31CxmciVB2Nc6Gk0kTu4LQzC0FJHZnxKIZls4DSaXDMIXl0ZJkZ9Bk7itjDQxW2cZjTrAZMLg2TCLTORRhO5idPCwIuSzqa4I8qUUKlm4NriNppn4CZOC4OoGJdKAycZiTSDyW+DhLgzadBoIjdxWhhoNJHbZKwwmCy0FAJTkSvmRM0zcBOnhUE+z2CKO6JMCZkKfQYQmIpcMZkU+AwcuWbFdWGgheqcphozUVLEmQqeRn0GTuK2MIhKWOuAd5FKHcgQ+AxcGScFeQaOXLPiuDDwVDNwmtFc5WaihENmovh1OnLJCo4LA6O1iZxmZKwKYZBwx0yk0URu4rQw0Ggit6kmmsglM5HmGbiJ08IgX5toavuhTA2ZXOgzqCTPwB0zkUYTuYnTwiAyE6k0cJIotHSSxW0gMBO5Mkw0z8BNnBYG4QzIlWQipZCRrEc6lYhWMpuIhLgzaQgXfQJ3TGOK48JAzURuEyx5Wdkt4JKZqDCayI1rVlwXBlrC2mkqWdgmJCHulKPwNJrISdwWBhpN5DQjWY/OdIXCIOFOzL2uZ+AmTgsDT/MMnCaT9SpyHoNbZiIv5jNw5ZoVx4WBLnvpNpmsX1FYKQS1iVzRIAujidy4ZqUCYSAiHSLypIhsE5HtIvLntn2diDwhIr0icq+IpG17u33fa/evjX3WZ237SyLy7lj7tbatV0RubcJ1liT0FbhykyuFjFThMxBxx0zkF9QmmsKOKC2lkmnRKPB2Y8wlwKXAtSJyFfBF4MvGmHOAk8BH7fEfBU7a9i/b4xCRC4AbgA3AtcBXRSQpIkngK8B1wAXAh+yxTUdLWLvNaBXCIJlwx0ykGchuMqkwMAFD9m2b/TPA24Hv2/a7gPfZ7evte+z+d0iwbuD1wD3GmFFjzG6gF7jC/vUaY3YZY8aAe+yxTcUYo2Yix6nGTJRwyEwUjybSSDt3qOhOsDP4rcBR4AHgFaDfGJOzh+wHVtntVcA+ALt/AFgUby86p1x7U4k//125yZVCMjmvorpEEC5u0+QOTRP8gnIUU9gRpaVUJAyMMZ4x5lJgNcFM/vxmdqocInKLiGwRkS19fX11fZan4XPOMzJWjZnInQSsgqqljlyzUmU0kTGmH/gZ8Hpgvoik7K7VwAG7fQBYA2D39wDH4+1F55RrL/X9txtjNhpjNi5ZsqSaro/DV1XYeTTprDRxTdkVAahUFk20RETm2+1O4F3AiwRC4f32sJuAH9rtTfY9dv9DJhhRm4AbbLTROmA98CTwFLDeRielCZzMmxpwbRPiayy182RyfhXRRA6ZidSE6iSpyQ9hBXCXjfpJAN8zxvxIRF4A7hGRvwKeAb5pj/8m8A8i0gucIHi4Y4zZLiLfA14AcsAnjDEegIh8EtgMJIE7jDHbG3aFZVBV2G083zCWqybPwJ1ZslYtdZNJhYEx5lngshLtuwj8B8XtGeADZT7r88DnS7TfD9xfQX8bhpqJ3KaaJS/BrWgiTTpzE2czkAvNRFPXD2VqyGTtwjZatXQcBYvb6M3hDO4KA63M6DQj4ZKXVRSqc2WYFIRdOyIAFYeFgaeqsNNEq5xVYSZyZZZcGE00hR1RWoqzwkDtom4TCoN2rVo6DtWa3cRdYWB9BiJajMtFQp9B5WYil0JLdaLkIu4KAzvI2xIJHfAOMjwWVFKptBxFQtx5MGqhOjdxVhiE6m8q6Y76r+Q5OjgKwJK57RUdn3TITBTeGwkpjLpTZjfOCoPwvk4l3IkfV/IcHswAsHxeR0XHi4gz5sQwuS6VTGg0kUM4KwzCQZ5OJZyJElHyHBnMMK8jVbnPwKEM5HBy1ObQGg6Kw8IgHOSpRMIZx6CS5/BAhuU9lWkF4ObiNqmkTpRcwl1hEPMZqCrsHkcGMyyr0EQEbpajaEu6E0GluCwM7CBv09mPkxwZHK1KGDi1BnJMa9aJkjs4KwyiaCKH1H8lwPMNfUOjFTuPwS0zUegoTyXFGT+J4rAw8GMRE75xxzmowLGhUTzfsKwKn4FLi9vkzUQJZ0xjigoD2pJi309lb5RWcnigurBSCMxErsTc+wVa8xR3RmkZDguD4DWVCIWBjnpXqDbHAIKkM1e0R40mchOHhUHeTARakMsljlhhsKynsuxjcMtM5Bn1p7mIu8LALzYT6aB3hcMDGVIJYXF3FcLAqfUMDCJBcT7PkWtWXBYGkZkoUfBemf0cHsywdG47CWsirISEQ2YizzckRZxa91lxWBhEKfdqJnKOI4OZqiKJwLWks+B6XbpmxWFhYIqjiXTQO8PhgUxVzmMIS1g3qUPTDN8YEolwDQdHLlpxVxh4RQ5kHfTuUG32Mbj1YPR9YzUDd8JpFYeFQVSOwtqNXYkUcZ2RMY+h0VzF6xiEOLUGsrE+A4cEoOKyMIgVqgveT2VvlFYRrn1c6QpnIUmHErCMCZLsXAqnVVwWBmomcpKslfptqeqGvji07KXnG5KJwIHsigBUHBYGYZREWqOJnCLn5RduqYaEQ8te+ibuM3DjmhWHhUFxOQpH7nPnCYVBqBFWStKhWXIQTaQ+A9dwWBgUlaPQQe8EkZkoWa1m4I726PvB9YrmGTiF88IgfCjooHeDSDNIVOszCDXI2T9OomgiEdWYHcJZYZBf3Cb4CVy4yRXI2pVbUlVqBsmEO6XOfWMQERIJ1ZhdwllhEI7x8KGgg94NckUFCisl9De7oEH6BdFEs/96lQBnhYGaidwkF2oGNZqJXHg4BrWJ3Eq0UyoQBiKyRkR+JiIviMh2Eflj275QRB4QkZ32dYFtFxG5TUR6ReRZEXlt7LNussfvFJGbYu2Xi8hz9pzbJLzzmsh4M1Gzv1GZDozVaSZyYZx4BdFEU90bpVVUMj3KAZ82xlwAXAV8QkQuAG4FHjTGrAcetO8BrgPW279bgK9BIDyAzwFXAlcAnwsFiD3mY7Hzrq3/0iYmvKlVM3CLKM+gytDSyEzkgDQwNs9AHIqgUioQBsaYQ8aYX9vtU8CLwCrgeuAue9hdwPvs9vXA3SbgcWC+iKwA3g08YIw5YYw5CTwAXGv3zTPGPG4CL+7dsc9qGp4pfCi4cJMrkPNDM1H1SWfghpkov56BO2s4KFX6DERkLXAZ8ASwzBhzyO46DCyz26uAfbHT9tu2idr3l2gv9f23iMgWEdnS19dXTdfHUZxnoIPeDbI1awbWTORADStfaxM5ScV3hIjMAX4A/IkxZjC+z87omz5qjDG3G2M2GmM2LlmypK7PKl720nPgJlfUTFQJUTSR+gycoqI7QkTaCATBd4wx/2ybj1gTD/b1qG0/AKyJnb7atk3UvrpEe1MpXvZSbaNuEJmJas4zmP3jRGsTuUkl0UQCfBN40RjzpdiuTUAYEXQT8MNY+402qugqYMCakzYD14jIAus4vgbYbPcNishV9rtujH1W0/CKSlirmcgNIjORhpaWxTNobSIHSVVwzBuB3wWeE5Gttu2/AV8AviciHwX2AB+0++4H3gP0AsPAzQDGmBMi8pfAU/a4vzDGnLDbHwfuBDqBn9i/pjIuz0AHvRPkagwtjRzIDpgTg2git9Z9VioQBsaYXwLl7px3lDjeAJ8o81l3AHeUaN8CXDhZXxqJXxxNpIPeCbJFGmGlhC4GF2bKYTRRQmsTOYXDGcjBqyaduUWoGaiZqDxxn4FqzO7grDDwxkUT6aB3gfx6BmomKofvQyKB+gwcw1lhYIryDHQG5Ab59QyqXNzGITORH2UgixPCTwlwVhiEeQX5lc5m/02uQDYX1qTSDORyeCbIM0gm3LheJcBZYTDegTyVvVFaRZhnkKxSGLjlMwiuVzOQ3cJZYWCMQSSv/uugd4OsZ2hLSvRwr5RkJAya0avphe8bkja01BjVml3BWWEQLu3n0nKGShBNVK2/ABxb3CaKJnJHACoOC4NgAQ+JZnwu3ORKsNJZtf4CcMtM5PnhegbBexeuWXFZGPihmUiFgUtka9QMXFrcxtiVzkQnSk7hrjCwEROh6diFm1wJ8gyqzTEAt8xE+WgidwSg4rAw8HxrJgo1Ax3xTpD1/arXP4agcBu4YTLxjbHRRMF7vTfcwFlh4NtiXOozcIucjSaqFpecqX6sNhG4IQAV14VBQqOJXCPn+1HWeTWEs2QXHoy+9RnkS3DM/mtWKithPSvxbWhp3IG85/hp2pIJVs7vnOLeKc0i69UWTZRs0oPR8w0PvHCYoVEvatuwch6vWTGvod8zEc/tH2Ddkm7mtKeiPiViPgOVBW7grDDw/CBaIjITGfj097axZG47X/vI5VPcO6VZ1JpnIE0yE23d188ffPvXBW3nL5/Lv/3Jmxv7RWXIej7/4eu/4tPvOpfff8vZQLiegTjlNFccFgbGGJIJkET+/cnhsZoeFMrMIVtnNFGjzUSDmSwAX//I5WxYOY+/+vELPH9gcJKzGkcm6zGW8+kfyUZtYUJmQut2OYWzTz7PN+OSzjJZn5GsN8mZykwm6/lVr2UAzVsDOTMWjLczF3WxZmEXC7vTjLWwUNZoLviuTGzc+yYoYZ0QjbRzCWc1gygDOWYXHcl6BTeFMvvI+YaOtuljJsrkgvHW0ZYEoD2VZLSFYzAvDPICyC+aKDXimgczWY4PjdV07tK57XS3O/uoahnO/sJBNBFR0plvDCNjXnRzKLOTnOeTquHBEpmJGiwNwodwKKDaU4mWjsFQ8BRqBuF6BvZ9A675ur99hAP9IzWde9GqHu77w6vr7oMyMU4Lg2Rs9pPzjGoGDpCtMc+gaWYiO946I80gEAbGJn41m1JmIs8vzEBuxDUfHszwrguW8RsXrajqvH96eh8vHR6q+/uVyXFWGEQ+AzvgQ1+B+gxmN7laM5CbZCYKx1tkJrKvY55PeyrZ2C8rQSgM4uPemEBjTjQoITPr+Xi+4eJVPbzvslVVnfvcgQG27Ruo6/uVynDWgWwMBUlnw2M5ANUMZjm11iaSJoVZhmai9lSi4LVVpqJSZqLiaKJ6Lzm8llDgVUNnW5KRrKcRTS3AWWEQlqOAwAQwNBoKA18H3iwm69dbtbSxY2M069GeSkSTknQoDLItEgaRZhBzINvs/EaF04aCpr0Gx31nOonnG7Ke3pPNxllhEJqJIMguHY5lgKoTefaSqzEDuVlmokzWozOdnzGHmkGrwkvDsR6PYPJ9iqKJGiMMOmowe4XahJpvm4+zwiAMLYXABHDamolATUWzmSDprHafQaNj7keyXsFDMvQTtCq8dDRXLpqocesZhAKnJs3ACgO9J5uPw8IgCC2FQjMRFMZcK7OLnO/XWLU0eG20mSiT9QvyHlrvMxjvQG70egaZIid5NXRZrWl4TIVBs3FaGCTLmIl0FjJ7CcxEdWgGDXcgewUPyXD23DJhUJR0Zoyx0USNq01U7CSvhshMpMKg6TgrDDw/H8ctQqFmkNOBN1sJlr2sJ8+gsf3J5PxCYTBFZqJQMwivrzCaqF4zUe2aQehPUZ9B83FWGBiTv8GTCSnwGegsZPaS9fy6QkubUZtoSs1E9nvGcj6+b6LrK1jPoF5hkK0vtBRUW28FzgqDIJoo2E4mis1E6jOYjQQPO2oKLW3WQi+ZnFdaM2ixzyD8ztAklEg0rjZRFFpag5moU81ELcNZYRDWX4HAPhoP5VMz0ewk6wf/43ryDJoRWhqPJgrzDMZaphnkx3qQ3BVsN3I9g7qSztKJqG9Kc5n0rhCRO0TkqIg8H2tbKCIPiMhO+7rAtouI3CYivSLyrIi8NnbOTfb4nSJyU6z9chF5zp5zm7SiIAuFwiBZ9JWtrBqptI6cTVyqJc+gaWairF8yz2C0RROSuBacyXpR6GwyQcN8BvloojocyHpPNp1K/jt3AtcWtd0KPGiMWQ88aN8DXAest3+3AF+DQHgAnwOuBK4APhcKEHvMx2LnFX9XU/BjPoPiZ4MOvNlJJAzqMRM1I88g7jNoeTRRoWaQ9xlIzDRW33fkzURu+Aw+/+MX+IfH90x1N6pm0rvCGPMwcKKo+XrgLrt9F/C+WPvdJuBxYL6IrADeDTxgjDlhjDkJPABca/fNM8Y8boIA7rtjn9VUgmiiYDtRJA3UZzA7yZuJps8ayJmsV/CQrCSaaM/x0+w9PtyQ748LnUzWi64vKOIYtNetGURmotrKUcDM8hn8868P8Bf3bWfH4datWNcIavUZLDPGHLLbh4FldnsVsC923H7bNlH7/hLtJRGRW0Rki4hs6evrq7HrAcYm1sB4DWEmzUKUysmbiaZP1dLRbHFoaXnNwBjDN3+5m3d96WE+/U9bG/P9xcIg8hnEMpAbFE1Ui2YQ+lNmStKZMYb+kSxZz/CZ7z9LroWr1tVL3Q5kO6NvSRUpY8ztxpiNxpiNS5YsqeuzvJjPIHzt6WwDVDOYrWTtjVlTaGmDZslxPN8w5vmRKQQmFgb3PrWPv/zRCwAcP13bqmHFxDWQTDYfTZSMRRPVm3WdyXm0JfMZzdWQSAgdbYkZM0E7NZrD8w2vPWM+2/YP8OPnDk1+0jShVmFwxJp4sK9HbfsBYE3suNW2baL21SXam05YjAvyGsH8rjSgPoPZSs4+6OoyEzVQGJRyrKaSCZIJKelA3nXsNB1tCd532UpOx5Ik62E05xdoxOGDX2I+g3ont8URU9USlrGeCQwMZwH4bbtuw8H+zFR2pypqFQabgDAi6Cbgh7H2G21U0VXAgDUnbQauEZEF1nF8DbDZ7hsUkatsFNGNsc9qKsUlrCEYdO2phEYTzVJClX26mInK1expTyVKhpYOjeaY056iuz1VkBdTD6M5L9KIRwqiiSSq3VV/BrIfLdpTC51tyRnjM+i3wmDZvA4SQsOEdiuYdKUzEfku8FZgsYjsJ4gK+gLwPRH5KLAH+KA9/H7gPUAvMAzcDGCMOSEifwk8ZY/7C2NM6JT+OEHEUifwE/vXdPyYzyC80TvTSTrTyRmjkirVEdbEr6lQnX0wNrI2UTnHarrMOsjDozm60inmtKc4PZZryNKYozmfns42Tg5nyWT9Ap9BoxLtMkURU9XSkZ45msHJ4cB8t7A7TXc6VVDmZrozqTAwxnyozK53lDjWAJ8o8zl3AHeUaN8CXDhZPxpNvIR1JAzaknSkkuozmKXk/Po1g0ZWLZ1IMyi1uM3pMY/u9hRd6RS+GZ+jUAujWZ+erjQcHw5CSwuiiRq00lnWryn7OKSzbeZM0PpHAs1gflcb3e2pGaUZuJuBHAstDQd9R1uSjrbEjJmFKNUROpDbangwNcNMFJo+xguDZEmfwenRHN3pJHPag+Pj9bRqJW4mGh2XZxAcU3c0UVHJjWqZWT6DQDPo6UzT3Z6cMVFQ4LIwKDATBW0dbQk6ZtAsRKmOyExU00pnwWsjzUTlqnm2lzETxTUDaIw9OjQTgc1AjkUTNUobyhSFz1ZLZ3rm+AxODuc1gzntM8tM5KwwKAgtjTuQ25KRLVeZXdSTgSwiiDTaTGR9BkWaSntbGWEwmqO7PUl3eygM6n9AjuZ85rQnSSbEZiAH7RLzGdS/noFXl5mooy1ZsEbzdKZ/OMuc9hRtyYSaiWYK8dDSZNyBPINimpXqCDOQa8kzgGC8NCOaqNju355KlowmCh3I3Y00E9kM6I5UwjqQY3kGjfIZ5OrUDNqSjDTgWltB/8hYpGl1q2YwM4iHlhY4kNVMNGvJRWai2oZ9UqShayCPTORALuUzGPOY0wQzUbs1jxbXJoqK801xNNFM8hn0D2eZ32WFQTrZEIHdKpwWBpHPwP4KHVE00cwYeEp15OrIQIbAdNLYpLPQTDS5z8AYw+nRHF3pJHMaZCYyxgTCIJWfBHklo4nqz0CupRRFyEzyGfQPj7HAJq8GZqKZ0W9wWBh4fr72SpR0lk7a1PeZYZ9UqiNbRwYyWDNRI/MMypR2TpcILR3zfHK+sQ7kxpiJwjU82lOJqOSDKZFnUK82FDiQ69AM0jMn3Lt/JEuP1QzUgTxDCArVBdtxM5Emnc1e6slAhmDS0JQM5BI+g2IzUTjD7I5pBsN1PmhC7aM9lYgeuKWiierPM6hTM2hLBsJwBhR9GxjOMj/mMxjL+VFI83THWWFQqlBdUI5i5tgnlerIRxNNFzORFQYVmIlC/0BXe4quyIFc3ziNqonGzKOl8gzq9hk0wIEcfs50JqxYGjcTAQ0rHdJsnBUGvh9b6SxMOksHttNS2Z/KzKeeZS+hGWaioEhcsdmqVGhpmLw0pz1FeypJW1LqdiCH2kdgJipyICca4zPwfcNYrr4M5I4ZsqZBWLF0fmQmCvo9NEOcyO4Kg4JyFEFbp81AHvP8hiYXKdODepa9hOaYiTrakuPqC5UKLQ1tz6G/oCtdfwx73EwUOJD90usZ1HHRoV+iIZrBNNfYw4ql8dBSmDnF6hwWBmV8BnbgtWoNWqV15NczqFUzaLCZKOcVrGUQUiq0dNjOLkN/QRC2WN8YjS9HGTqQI59BLJqonkuuZ/3jkPA3mu7m27BIXbGZaKY4kZ0WBsVmos50Ir8A9zRXSZXqCdczSNcoDESkocJgZKy0Lb09lSTrmYIZeehADnMMGpHdGmkGbYmoGFx4fdKg2kSZOlY5C+lMB/+v6V7npz9WigKgu4H5IK3AXWHg58tQhEIhnCHB9HdWKdWTzdWXZ5AUqXtx+DiZnEd7iRlz2trX46ai8IESZh93taca50BO5WtyhddXGE1UuzDI11+qrxwFTP8JWrxiKeT/VyoMpjkFGcgFeQYzwz6pVE+YZ1Crz6DRZqLRMiuA5Ze+zI/B0EwUmh7mtCcb6EDOV+vNRxM1Zj2DKLGuDp9BqA1N93uy35qJwhUTG5Uc2CqcFQaeMVFNomSBA7m1wuCXO4/NmDjkmU7O80klpOYFYRKJ6stRHBoYKVvcrtx6BKG2EI8oGoryDIIHTKMdyJ1thXkGiQbVJsr7JWa/z6C/nANZo4mmL8YYjMlHSxTXJoLWCIPeo0N85JtPcP8MWjR7JpPzTc0mIgjGSTWy4PkDA7zhCw/x2CvHS+4fKVOzJ7Svx0Och8dyJCRvbpnTnqrbhj6ay8/a24seuAXrGdQhDeLfUSudM8VMFKtYCnnNYKY4kCdd6Ww2kg+fKyphnU5G5YRbkf6+98RpAPYcH276dylBNFGtReqgejPRQzuOYgy8dOQUbzhn8bj9mawXZavGCWfRY17+4Xd61KM7nYomMF3pBpiJYrP28IEbfmZSpCFluxsRTdRhHcihoDrQP8LHv/NrMlUKh9+4eAV/9I71NfdjMk4O5yuWQvC7JhP154O0CkeFQZhyH7wPZ0ChIw1aoxkc6M8ErydHmv5dSpBnUJdmkJCqZsmP7OwDYH+Z/2+YZ1BMe4kJyenRXJR5DDaaqE7zQzyaKOxHqG2ElrREnZVa4+GrtVKcZ3DvU/t4dn8/11ywDKGy/+dzBwb4wa/3N00YeL7hV68c4+LV86M2EQlCgGeIz8BJYRDe0PFCdZ02+ScvDJqvGYRC4OCACoNWkPP9mnMMoDoz0alMll/v7QfKC/tyK4C1R7kuMWEwlots0BD4DjLZoF5PrdeU9xnko+hCYRD6C5J1ruGQicxEjYkmMsZw37aDXLVuEd/43Y0Vf8Zfb97BN36xq67fayKe3H2CI4OjvPeSlQXtM6lYnaM+g+A1HPBd6VQUDtZKZ9XB/uAhcaBfhUEryHqmpiUvQyYyE/30hSP82abt0fvHXjmO5xvmdaTY3x+YAX/87CG++G87omPK1fkvFU0UrH8cEwZWSxiuY5zGy1F0RppB8OAKTagi9UUTjTZAM2hLJmhLBiuxPX9gkN3HTvPeS1dOfmKMMxZ2kfMNhwYyNfdjIjZtO0hXOsk7X7OsoL1rBq125qQw8GLhcwD/5a1n862bXwfkZzCtMBOFwuBgf/mIE6Vx1DsrTEh5M9H3n97Pnb96NXqYPrLzGF3pJO/esDzSDO55ai9//8iuKHqsnJkonRofTRSsf1xoJoL6YtiL8wyCzws1A+xrfYl2mQY4kMPzh8c8Nm07QFtSuO7C5VWdf8bCbqA5/rmxnM9Pnj/Euy5YNi46rLsB+SCtwkkzUbwyI8DiOe0sntMO5FX01vgMRhAJzAUnTo+xyPZhIh575Ti9fUP87lVnNr1/05l9J4b5n5tfYjTrsXJ+Jx+9eh1rFnZNeE62AdFE5SbJLx05BcCuvtNcuKqHR3b2cdVZi1i3pJuTT2c5PZpjx+FTZD3Dq8dOs37ZXFvNcwLNoCiaaNncjuh9tKZBHfbo0ZxPOpVARKJw1lC4xCPt6ol8jjSDOsxEEGjsr/QN8dLhU7x5/ZIolr9SzlgUjI29J4YZzXn82aYXoryAejmVydE/nB1nIoLG5IMA7D0+zN/8+0tkPZ+ezja+8B8urvszi3FTGPiFwiBOeHN+69FX+dlLR/nr91/CyvmdDe9D1vM5MpjhvGVz2XH4FAf6RyoSBl/9eS9P7D7B72xcE80gQ57ec5Iv/mQHOd/n6vVL+D/fdW7D+z1d+NdnDnDftoOct2wuP3+pj28/vocNq3poSwifvuY8Xn/2onHn5Dy/5lIUEKyIV0qDy2Q9Xj0eRIb1Hh1ieU8Hrx4f5j9eeQbLe4Kx89yBAfpOjQKB4DhzUTeeb8oknY2vj3V61KNrcaHPIGivQzPI5ReqXzYvEDR7bIRbMhIG9WYgl17NrVo600ke2XmM7nSS//yms6o+f/m8DtqSwt4Twzy95yTffXIvaxZ21t2vkDetX8yb1i8Z196dTnF8qH5t5J+f2c99zx7knCVzotpHjcZNYVDkM4iTTia46fVnsvPoEI/2Huffnj/M7129ruF9ODKYwTfwurUL2XH4FAf7RwoiEUphjOHZ/QOM5Xx2HB4cd/z3n97Hswf6WT6vg9sffoVPvu2ccQJjtrB1Xz9nL+lm86fezKGBEf73w7vp7Rvi6VdPcM9Te8sIg/o0g3JrIO88MhT5oXqPDkX+p4tWzY9+/4d2HI2Of/nwKdYtDswWqxeOn2i0lylHMaeUmaiOiKJwyUuAdYu66elsY9u+AaAw7LouM1HWK1mmu1o+9qaz6Ds1yn96w1oWdFf/MEwmhDULuth74jTP7A1+u/s+eXXVGka1NMqBvHVfP+cuncvmT725Ab0qzex8UkxCfp3X8ftEhD+//kL+8WNXsXpBJ0/sLp0wVC8HbVjp69YtBPJhphOx5/gwA7b+ybZ9/eP2P7n7BG84ezG3XvcaMlmf5w6MP2Y2YIxh675+Ll2zAIAVPZ38j9+6gLt/7wreev5Sntp9ouR5Y55f8ypnEBaqG9++4/AgEGiVvUeHeP5A8EDdsGoeqxcED/ufvngEgIXdaXYcPsUTu4I+XrluvNAqlYE8POZFZRkg5kCux0yUza8zkEgIl66Zn086C30GdRbny9hVzmrN+g75yFVn8ql3nVuTIAhZs7CLvSeGeWZvP2ct6W66IIDGFBQ0xvDM3n4uO2N+YzpVBieFQajqJyaJLLly3SKe3H2iKc7dAzbC5IIV8+hKJyvKNdi2vx8IZjnPFAmDY0OjvNJ3mtetXcgVVsA8vqv0Q3Gms//kCMdPj3FpiZvjirULOTiQYf/J8ap5zjN1zVATZRKwXj5yinQqwRvPXkxv3xDPHRhg3eJu5nW0sWROO+lkgl19p1nYnebKdQt5+cgpntx9gtULOkuaIPNmokAYGGPGh5Y2RDMoLJQXf9jko4nq9BmU8YtMBWcu6mLP8WG27jvJZXYi0WwCYVCf/3H3sdMMjGRVGDQDz5T3GcS5ct1CTg5n6T061PA+hJrBqvnBA+FgBeGl2/YN0NGW4E3rF4/TDLa8Gjz4r1i3gIXdac5dNocny8yQZzqhILxszfxx+0JBWOrac359mkGyTNLZjsOnWL90Ducun8urx06zbd8AF67qAYIJx8r5gT3+vGVzOW/5XPacGOaxXcdLagUwPrR0xC5U3x2LVMn7DOpzIMdDPl97Rv4BGeUZlPGTVEq5iKmp4IyFXZzK5Dg2NNb0B2tIdzpYv7l4saJq2GrH+6VNFmBOCoPIZzCZMDjLzrCb8FA90D/Cwu40nekkq+Z3VpRrsG1/Pxeu7GHjmQt4pe90ZDICeHL3SdpTCS5aNR8IHopP7zk5IxYRr5ate/tpTyU4b/nccfvOWzaXeR0pnnp1/P8sW6fPoNx6Bi8fOcV5y+ZyzpI55HzD4cEMF66cF+1fvSCIZDlv+VzOXz4XY2BgJMuVVnAVUxxNNBRb/zikEeWRR4uWo7wkJlwLMpDrrFpaT5G6RnJGLNosLviaSbQOch0a3DN7+5nTnuKcpXMa1a2STI//UovxowzkiY87Y2EXy+a1N2WGfeDkCKusiaASzSDr+Ww/OMDFq+dHN+1z+wei/U+9eoJL1+QdlleuW8TQaI4XDg02vO9TzdZ9J7loVU/JtYwTCeF1axfyRBnNoNb1j6H0egb9w2McGRzlvOVzC27Wi6xmAET/5/OXz+XcZXkBFk42ihER0sn8OsihXyDuQA79B3WZibJewYO6p7MtuoZkLLS03qql00YzsOGlXekk5y5r7oM1pBHF6rbu6+fi1T0lA14aiaPRRGFtool/XBHhynWLeLT3GN96dDcJW7hLJKiIEr5PCAj5fcH6sfE1loN94VKCqaSw69gQF6wIZo+r5ndw/PQYP9txlDkdqah6ansqQf9wlr6hDKcyOTJZn0vW9ERRRD+21U6zvs8Lhwb5+FvPjvoezjpvf3gXl58ZzIIkdl3BKwXtYYMUvo3qvxQfX7yfcfvLfF6Zz2Wy82zL8wcHuXGCPIvXrVvIgzuO8sOtB5jTnmJFTycLu9Nksn7NaxlA4FQdsbP1sZzPd57Yw45DQX7BecvncnZMGGyICYPQiXze8rmcuaib9lSC+V1tBTPVYuJLX4YP/LgDOZkQOtoS/HLnsZqvZ++JYdYvK9SuLlszn96jQ7FoIth59BR3Prq7pu/Yfex0gUYzlayxGtrFq3uaUpKiFKFm8J0n9rKkgtDxYgzw4qFBfv8t1YfTVsv0+C8BInIt8HdAEvh7Y8wXmvVdxVVLJ+Idr1nKpm0H+fP7Xmh4P37z4iBJ5ZylwQ15851PTXrOpWvm09PZxvnL5/LdJ/fy3Sf3RvvecHa+MubSeR1cvLqHHz17iB89O/tKZF91Vml7O8DVtkLoH9+zddy+uPmmWi5YMY///chuNm8/zEMvHuXeLftICCzqTnPx6vlW8HSQTiUKqle+7fylbN3Xz2tWzCOZEK4+ZzFrFnZNGGHT3pZgYDjL0cFMFFwwp+iheuHKHrbsOcmWPSdrvqbiRKn3XLSCZ/cPRA+x1fO7eGzXcZ6NaaH1fsdU0d2e4g1nL6o6e7kezlzURTIhfO3nr9T8GSJw9TnjcxgajUyHMggikgReBt4F7AeeAj5kjCn7BN64caPZsmVLTd/Xe3SId37pF9z2ocsqGqinMllynsEQaBXG2DURYu/z7WAw+PaYQPAEr54frGubs68bVs6joy2JMYbeo0MMZrKMjPmMZD1Gsh6ZMY+erjaWzG3nxNAYyYTwtvOXAnD0VKYgtb6zLcmGlfMKHjA5z+f0qEfQ03xNpvA/Hv7v8+/D/YUHlttf9vOKhlTF5xUdT5nj08kEZy6a+GG65/hpTo96jOY8Dg1kooVH3rR+8aSZyuXIZD0++I3HeOHgIDnf8IdvP4dPX3NewTF/+9OX6WxL8vtvObvMp1TGW/76Z+NKJ9z3yau5aHVe42jEvTtZyGfO8zmVqS80sqezbdLIvdnM8FiObK72/1UqKQWRZPUgIk8bY0pW+JsumsEVQK8xZheAiNwDXA80fDp+87ee5KR9MFQ6Pud2jK8530hEZJy6PhlL53awNFaeoBSpZIKeLifdQpy5qDvavqxBn9nRluTrH7mc/+Orv+L1Zy/iU+8cn+H9JyXaauFvf+fSAn/P3I42NhRpNfXG7ldCKpmoK7Zfsea9GfATThdhsArYF3u/H7iy+CARuQW4BeCMM86o6YuyNtb8TesXc8kkGb+KUszK+Z088pm31eWIroTLzljAZS2KeFEUmD7CoCKMMbcDt0NgJqrlM779n8fJGEWpimYLAkWZCqbLqD4ArIm9X23bFEVRlBYwXYTBU8B6EVknImngBmDTFPdJURTFGaaFmcgYkxORTwKbCUJL7zDGbJ/kNEVRFKVBTAthAGCMuR+4f6r7oSiK4iLTxUykKIqiTCEqDBRFURQVBoqiKIoKA0VRFIVpUpuoFkSkD9hT4+mLgdrLPU4NM63PM62/oH1uFTOtzzOtv1C+z2caY0pWvZuxwqAeRGRLuWJN05WZ1ueZ1l/QPreKmdbnmdZfqK3PaiZSFEVRVBgoiqIo7gqD26e6AzUw0/o80/oL2udWMdP6PNP6CzX02UmfgaIoilKIq5qBoiiKEkOFgaIoijJ7hYGI/JmI/OlU90NRFGUmMGuFwWxCRBaKyAMistO+LrDtIiK3iUiviDwrIq+NnXOTPX6niNzU4v6uE5EnbL/utWtUICLt9n2v3b/Wtn9YRLbG/nwRubTK71wrIs/X2e+qf08RuVxEnrPn3CatWJi4sM9V9avcWHKlv7VOEkXks7ZvL4nIu2Pt19q2XhG5NdZe8h6Y1hhjZsUfcCPwLLAN+Afgz4A/tfs+RrCAzjbgB0CXbf8A8Lxtf9i2bQCeBLbaz1s/Da7tfwK32u1bgS/a7fcAPwEEuAp4wrYvBHbZ1wV2e0EL+/s94Aa7/XXgv9jtjwNft9s3APeWOPci4JUavnMt8Hyd/a7697Rj5Sp7zk+A61r4O1fdr3JjyZX+xp8LVZxzgX1GtAPrgFcI1l1J2u2zCJa83wZcMNE9MJ3/prwDDRpkG4CXgcWxQRf904FFsWP/CvhDu/0csMpuz7ev/wv4sN1OA50N6N+/Ak8D24FbbNsQ8GXb9iCwZILzXwJW2O0VwEt2+xvAh4qPAz4EfCPWXnBcLX0G3ksgILfa79ld5jwhSINP2fevBzbb7c3A6+12yh4nRef/P8Dna/iN1wI7gO8ALwLfB7qAV4E/B35t/9/nT/AZVf2edt+OWHvBcRX2u3gScydwG/Argofl+yc4t+p+lRtLdfb3/bH9Q9Opv/a8/07wfPgl8F3gT4GfAxvt/sXAqxOc/1ngs7H3m+24jsZ2/LiJ7oHp/DdbzERvB/7JGHMMwBhzomj/hSLyiIg8B3yYQHgAPArcKSIfI5DyAI8B/01EPkNQx2OkAf37PWPM5cBG4I9EZBHQDWwxxmwAfgF8boLzlxljDtntw8Ayu70K2Bc7br9tK9dec5+BR40xlxpjLiV4EPxNmfMWAf3GmFyJ7476ZfcP2OPj/A7BDVsL5wFfNca8Bhgk0EQAjhljXgt8jeBBUI5qf89Vdru4vSJEZAPwfwFvN8ZcAvyx3bUCuBr4TeALNfa3XL/KjaV6+lspLe2v7fPlBFropQSa3+uq7DNUPy4mugemLbNFGEzGncAnjTEXEcwSOwCMMX9AMLjXAE+LyCJjzD8SzIJHgPtF5O0N+P4/EpFtwOP2u9YDPnCv3f9tgpt/Ukww1WhFckipPiMi/xUYMcZ8pdFfKCJXAsPGmFpt//uMMY/a7fhv+s/29WkCDWK6UG4S86/GGN8Y8wJVPvyqoYaxNNmkq6nUOPbfBPyLMWbYGDOIrq1eltkiDB4CPmBn3IjIwqL9c4FDItJGoBlgjzvbGPOEMeZ/AH3AGhE5C9hljLkN+CFwcT0dE5G3Au8kMI9cAjyDFUZFTDTIj4jICvt5K4Cjtv0AwYM6ZLVtK9deV59F5J0EfpY/mOD048B8EQmXVI1/d9Qvu7/HHh9yA7VrBTD+Nwzfj9pXj4mXeq329zxgt4vb62U0tj2RQ7qWfpUbS7WSwz5HRCRBYFqdzv0NifpN6fsxTrXjYqJ7YNoyK4SBMWY78HngF3Y2+6WiQ/5v4AkCs9COWPtf2wiG5wlstNuADwLPi8hW4ELg7jq71wOcNMYMi8j5BE4yCH7799vt/0hgzyzHJiCMvLiJQEiF7TfaKJirgAGrUm8GrhGRBTb64hrbVk+fzwS+AnxgItOZnb39LHZtxf0Nr+P9wEP2+PBB8kHgnir6WcwZIvJ6uz3Zb1qKqn5Pu29QRK6y0S83kr/WSphsEjMZtfSr3Fiqtb+vApfb/e8F2qZRfwEeBt4nIp0iMhf4Ldse7/f7S50YYxNwgwTRcOsItOQnCYJS1tvIoTTBZGbTJPfA9GWqnRaz/Y8gAuEnBE7NfyVwXL2VwIH8JYJopoeY2IG8iMDJvBP4KbAwfIYSPKBfIXCOboyd83tAr/27uQF9/hyBU2yr/bt/gvPPIrhZeoF/Atpte4d932v3nxU7563A43X8zmsJBP23bb9/QN6BHAYWbAR+PsFnVP172s983p7z/1HkEK+g3zeRj2i7kyocsrX0q9xYqqO/ywhMiduAL063/trPiDuQ/5HAb3Q+gSP8GYKgklcr+IxXCBza18Xa32M/+xXgv092D0znP61NNEWIyJAxZs5U90NRFAVmiZlIURRFqQ/VDKYRIvIV4I1FzX9njPnWVPRnMkTkXwiScOJ8xhhTjX+i5YjIzYwPi3zUGPOJqejPZIjIRQQx/XFGjTFXTkV/JmOm9TfEZhZ/sah5tzHmt6eiP61GhYGiKIqiZiJFURRFhYGiKIqCCgNFURQFFQaKoigK8P8DNeESI2oHgCoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_train.isnull().sum().plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Çok fazla boş değere sahip olan attributeları kaldıralım."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "row_len = df_train.shape[0]\n",
    "df_train = df_train[df_train.columns[df_train.isnull().sum() < (row_len*0.2)]]\n",
    "df_test = df_test[df_train.columns]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Geri kalan boş değerleri o sütunun ortalaması ile dolduralım."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = df_train.fillna(df_train.mean())\n",
    "df_test = df_test.fillna(df_test.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verisiz alanlardan kurtulmuş olduk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD5CAYAAAAqaDI/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUHUlEQVR4nO3cfbRldX3f8fenTECNyrOE8OBAoEkGNRhPUJexIchjGkQNiaCt02hl9YHa1kWXQ0kECVlLkjYkVoxStFJSA8bUOG2SEoRAWmqQO8qjijOiWYCjDg+BUi2G+O0f+wceLvdxzpm5XH7v11pn3b1/+3f2/p67976f3977zKSqkCT16++sdAGSpJVlEEhS5wwCSeqcQSBJnTMIJKlza1a6gO2xzz771Nq1a1e6DElaVTZt2nRfVe07u31VBsHatWuZmZlZ6TIkaVVJ8ldztXtrSJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNAkjpnEEhS5wwCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6N5UgSHJikjuTbEmyYY7luyW5si2/McnaWcsPTvJIkrOmUY8kaekmDoIkuwAXAycB64DTk6yb1e1twINVdRhwEXDhrOW/BfzppLVIkpZvGlcERwFbququqvoucAVwyqw+pwCXtelPAK9JEoAkrwO+CtwxhVokScs0jSA4ALh7bP6e1jZnn6p6DHgI2DvJc4F3Ae9ZbCNJzkgyk2Rm27ZtUyhbkgQr/7D4POCiqnpksY5VdUlVjapqtO++++74yiSpE2umsI57gYPG5g9sbXP1uSfJGmB34H7g5cCpSX4D2AP4XpL/V1Xvn0JdkqQlmEYQ3AQcnuQQhj/4pwFvmtVnI7Ae+AxwKnBtVRXw6sc7JDkPeMQQkKSda+IgqKrHkpwJXAXsAnykqu5Icj4wU1UbgQ8DlyfZAjzAEBaSpKeBDAPz1WU0GtXMzMxKlyFJq0qSTVU1mt2+0g+LJUkrzCCQpM4ZBJLUOYNAkjpnEEhS5wwCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNAkjpnEEhS5wwCSercVIIgyYlJ7kyyJcmGOZbvluTKtvzGJGtb+3FJNiW5rf08Zhr1SJKWbuIgSLILcDFwErAOOD3Julnd3gY8WFWHARcBF7b2+4CTq+rFwHrg8knrkSQtzzSuCI4CtlTVXVX1XeAK4JRZfU4BLmvTnwBekyRV9fmq+nprvwN4dpLdplCTJGmJphEEBwB3j83f09rm7FNVjwEPAXvP6vMLwOeq6tEp1CRJWqI1K10AQJIjGG4XHb9AnzOAMwAOPvjgnVSZJD3zTeOK4F7goLH5A1vbnH2SrAF2B+5v8wcCnwTeUlVfmW8jVXVJVY2qarTvvvtOoWxJEkwnCG4CDk9ySJJdgdOAjbP6bGR4GAxwKnBtVVWSPYA/BjZU1Q1TqEWStEwTB0G7538mcBXwReDjVXVHkvOTvLZ1+zCwd5ItwDuBx79ieiZwGPDuJDe31wsmrUmStHSpqpWuYdlGo1HNzMysdBmStKok2VRVo9nt/stiSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNAkjpnEEhS5wwCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6N5UgSHJikjuTbEmyYY7luyW5si2/McnasWVnt/Y7k5wwjXokSUs3cRAk2QW4GDgJWAecnmTdrG5vAx6sqsOAi4AL23vXAacBRwAnAh9o65Mk7SRrprCOo4AtVXUXQJIrgFOAL4z1OQU4r01/Anh/krT2K6rqUeCrSba09X1mCnU9xXv+2x184esP74hVS9IOt+6Hn8+5Jx8x9fVO49bQAcDdY/P3tLY5+1TVY8BDwN5LfC8ASc5IMpNkZtu2bVMoW5IE07ki2Cmq6hLgEoDRaFTbs44dkaSStNpN44rgXuCgsfkDW9ucfZKsAXYH7l/ieyVJO9A0guAm4PAkhyTZleHh78ZZfTYC69v0qcC1VVWt/bT2raJDgMOBz06hJknSEk18a6iqHktyJnAVsAvwkaq6I8n5wExVbQQ+DFzeHgY/wBAWtH4fZ3iw/Bjwz6vqbyetSZK0dBkG5qvLaDSqmZmZlS5DklaVJJuqajS73X9ZLEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNAkjpnEEhS5wwCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNAkjo3URAk2SvJ1Uk2t597ztNvfeuzOcn61vacJH+c5EtJ7kjy3klqkSRtn0mvCDYA11TV4cA1bf5JkuwFnAu8HDgKOHcsMP5dVf0Y8FLgVUlOmrAeSdIyTRoEpwCXtenLgNfN0ecE4OqqeqCqHgSuBk6sqm9X1Z8DVNV3gc8BB05YjyRpmSYNgv2qamub/gaw3xx9DgDuHpu/p7U9IckewMkMVxWSpJ1ozWIdknwa+KE5Fp0zPlNVlaSWW0CSNcDvA++rqrsW6HcGcAbAwQcfvNzNSJLmsWgQVNWx8y1L8s0k+1fV1iT7A9+ao9u9wNFj8wcC143NXwJsrqrfXqSOS1pfRqPRsgNHkjS3SW8NbQTWt+n1wKfm6HMVcHySPdtD4uNbG0kuAHYH/tWEdUiSttOkQfBe4Lgkm4Fj2zxJRkkuBaiqB4BfA25qr/Or6oEkBzLcXloHfC7JzUn+8YT1SJKWKVWr7y7LaDSqmZmZlS5DklaVJJuqajS73X9ZLEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNAkjpnEEhS5wwCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXMGgSR1ziCQpM4ZBJLUOYNAkjpnEEhS5yYKgiR7Jbk6yeb2c895+q1vfTYnWT/H8o1Jbp+kFknS9pn0imADcE1VHQ5c0+afJMlewLnAy4GjgHPHAyPJG4BHJqxDkrSdJg2CU4DL2vRlwOvm6HMCcHVVPVBVDwJXAycCJHku8E7gggnrkCRtp0mDYL+q2tqmvwHsN0efA4C7x+bvaW0Avwb8e+Dbi20oyRlJZpLMbNu2bYKSJUnj1izWIcmngR+aY9E54zNVVUlqqRtOciTwI1X1r5OsXax/VV0CXAIwGo2WvB1J0sIWDYKqOna+ZUm+mWT/qtqaZH/gW3N0uxc4emz+QOA64JXAKMnXWh0vSHJdVR2NJGmnmfTW0Ebg8W8BrQc+NUefq4Djk+zZHhIfD1xVVb9bVT9cVWuBnwa+bAhI0s43aRC8FzguyWbg2DZPklGSSwGq6gGGZwE3tdf5rU2S9DSQqtV3u300GtXMzMxKlyFJq0qSTVU1mt3uvyyWpM4ZBJLUOYNAkjpnEEhS5wwCSeqcQSBJnTMIJKlzBoEkdc4gkKTOGQSS1DmDQJI6ZxBIUucMAknqnEEgSZ0zCCSpcwaBJHXOIJCkzhkEktQ5g0CSOmcQSFLnDAJJ6pxBIEmdMwgkqXMGgSR1LlW10jUsW5JtwF9t59v3Ae6bYjk7ymqoczXUCNY5TauhRrDO+bywqvad3bgqg2ASSWaqarTSdSxmNdS5GmoE65ym1VAjWOdyeWtIkjpnEEhS53oMgktWuoAlWg11roYawTqnaTXUCNa5LN09I5AkPVmPVwSSpDEGgSR17hkZBEnOS3LWStchSavBMzIIepDk7CRbktyZ5IQVquHEtv0tSTaMtR+S5MbWfmWSXVv7C5Nck+TWJNclOXAJ21ib5PYJ69wrydVJNrefe7b2JHlfq/PWJD859p4Lk9zeXm+cZPttfY8ssnx9q29zkvVj7S9Lclur8X1JstBn2tHmO+4WOBbObG2VZJ/t3OayB3bbuc/n3Ac7ygLnyW5tfktbvra1r03ynSQ3t9cHp1ZMVa36F/AW4FbgFuBy4DzgrLbs7cBNbdkfAs9p7b8I3N7a/6K1HQF8Fri5re/wlf5s83zeda3u3YBDgK8Au+zkGnZp2z0U2LXVs64t+zhwWpv+IPBP2/QfAOvb9DHA5UvYzlrg9glr/Q1gQ5veAFzYpn8O+FMgwCuAG1v73weuBtYAP9iOn+dPWMMjCyzbC7ir/dyzTe/Zln221ZZW60kLfaaVOO4WORZe2vbh14B9tnO7T5zPO3Cfz7sPduDvc77z5J8BH2zTpwFXTutcmLeWHX3w7ISD8wjgy48fZG1HPnHgAHuP9b0A+Bdt+jbggDa9R/v5H4A3t+ldgWdPsc4/AjYBdwBntLZHgN9sbZ8GjgKuawfhaxdY19nA2WPzVwGvnLQm4K3Ab48tfztw0TzvfSVw1eya2gl2H7Bmdr+2nYPadICHl1DjWuBLwH8Bvgh8AngO8G6GP9C3M3wFLwus405g/za9P3Bnm/4QcPrsfsC/AX51rP3DwC8tUufswcghwGfacXYBCwfB6cCHxuY/1Nr2B740V7/5PtMS9/vsWp8yKFrOcTffsTDrvV9jGUEAnMNwXv8v4PeBs4Ajgb9stX+SBf5Qb8c+n3MfLFLjP+D7A8cPMQTi8W2/f45h4PPced670HnyxPnMMBi5r/Vfyw4KgmfCraFjgD+oqvsAquqBWctflOR/JrkNeDNDcADcAHw0ydsZdiAMO/DfJnkXw//J8Z0p1vnWqnoZMALekWRvhtHmtVV1BPB/GP5gHAe8Hjh/gXUdANw9Nn9Pa5uoJuAa4OQkP9CW/zLwkWXWsDfw11X12By13QK8oU2/Hnhe+z0s5keBD1TVjwMPM4yY3l9VP1VVLwKeDfz8Au/fr6q2tulvAPst8hluAU5M8px2O+NngYPmW3mSI4BfAY6pqp8A/iXwO8DvVtWLga3zvXeROg5o07PbF/pMC5qn1ncDJ7T5125nndM4Hh+v8WUMI+EjGUbwP9UW/WfgXVX1EoaAPXeB1Sx3ny/rMyT5ceCNwKuq6kjgbxn+vvwKcGxV/SQwA7xznlUsdJ48UUtb/lDrD3BIks8nuT7Jq+erb7meCUGwmI8CZ7YT8j3AswCq6p8w7LSDgE1J9q6qjzGcCN8B/iTJMVOs4x1JbmEY0RwEHA58F/gfbfltwPVV9Tdteu0Ut73UmvYHrgV+PsmPAT9QVbdNcXtnAT+T5PPAzwD3MpxAi7m7qm5o078H/DTws+3+6W0Mg4Ej5n33mBqGWQv+45mq+jPgT4D/zTAa/cwidc41GHlVey8Mo+4dZimfacxctc41KFpJrwY+WVXfrqqHgY0Mg6Y9qur61ucy4O8tZWXL/P0s1WuAlwE3Jbm5zb+D4fbZDa1tPfDCKW5zK3BwVb2UIWA+luT501jxMyEIrgV+8fGRZZK9Zi1/HrC1jXLf/Hhjkh+pqhur6t3ANuCgJIcCd1XV+4BPAS+ZRoFJjgaOZbjc+wng8wyB9DftIAX4HvAoQFV9j+GScD738uQR6oGtbRo1XQr8I4argf+0HTXcD+yRZM2sdqrq61X1hnYgn9Pa/noJ5c4+iQv4AHBqC/j/2GqfzzeT7A/Qfn5rkc9AVf16VR1ZVccxXJZ/eQl1Llb3fOar4942/ZT6mP8zLb/IOQZF21HnRMfjDrDcfb7czxDgsnaMHFlVP8ow0Lx6rG1dVb1tnvfPe56M19KW7w7cX1WPVtX9AFW1ieG5zN9d8LewRKs+CKrqDuDXgevb6Pa3ZnX5VeBGhlHPl8baf7N9G+N2hpHfLcAvAbe3NH8Rw6XoNOwOPFhV324j7VdMuL6NwGnt2wWHMFxdfHYaNVXVjQwH4Zv4/oh2LjcBh7dvPuzKcCm/sQXbnwOntn7rGUKVJPskefyYO5v5bzvNdnCSV7bpNzHcNwa4L8lzx7Y1n42tjifV09rf0r5J8grgoarammSXsYHFSxgGBH+2wPrnGozcwPA7gbEByDyuAo5Psmf7dsvxDPeLtwIPJ3lFkjDc2x+vfa7PtJin1DrXoGie98533M15LCyxnrn8BfC6JM9O8jzgZOD/Ag+O3Q75h8D1862AZe5z5tkHC6z/GuDUJC+AJ/b5rcCrkhzW2n4wyZx/qBc6T2bVfirD7eNKsm+SXdq6D2X4/d+1QI1LtyMePPh6yoOh3Ri+qfBFhge01wFHM/YAkVnfjGCBh4tt+TkMI4I7ad8kmUZNbdkG4IolrOPnGEbKXwHOGWs/lOEPxBaGB2a7tfZTgc3tPZc+3r7INtYyBPjvtVr/kOFh8QVtuzcwXLmct8A69mY4cTczPJTfq7UHuLit5zZg1NqfBXyhvf4SOHIJda7n+w9cP8oyHha397+1/b62AL881j5q6/0K8H6+/9/CzPmZlrjvZ9f6X1udtzM821jowfucx90Cx8I7GO5/PwZ8Hbh0iTWOPyz+GE99WPxHLPyweFn7fKF9sMA23sj3v2G4iWEwdQxDMN7aXgt96WO+8+RZbX5LW35oa/8Fhi9c3MzwMPrk5Z738738v4b0FEn+O8O3ha5Z6Vok7Xir/taQpifJHkm+DHzHEJD64RXB01iGf7l54azmr1bV63diDY9fYs/2mmoPrp4uklzM8G2dcb9TVQs99N5pkryYp36D6NGqevlK1DOfp8Nxt1RP930OkOSTDLcKx72rqhZ6BrFTGQSS1DlvDUlS5wwCSeqcQSBJnTMIJKlz/x/iJFTcsGUp3AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_train.isnull().sum().plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Veri Bölümü"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verilerimizi x ve y olarak bölelim. Train ve Test örneklerimiz farklı dosyalarda olduğu için split işlemi yapmamıza gerek kalmadı."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "x_train = df_train[df_train.columns[1:]]\n",
    "y_train = df_train[df_train.columns[0]]\n",
    "y_train = y_train.replace('neg', 0)\n",
    "y_train = y_train.replace('pos', 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_test = df_test[df_test.columns[1:]]\n",
    "y_test = df_test[df_test.columns[0]]\n",
    "y_test = y_test.replace('neg', 0)\n",
    "y_test = y_test.replace('pos', 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['aa_000', 'ac_000', 'ae_000', 'af_000', 'ag_000', 'ag_001', 'ag_002',\n",
       "       'ag_003', 'ag_004', 'ag_005',\n",
       "       ...\n",
       "       'ee_002', 'ee_003', 'ee_004', 'ee_005', 'ee_006', 'ee_007', 'ee_008',\n",
       "       'ee_009', 'ef_000', 'eg_000'],\n",
       "      dtype='object', length=146)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Feature Seçimi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Veri setine baktığımız zaman çok fazla attribute var. Bunların çoğu da anlamlı bilgi içermektedir. Anlamlı verininde etkisini azaltmaktadır. Bunları saptamamız lazım."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\dodo\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\feature_selection\\_univariate_selection.py:114: UserWarning: Features [79] are constant.\n",
      "  warnings.warn(\"Features %s are constant.\" % constant_features_idx,\n",
      "c:\\users\\dodo\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\sklearn\\feature_selection\\_univariate_selection.py:116: RuntimeWarning: invalid value encountered in true_divide\n",
      "  f = msb / msw\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "SelectKBest(k='all')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_select = SelectKBest(score_func=f_classif, k='all')\n",
    "feature_select.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Grafikte hangi numaralı özelliklerin daha etkili olduğu görebilmekteyiz. Buna uygun bir eşik değeri belirleyip bazılarını çıkartmamız modelimizin başarsını artıracaktır."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD4CAYAAAAO9oqkAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAATgUlEQVR4nO3df6xcd5nf8fenNkkX6NYO8bqpbdUpWFQGdUPWCq5YVRTYxElW66yEqkQr4rbpeiUSFSqkXQekZguLZNQudCNBqkBcnAoRaGAbC8xmXTcSQmpCbiDkJ6kvwbux5cReHBLUlWDDPv1jvlcMl3t9r++vc+bO+yWN7sxzzpl55syPz5zvOTM3VYUkabz9na4bkCR1zzCQJBkGkiTDQJKEYSBJAtZ23cBCXXzxxbV169au25CkkfLII4/8VVVtmF4f2TDYunUrExMTXbchSSMlyV/MVHeYSJJkGEiSDANJEoaBJAnDQJKEYSBJwjCQJGEYSJIwDCRJGAZSL23d91W27vtq121ojBgGkiTDQJJkGEiSMAwkScwjDJJsSfJAkqeSPJnkfa3+h0lOJnm0na4ZWubWJJNJnkly1VB9V6tNJtk3VL80yUOt/oUkFyz1HZUkzW4+WwavAB+oqu3ATuDmJNvbtE9U1WXtdBigTbseeBOwC/hUkjVJ1gCfBK4GtgM3DF3Px9p1vQF4Ebhpie6fJGke5gyDqjpVVd9q538EPA1sOsciu4F7qurHVfV9YBK4op0mq+rZqvoJcA+wO0mAdwD3tuUPAtct8P5IkhbgvPYZJNkKvAV4qJVuSfJYkgNJ1rfaJuC5ocVOtNps9dcBP6yqV6bVZ7r9vUkmkkycOXPmfFqXJJ3DvMMgyWuBLwHvr6qXgTuA1wOXAaeAP16OBodV1Z1VtaOqdmzY8Av/wlOStEDz+h/ISV7FIAg+V1VfBqiqF4amfxr4Srt4EtgytPjmVmOW+g+AdUnWtq2D4fklSStgPkcTBbgLeLqqPj5Uv2Rott8GnmjnDwHXJ7kwyaXANuCbwMPAtnbk0AUMdjIfqqoCHgDe3ZbfA9y3uLslSTof89kyeBvwHuDxJI+22gcZHA10GVDAceD3AKrqySRfBJ5icCTSzVX1U4AktwD3A2uAA1X1ZLu+PwDuSfJHwLcZhI8kaYXMGQZV9Q0gM0w6fI5lPgp8dIb64ZmWq6pnGRxtJEnqwFh+A9lfhJSknzeWYSBJ+nmGgbRAbmFqNTEMJEmGgSTJMNCYcEhHOjfDQJI0v5+jkDS76Vscx/df21EnS2vqfq2W+6Nzc8tAkmQYSJIMA0kShoEkCcNAkoRhIEnCMJAkYRhIkjAMRpI/rSBpqRkGkiTDQJJkGEiSMAwkSRgG0ry5416rmWGwQnwjkdRnhkGPGR6rn4+x+sIwkKbxDVrjyDCQJBkGo8BPqpKWm2EgSTIMJEmGgSQJWNt1A5o/9x2MhqnH6fj+azvuRJq/ObcMkmxJ8kCSp5I8meR9rX5RkiNJjrW/61s9SW5PMpnksSSXD13Xnjb/sSR7huq/luTxtsztSbIcd1aSNLP5DBO9AnygqrYDO4Gbk2wH9gFHq2obcLRdBrga2NZOe4E7YBAewG3AW4ErgNumAqTN87tDy+1a/F2TJM3XnGFQVaeq6lvt/I+Ap4FNwG7gYJvtIHBdO78buLsGHgTWJbkEuAo4UlVnq+pF4Aiwq0375ap6sKoKuHvouiRJK+C8diAn2Qq8BXgI2FhVp9qk54GN7fwm4LmhxU602rnqJ2aoz3T7e5NMJJk4c+bM+bQurTh/j0qjZN5hkOS1wJeA91fVy8PT2if6WuLefkFV3VlVO6pqx4YNG5b75iRpbMwrDJK8ikEQfK6qvtzKL7QhHtrf061+EtgytPjmVjtXffMMdUnSCpnP0UQB7gKerqqPD006BEwdEbQHuG+ofmM7qmgn8FIbTrofuDLJ+rbj+Erg/jbt5SQ7223dOHRdkqQVMJ/vGbwNeA/weJJHW+2DwH7gi0luAv4C+Jdt2mHgGmAS+GvgXwNU1dkkHwEebvN9uKrOtvPvBT4L/BLwtXaSJK2QOcOgqr4BzHbc/ztnmL+Am2e5rgPAgRnqE8Cb5+pFkrQ8/DkKSZJhIEkyDCRJGAaSJAwDSRKGgTQrf0pC48QwkKQO9O23qwwDSZJhIGl16Nsn7VFjGEiSDANJkmEgScIw0IhybFhaWoaBJMkwkCQZBtKq55Ca5sMwkCQZBpIkw0CShGEgScIwkCRhGEiSMAykkeAvcmq5GQaSJMNAkmQYaMQ4VCItD8NAkmQYSJIMA0kShoEkCcNAksQ8wiDJgSSnkzwxVPvDJCeTPNpO1wxNuzXJZJJnklw1VN/VapNJ9g3VL03yUKt/IckFS3kHtbp5dJG0NOazZfBZYNcM9U9U1WXtdBggyXbgeuBNbZlPJVmTZA3wSeBqYDtwQ5sX4GPtut4AvAjctJg7JEk6f3OGQVV9HTg7z+vbDdxTVT+uqu8Dk8AV7TRZVc9W1U+Ae4DdSQK8A7i3LX8QuO787kK3/GQqaTVYzD6DW5I81oaR1rfaJuC5oXlOtNps9dcBP6yqV6bVZ5Rkb5KJJBNnzpxZROuSpGELDYM7gNcDlwGngD9eqobOparurKodVbVjw4YNK3GTkjQWFhQGVfVCVf20qv4W+DSDYSCAk8CWoVk3t9ps9R8A65KsnVaXpFVhVIaSFxQGSS4ZuvjbwNSRRoeA65NcmORSYBvwTeBhYFs7cugCBjuZD1VVAQ8A727L7wHuW0hPkrrnT22PrrVzzZDk88DbgYuTnABuA96e5DKggOPA7wFU1ZNJvgg8BbwC3FxVP23XcwtwP7AGOFBVT7ab+APgniR/BHwbuGup7pwkaX7mDIOqumGG8qxv2FX1UeCjM9QPA4dnqD/Lz4aZJEkdmDMMpK5MDTcc339tx51oLg4NjT5/jkKSZBhIkgyD8+bmsKTVyH0GWjHz3Qdg4Eorzy0DSVoBff8OhmEgSTIMJEmGgUZA3zevpdXAMJAkGQbSXNwq0TgwDCRJhoEkyS+dSdKK6uuwo1sGkrSEzvfNvi9HyxkGkiTDQJJkGOg89WWTVtLSMgwkSYaBtFq5BafzYRhIkgwDSZJhIEnCMJAkYRhIkjAMtEp45Iy0OIaBJMkwkFaLLraO3CJbPQwDSZJhIEmaRxgkOZDkdJInhmoXJTmS5Fj7u77Vk+T2JJNJHkty+dAye9r8x5LsGar/WpLH2zK3J8lS30lJ0rnNZ8vgs8CuabV9wNGq2gYcbZcBrga2tdNe4A4YhAdwG/BW4ArgtqkAafP87tBy029LkrTM5gyDqvo6cHZaeTdwsJ0/CFw3VL+7Bh4E1iW5BLgKOFJVZ6vqReAIsKtN++WqerCqCrh76LokSStkofsMNlbVqXb+eWBjO78JeG5ovhOtdq76iRnqM0qyN8lEkokzZ84ssHVJ0nSL3oHcPtHXEvQyn9u6s6p2VNWODRs2rMRNStJYWGgYvNCGeGh/T7f6SWDL0HybW+1c9c0z1CVJK2ihYXAImDoiaA9w31D9xnZU0U7gpTacdD9wZZL1bcfxlcD9bdrLSXa2o4huHLquseUXedR3PkdXn/kcWvp54P8Ab0xyIslNwH7gN5IcA97VLgMcBp4FJoFPA+8FqKqzwEeAh9vpw61Gm+czbZnvAV9bmru2vHwxaNT4/6t1LmvnmqGqbphl0jtnmLeAm2e5ngPAgRnqE8Cb5+qj77bu+yrH91/bdRuSz0UtiN9A7gk/tUnqkmEgSctg1D7cGQaSJMNAy2/UPiFJ48gwkCQZBpIkw0DSHDzSbTwYBpIkw0CSlsKobz0ZBpIkw0CSZBhIkjAM5sWjKSStdoaBVhWDW1oYw0CSZBhIGj9TW49uRf6MYSBJMgwkSYaBJAnDQJKEYSBJwjCQRpLfp9BSMwwkSYaBJC3UatpCMwykHuvbm02fetHSMgwkSaztugGNBj8RSqubWwaSJMNAkmQYSJIwDCRJLDIMkhxP8niSR5NMtNpFSY4kOdb+rm/1JLk9yWSSx5JcPnQ9e9r8x5LsWdxdkiSdr6XYMvgXVXVZVe1ol/cBR6tqG3C0XQa4GtjWTnuBO2AQHsBtwFuBK4DbpgJE0urUt+9PaHmGiXYDB9v5g8B1Q/W7a+BBYF2SS4CrgCNVdbaqXgSOALuWoS9JY8CgWZjFhkEBf57kkSR7W21jVZ1q558HNrbzm4DnhpY90Wqz1X9Bkr1JJpJMnDlzZpGtS5KmLPZLZ79eVSeT/ApwJMl3hydWVSWpRd7G8PXdCdwJsGPHjiW7Xkkad4vaMqiqk+3vaeBPGYz5v9CGf2h/T7fZTwJbhhbf3Gqz1SVJK2TBYZDkNUn+3tR54ErgCeAQMHVE0B7gvnb+EHBjO6poJ/BSG066H7gyyfq24/jKVpMkrZDFbBlsBL6R5DvAN4GvVtWfAfuB30hyDHhXuwxwGHgWmAQ+DbwXoKrOAh8BHm6nD7eaRow77dQn7kg+PwveZ1BVzwK/OkP9B8A7Z6gXcPMs13UAOLDQXiRJi+M3kCVJhoEkyTCQJGEYSJIwDJacRzD0g4+DdH4MA0mSYSBJfdLVVu1if5tIq5xDLf029fgc339tx538vL72pdm5ZSBJcstAq5tbNtL8uGUgSXLLQDPzE7U0XtwykCQZBuqOXwyT+sNhIi2KhxAuLcNRXXHLQBozfdoi61Mv0L9+VpJhcA7j9MQYl/upnxmn57fm5jCRxsr0N7+FDG+ttjdQh/oEhoGmWW1vdJLmx2EiqTEINc4MgzHnG6D6biWeo+4/cZhIWnZLOSY/7m9YfbEcj0PXj61bBmOm6yec1CWf/7MzDEZYXzdt+9iTdD76+tpaToaBJMkwWE3G8dOMVh+fw90wDFYxw2FuriNpwDBYYV28+XT1huebrLRwK/26NQyWSd8/cfa5N3Wj78/ZLozTOjEMJEmGwWrU908zfe9P3VjO54TPt7n15hvISXYBfwKsAT5TVfs7bmlJ+CSUNAp6sWWQZA3wSeBqYDtwQ5Lt3XYlLZ8+f0hYyi03twJHRy/CALgCmKyqZ6vqJ8A9wO7lvtHV8kQd9SOUVsvjMB/jcj+XQp/W1Tg8R1NVXfdAkncDu6rq37bL7wHeWlW3TJtvL7C3XXwj8MwibvZi4K8WsfxKGYU+R6FHsM+lNgp9jkKPsLJ9/qOq2jC92Jt9BvNRVXcCdy7FdSWZqKodS3Fdy2kU+hyFHsE+l9oo9DkKPUI/+uzLMNFJYMvQ5c2tJklaAX0Jg4eBbUkuTXIBcD1wqOOeJGls9GKYqKpeSXILcD+DQ0sPVNWTy3yzSzLctAJGoc9R6BHsc6mNQp+j0CP0oM9e7ECWJHWrL8NEkqQOGQaSpPELgyS7kjyTZDLJvq77mZJkS5IHkjyV5Mkk72v1i5IcSXKs/V3fg17XJPl2kq+0y5cmeait0y+0gwC67nFdknuTfDfJ00n+WU/X5b9vj/cTST6f5O/2YX0mOZDkdJInhmozrr8M3N76fSzJ5R33+Z/a4/5Ykj9Nsm5o2q2tz2eSXNVln0PTPpCkklzcLneyPscqDHr+sxevAB+oqu3ATuDm1ts+4GhVbQOOtstdex/w9NDljwGfqKo3AC8CN3XS1c/7E+DPquqfAL/KoN9ercskm4B/B+yoqjczOHjievqxPj8L7JpWm239XQ1sa6e9wB0r1CPM3OcR4M1V9U+B/wvcCtBeT9cDb2rLfKq9J3TVJ0m2AFcCfzlU7mR9jlUY0NHPXsxHVZ2qqm+18z9i8Oa1iUF/B9tsB4HrOmmwSbIZuBb4TLsc4B3AvW2WPvT494F/DtwFUFU/qaof0rN12awFfinJWuDVwCl6sD6r6uvA2Wnl2dbfbuDuGngQWJfkkq76rKo/r6pX2sUHGXxvaarPe6rqx1X1fWCSwXtCJ302nwB+Hxg+kqeT9TluYbAJeG7o8olW65UkW4G3AA8BG6vqVJv0PLCxq76a/8Lgyfu37fLrgB8Ovfj6sE4vBc4A/60NZ30myWvo2bqsqpPAf2bwqfAU8BLwCP1bn1NmW399fl39G+Br7Xyv+kyyGzhZVd+ZNqmTPsctDHovyWuBLwHvr6qXh6fV4Djgzo4FTvKbwOmqeqSrHuZpLXA5cEdVvQX4f0wbEup6XQK0MffdDMLrHwKvYYahhD7qw/qbS5IPMRh+/VzXvUyX5NXAB4H/0HUvU8YtDHr9sxdJXsUgCD5XVV9u5RemNhHb39Nd9Qe8DfitJMcZDLG9g8HY/Lo2zAH9WKcngBNV9VC7fC+DcOjTugR4F/D9qjpTVX8DfJnBOu7b+pwy2/rr3esqyb8CfhP4nfrZl6n61OfrGXwI+E57PW0GvpXkH9BRn+MWBr392Ys29n4X8HRVfXxo0iFgTzu/B7hvpXubUlW3VtXmqtrKYN3976r6HeAB4N1ttk57BKiq54Hnkryxld4JPEWP1mXzl8DOJK9uj/9Un71an0NmW3+HgBvbUTA7gZeGhpNWXAb/KOv3gd+qqr8emnQIuD7JhUkuZbCD9ptd9FhVj1fVr1TV1vZ6OgFc3p673azPqhqrE3ANgyMMvgd8qOt+hvr6dQab3Y8Bj7bTNQzG5I8Cx4D/BVzUda+t37cDX2nn/zGDF9Uk8D+AC3vQ32XARFuf/xNY38d1CfxH4LvAE8B/By7sw/oEPs9gP8bfMHijumm29QeEwVF63wMeZ3B0VJd9TjIYc596Hf3Xofk/1Pp8Bri6yz6nTT8OXNzl+vTnKCRJYzdMJEmagWEgSTIMJEmGgSQJw0CShGEgScIwkCQB/x9/pRNRdB8g0gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.bar([i for i in range(len(feature_select.scores_))], feature_select.scores_)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Eşik değerini grafik yardımı ile 5000 belirledikten sonra bu değerin altında kalan sütunları kaldırıyoruz."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "threshold = 5000 \n",
    "remove_cols = []\n",
    "for i in range(0, len(feature_select.scores_)):\n",
    "    if(feature_select.scores_[i] < threshold):\n",
    "        remove_cols.append(x_train.columns[i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\users\\dodo\\appdata\\local\\programs\\python\\python39\\lib\\site-packages\\pandas\\core\\frame.py:4305: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  return super().drop(\n"
     ]
    }
   ],
   "source": [
    "x_train.drop(remove_cols, axis=1, inplace=True)\n",
    "x_test.drop(remove_cols, axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Outlier Analizi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Başlangıçta Train veri sayımız 60000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 60000)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.shape[0], y_train.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Boxplot ve Zscore yöntemlerini denedim. Çok fazla veri kaybı yaşadım. Çünkü bu yöntemler global olarak outlier analizi yapıyor. Veri setimde ise çok fazla local maksimumlar mevcut. Bu nedenle LocalOutlier analizi ile yakın komşuları arasındaki ilişkiye bakarak tespit etmek çok daha verimli sonuç verdi."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "lof = LocalOutlierFactor(n_neighbors=3)\n",
    "yhat = lof.fit_predict(x_train)\n",
    "mask = yhat != -1\n",
    "x_train.shape[0], y_train.shape[0]\n",
    "x_train, y_train = x_train[mask], y_train[mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Outlier çıkarıldıktan sonra Train veri sayımız."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(57699, 57699)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.shape[0], y_train.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  &&&&&&&&&&&&&&&& Sınıflama Algoritmaları &&&&&&&&&&&&&&&&&&"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  --------------------------------------------SGDClassifier---------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Supervised sınıflandırma algoritması olan SGD Classifier bir doğrusal sınıflayıcıdır. Stokastik gradyan kullanır. Stokastik gradyan inişi, tüm eğitim verilerini dikkate alan gradyan inişinden farklı olarak ağırlıkları değiştirirken yalnızca 1 rastgele noktayı dikkate alır. Bu tür stokastik gradyan inişi, büyük veri setleriyle uğraşırken gradyan inişinden çok daha hızlıdır."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model oluşturma ve eğitim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SGDClassifier()"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = SGDClassifier(loss=\"hinge\", penalty=\"l2\", max_iter=1000)\n",
    "clf.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test Verisinin Tahmini"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 0, 0, 0], dtype=int64)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_predict = clf.predict(x_test)\n",
    "y_predict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "cm = confusion_matrix(y_test, y_predict)\n",
    "df_cm = pd.DataFrame(cm, index=['pos', 'neg'], columns=['True', 'False'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Verimizde negatif örneğin çok daha fazla olmasından dolayı dengesiz bir dağılım var."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[15581,    44],\n",
       "       [  145,   230]], dtype=int64)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD4CAYAAAAn3bdmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAej0lEQVR4nO3de5wXdb3H8dd7F7xgCSjKMfAIlicPmhkkXlMTAzQRLDRM5aK5x0tqeSksC9Oy0srQoyYqCnhBwRQ0lQgo0wRBJG9o7vEKXiCBxVxULp/zx++L/RB297c3lhnez8djHvub73xn5jvs8Pl99zPfmVFEYGZm2VDW0g0wM7PSOWibmWWIg7aZWYY4aJuZZYiDtplZhrRq7h1cvHtrD0+x9Vw8982WboJtitp0UGM3UZ+Yc/HzKxu9v43NPW0zswxp9p62mdnGlLmucz05aJtZrpTnPGo7aJtZrshB28wsO3Iesx20zSxf3NM2M8uQvA+Jc9A2s1xxT9vMLEPKHLTNzLIj5zHbQdvM8sU9bTOzDMl5zHbQNrN88YVIM7MMKVe+HyzqoG1muZLzjraDtpnli4O2mVmGOKdtZpYhOY/Zub9N38w2M2UqfaqLpNGSFkl6ZgPLzpMUkjqkeUm6SlKlpKckdS+qO0TSi2kaUlTeQ9LTaZ2rpLr/TnDQNrNcacqgDdwC9P14oaSdgd7Aa0XFRwC7pakCuC7V3Q4YAewL9ARGSGqf1rkOOLVovfX2td7xldRsM7OMUD2mukTEw8CSDSy6EvgeUDy+sD8wNgpmAu0k7QT0AaZGxJKIWApMBfqmZdtGxMyICGAsMKCuNjlom1mu1KenLalC0pyiqaKu7UvqDyyMiL9/bFEn4PWi+QWprLbyBRsor5UvRJpZrtTnQmREjAJGlbxtqQ3wAwqpkRbhnraZ5YpU+tQAnwa6An+X9ArQGZgr6T+AhcDORXU7p7LayjtvoLxWDtpmlitNfCFyHRHxdETsGBFdIqILhZRG94h4C5gMDE6jSPYDqiLiTWAK0FtS+3QBsjcwJS1bLmm/NGpkMDCprjY4PWJmudKUPVFJdwCHAh0kLQBGRMRNNVR/ADgSqASqgWEAEbFE0qXA7FTvkohYe3HzDAojVLYGHkxTrRy0zSxXmvKOyIg4vo7lXYo+B3BmDfVGA6M3UD4H2LM+bXLQNrNcyfsdkQ7aZpYrfnONmVmG5DxmO2ibWb60ynnUdtA2s1zxo1nNzDIk7zefOGibWa64p21mliFlfrGvmVl2OD1iZpYh5U6PmJllh3PaZmYZ4vSImVmG+DZ2M7MMcXrEzCxDfCHSzCxDnNM2M8sQ57TNzDIk5zHbQdvM8iXvPe28p3/MbDOjekx1bksaLWmRpGeKyq6Q9LykpyTdI6ld0bILJVVKekFSn6LyvqmsUtLwovKukmal8jslbVFXmxy0zSxXWpVFyVMJbgH6fqxsKrBnROwF/AO4EEBSN2AQsEda51pJ5ZLKgWuAI4BuwPGpLsAvgSsj4jPAUuCUuhrkoG1mudKUPe2IeBhY8rGyP0bEqjQ7E+icPvcHxkfEBxHxMlAJ9ExTZUS8FBEfAuOB/pIEHAZMTOuPAQbU1SYHbTPLlTKVPkmqkDSnaKqo5+5OBh5MnzsBrxctW5DKairfHlhW9AWwtrz246tnAzcr/X92Axc8upAzJj9ZY50uPQ/mtHvmcMZ98xg6blqj91neegsG/uY2zp4yn2/d+SjtOu2yzvK2O+3MD55YygEnf7fR+7KWsXr1agYMGsr/nH3BOuU//eWVfOGAw1uoVflRn552RIyKiC8WTaNK3o/0Q2AVcFsTH0KtHLRrMe+eMdx66lE1Lt/qk2356o+v5o4zjuHafnsz4ZxBJW+7XaddGDr2T+uVdx94Mu8vX8ZVff6bmWNGcvh5l62zvM/wK3jxrw+VfhC2yRl7+wQ+3bXLOmVPPzufqnffbZkG5Ux9etoNJWkocBRwQkSsTY4vBHYuqtY5ldVU/g7QTlKrj5XXfnwNb3b+vTrnEVZULalx+eeOOp75U++l6s3CXz7vLVn80bK9+n2TU+/6G6fdM4ejfnItKivtn/qzvfox795xADw35W523f+wj5bt3utoli54hcWVzzXkcGwT8Nbbi/jzI39j4DH9PipbvXo1l//2Gi4454wWbFl+lNVjaghJfYHvAUdHRHXRosnAIElbSuoK7AY8DswGdksjRbagcLFycgr2M4CBaf0hwKRSjq+URl4uaVtJrSVNk7RY0omlHmRebd9lN7batj1Dx/6Jirtn8fn+hX+SDrvuzh5HHstN3zyY3x3zRWL1avbq982Strntjp9iefoSWLN6Ne+/W0WbdtuzRZttOPDUC/jLNZc22/FY87vsipFccM4ZlBV182698256HXIQO+7QoQVblh/lZaVPdZF0B/AY8FlJCySdAvwv8ElgqqR5kn4HEBHPAncBzwEPAWdGxOqUs/42MAWYD9yV6gJ8HzhXUiWFHPdNdbWp1JtrekfE9yQdA7wCfA14GLi1hgOtACoAjupYRo92+ezQl7Vqxaf26M6YYb1pveXWnDL+ryz4+yx23f8wPrVHdyomzASg1VZb8d6SRQB84+oJtO/clfLWrWm7039y2j1zAJg57mrm/X5Mjfs69Ns/ZuYtI/mw+r3mPzBrFjMefpTttmvPnt12Z9acuQC8vWgxD02dwbgbrm7h1uVHU95bExHHb6C4xsAaET8DfraB8geABzZQ/hKF0SUlKzVor633VWBCRFSplucfpmT+KICLd2+d27dsLn9rAdXL3mHlimpWrqjm1TmP0PGze4HEvHvHMe03F623zp1nHQsUctoDfn4Ttwxe98LT8kVvsO1OO7P87YWUlZez1SfbUr3sHTrt1ZNufb7GVy74OVt9sh2xZg2rPviAx2+7dqMcqzXe3HlPMf0vj/DwI4/xwYcf8q/33uOogSexxRat6X30NwBY8f77fOXo45g6+a4Wbm121Rab8qDUoH2/pOeBFcDpknYA3m++ZmXD89Pu48gfjaSsvJzy1lvQea99mDlmJIsrn2PQNXcz85aRvLdkMVu3bc8W23ySqjdeq3ObL0y/n70HnMSCeTPp1ufrvDxzBgA3n/jlj+oc+u0f8WH1vxywM+a8s0/nvLNPB2DWnLmMHnsH1191xTp1vnDA4Q7YjZTzmF1a0I6I4ZIuB6oiYrWk9ygMJM+1r/96HF32OYQ27Ttw7p9fZsbVl1DeqjUAc+4cxT9fep7Kv07h9ElziTVrmDvxZha9WEhVTR85gpNuehCVlbF61UoeuOTskoL2kxNHc8zlt3D2lPmsqFrKxHNPaNZjNMudnEdt/Xu0Si2VpNbA6cDBqegvwO8iYmVd6+Y5PWINd/HcN1u6CbYpatOh0RH3paNLjzm7Tl6ZuQhfanrkOqA1sPbv8ZNS2beao1FmZg3lnHbBPhHx+aL56ZL+3hwNMjNrlHwOVvtIqYe3WtKn185I2hVY3TxNMjNrOEklT1lUak/7AmCGpJfSfBdgWLO0yMysETIai0tWak/7UeB6YA2FxxReT+EuITOzTYp72gVjgeXA2nuovwmMA45tjkaZmTWUcv6+sVKD9p4R0a1ofoYkP7XIzDY5Ge1Al6zU9MhcSfutnZG0LzCneZpkZtZwTo8U9AD+JmntLX3/Cbwg6Wkg0rvSzMxaXkaDcalKDdoff7GlmdkmKecxu+Rnj7za3A0xM2sKWU17lKrUnraZWSZ49IiZWYbkvKPtoG1m+eL0iJlZhuQ9aOf8eVhmtrmRSp/q3pZGS1ok6Zmisu0kTZX0YvrZPpVL0lWSKiU9Jal70TpDUv0XJQ0pKu8h6em0zlUq4RvHQdvMckVlKnkqwS2sP+R5ODAtInYDpqV5gCOA3dJUQeGdA0jaDhgB7EvhJb4j1gb6VOfUovXqHF7toG1mudKUd0RGxMMUHpJXrD8wJn0eAwwoKh8bBTOBdpJ2AvoAUyNiSUQsBaYCfdOybSNiZhReITa2aFs1ctA2s1ypT3pEUoWkOUVTRQm76BgRa9+X9xbQMX3uBLxeVG9BKqutfMEGymvlC5Fmli/1uBAZEaOAUQ3dVUSEpI36Hlz3tM0sVzbCA6PeTqkN0s9FqXwhsHNRvc6prLbyzhsor5WDtpnlSlOOHqnBZGDtCJAhwKSi8sFpFMl+QFVKo0wBektqny5A9gampGXLJe2XRo0MLtpWjZweMbNcacrb2CXdARwKdJC0gMIokF8Ad0k6BXgVOC5VfwA4EqgEqkmvZIyIJZIuBWanepdExNqLm2dQGKGyNfBgmmrloG1muaKypksgRMTxNSzqtYG6AZxZw3ZGA6M3UD4H2LM+bXLQNrN8yfkdkQ7aZpYvDtpmZtkh5Xt8hYO2meVLE+a0N0UO2maWK3l/yp+Dtpnli9MjZmbZ4deNmZllidMjZmYZ4vSImVl2NOUdkZsiB20zyxenR8zMssM315iZZYl72mZm2eEhf2ZmWeL0iJlZdnj0iJlZljinbWaWIU6PmJllR96f8pfvryQz2/yUqfSpDpK+K+lZSc9IukPSVpK6SpolqVLSnZK2SHW3TPOVaXmXou1cmMpfkNSnUYfXmJXNzDY1Kisveap1O1In4GzgixGxJ1AODAJ+CVwZEZ8BlgKnpFVOAZam8itTPSR1S+vtAfQFrpVU+85r4aBtZvkilT7VrRWwtaRWQBvgTeAwYGJaPgYYkD73T/Ok5b1UyNX0B8ZHxAcR8TJQCfRs6OE5aJtZrkiqz1QhaU7RVLF2OxGxEPgV8BqFYF0FPAEsi4hVqdoCoFP63Al4Pa27KtXfvrh8A+vUmy9Emlm+1GP0SESMAkZtcDNSewq95K7AMmAChfRGi3JP28zypenSI4cDL0fE4ohYCfweOBBol9IlAJ2BhenzQmDnQhPUCmgLvFNcvoF16q3Ze9oXz32juXdhWRSrW7oFllNNOOTvNWA/SW2AFUAvYA4wAxgIjAeGAJNS/clp/rG0fHpEhKTJwO2SfgN8CtgNeLyhjXJ6xMzypbzBAzPWERGzJE0E5gKrgCcppFL+AIyX9NNUdlNa5SZgnKRKYAmFESNExLOS7gKeS9s5M6LhvRZFREPXLU314mbegWVSrGnpFtimaJuOje4mrxn55ZJjTtk5MzJ3J4572maWLzm/I9JB28zyxc8eMTPLEPe0zcwyxD1tM7MMqeOZIlnnoG1m+eL0iJlZhjhom5lliHPaZmYZ4p62mVmG+EKkmVmGOD1iZpYhTo+YmWWIe9pmZhninraZWYa4p21mliEePWJmliFOj5iZZYjTI2ZmGZLznna+v5LMbPOjstKnujYltZM0UdLzkuZL2l/SdpKmSnox/Wyf6krSVZIqJT0lqXvRdoak+i9KGtKYw3PQNrN8kUqf6jYSeCgidgc+D8wHhgPTImI3YFqaBzgC2C1NFcB1heZoO2AEsC/QExixNtA3hIO2meVLeXnpUy0ktQUOBm4CiIgPI2IZ0B8Yk6qNAQakz/2BsVEwE2gnaSegDzA1IpZExFJgKtC3oYfnoG1m+VKPnrakCklziqaKoi11BRYDN0t6UtKNkrYBOkbEm6nOW0DH9LkT8HrR+gtSWU3lDeILkWaWL/UYPRIRo4BRNSxuBXQHzoqIWZJG8u9UyNr1Q1I0tKkN4Z62meVL0+W0FwALImJWmp9IIYi/ndIepJ+L0vKFwM5F63dOZTWVN4iDtpnlSxONHomIt4DXJX02FfUCngMmA2tHgAwBJqXPk4HBaRTJfkBVSqNMAXpLap8uQPZOZQ3i9IiZ5UtZk4a1s4DbJG0BvAQMo9DZvUvSKcCrwHGp7gPAkUAlUJ3qEhFLJF0KzE71LomIJQ1tkCKaOR1TvXij5nssI2JNS7fANkXbdGz0nTFr7ju35JhT1u83mbsTxz1tM8sX38ZuZpYhDtpmZhmS82ePOGibWb64p21mliFNO3pkk5PvozOzzY/TI2ZmGeL0iJlZhjhom5llSJmDtplZdvht7GZmGeL0iJlZhnj0iJlZhrinbWaWIQ7aZmYZ4qBtZpYhHj1iZpYh7mmbmWWIR4+YmWVIznva+T66Rrrw4svY/7CjOGrgSbXWe+rZ+XT74iE8NHVGo/e5rGo5w077Dr2PHsSw075D1fLlAEx+4I/0O24I/Y4dzKAhp/H8Cy82el9Wf2++9TYnVZzDkV8/ia8OHMyY2yesV+dPf/4r/Y4bSv9BJ/O1E05lzpNPNXq/y6qWM+z0c+nd/3iGnX4uVcvfBdaeF0Ppd9wQBg09nef/UdnofWVeE72N/aPNSeWSnpR0f5rvKmmWpEpJd6aX/iJpyzRfmZZ3KdrGhan8BUl9GnN4Dtq1+Fq/I7nxml/XWmf16tX8auR1HLjfPvXa9qw5cxn+45+tVz7q5lvZv2cP/jh5PPv37MGom28FoPOnduLWG6/mvgljOf3UIfzop5fXa3/WNMrLyxn+3TN44O5x3Dnmd9x+1z1UvvTKOnX279mDyXfezKTxo7lsxHAuurT039WsOU8yfMRl65WPuvk29u/ZnT9OuoP9e3b/93nRKZ0Xd41J58UVjTq+XGjioA2cA8wvmv8lcGVEfAZYCpySyk8BlqbyK1M9JHUDBgF7AH2BayU1+Gqpg3Yt9umxN23bbltrnXHj76ZPr0PYfrv265TfOOZ2vn7Ct+h33BCuuu6mkvc57c9/ZUC/IwAY0O8I/jTjrwB03/tztN220Ja999qDt95eXJ9DsSay4w4d2OO/PwvAJ7Zpw65dd+HtRev+LrZp0walvOqKFSsozrDeOOYOvn5iBf2OG8pV140ueb/T/vIIA47qC8CAo/rypz8/AkD3z3+Ottt+EoC9P+fzAoDy8tKnOkjqDHwVuDHNCzgMmJiqjAEGpM/90zxpea9Uvz8wPiI+iIiXgUqgZ0MPz0G7Ed5etJg/TX+Y4489Zp3yRx57nFdfe52Jt97ApPE38+z8F5j9xLyStvnOO0vZcYcOAOzQYXveeWfpenUm3ns/Bx+4X6Pbb42z4I03mf/Ci3x+z27rLZs6/WH6fu1E/uec73PZiOHA2vNiARPHXc+k8aOb6bzYt+EHlBf16GlLqpA0p2iq+NjWfgt8D1iT5rcHlkXEqjS/AOiUPncCXgdIy6tS/Y/KN7BOvZV0IVLSu0B8rLgKmAOcFxEvfax+BVABcP3Vv6Li5MENbd8m7WdXjOT8c06j7GOPgnz0scd59LHZDBg0DIDqFSt45bUF7NNjb4496VQ+/HAl1StWUFW1nP7fGArA+eeczpcOWPc/nKT1LoTPnD2Xiff+gdtHX9tsx2V1e6+6mrPP/xE/OO8sPvGJbdZb/pXDDuYrhx3M7CfmMfK6m7jld1fy6MzZPDpzNgOOL/w1XV29gldeT+fF4P8pnBfVK6havpz+g04G4PyzT+NLB6zbKav9vLimeQ44S+pxITIiRgGjNrgZ6ShgUUQ8IenQJmlbEyh19MhvKXw73A6IQn7m08BcYDRwaHHldf4hqhd/PNjnxjPPvcC5wy8GYOmyKv7yyGO0alVORFBx8okMGjhgvXUmjLsBKOS075n8IL+45IfrLN9++/YsWvxPdtyhA4sW/5PtitIuz/+jkosu+QU3/O+vaN+ubbMdl9Vu5cpVnH3+j+h35Ffo3euQWuvu02NvXr/45yxZuqxwXgw7gUED+69Xb8LY64FCTvue+x7kFz/5wTrLaz8v/o+LLr2cG66+wucFNOWQvwOBoyUdCWwFbAuMBNpJapV6052Bhan+QmBnYIGkVkBb4J2i8rWK16m3Ur+Sjo6I6yPi3YhYnoJyn4i4E2hf18p5Nf0PE5j+wESmPzCRPocfyogLz+PwLx/MQQfsy92T/sB71dVAIY3yzpL1/5zdkMMOOYh773sQgHvve5Beh34JgDfefIuzzv8hl1/6I7ru8p/Nc0BWp4jgh5f8kl277sKwE7+xwTqvvraAiEJf5dn5L/Dhhytp364tB+3fk7snP9Cw8+LgA7n3/ocAuPf+h+h1yEEAvPHm25x1/kVcfukP6brLzrVtYjOiekw1i4gLI6JzRHSh0FGdHhEnADOAganaEGBS+jw5zZOWT4/CiTAZGJRGl3QFdgMeb+jRldrTrpZ0HP9Ovg8E3k+fc9uTPnf4CB5/Yh5Lly3j4D7HcNZpp7BqVSGVdfyxA2pc76D9e/J/L7/CoCGnAdBm66254mc/Xu9i5YZUDDuR73z/x0y89w98aqeO/PbySwG4ZtQtLFtWxU9+XhjNUl5ezu9vL/0CpzWNJ+Y9zaQ/TOG/PrPrRymMc799Km+8tQiA4wf2Z8r0vzDp/im0atWKrbbckit/cTGS0nnxKoOGng5Am63bcMVPLyrxvDiB73x/RDov/oPf/vInAFxzwy0sq6riJz+/EkjnxW03NMehZ0fz38b+fWC8pJ8CTwJr/yPeBIyTVAksoRDoiYhnJd0FPAesAs6MiNUN3bnW9ghqrSTtSuHPgv0pBOmZwHcpdPF7RMQjNa6c4/SINUKsqbuObX626djo3MaaZ+4qOeaU7Xlc5m6fLKmnnS409qthcc0B28xsY8v5bewl5bQl/ZekaZKeSfN7SbqoeZtmZtYAUulTBpV6IfIG4EJgJUBEPEXK15iZbVrK6jFlT6kXIttExONa95tpVU2VzcxaTEZ70KUqNWj/U9KnSSNFJA0E3my2VpmZNVTDH+uRCaUG7TMp3Cyzu6SFwMvACc3WKjOzhnJPGygM7buZwqDy7YDlFAaRX9JM7TIza5icP0+71KA9CVhG4bb1N5qtNWZmjeaeNkDniOjbrC0xM2sKOU+PlPp3xN8kfa5ZW2Jm1hSa/iUIm5RSe9oHAUMlvQx8QOHvj4iIvZqtZWZmDdCIl8JkQqlB+4hmbYWZWVPJeXqk1GePvNrcDTEzaxIO2mZmWZLNXHWpHLTNLF/c0zYzyxBfiDQzyxD3tM3MMiSj469L5aBtZjmT7552vr+SzGzz00RvrpG0s6QZkp6T9Kykc1L5dpKmSnox/WyfyiXpKkmVkp6S1L1oW0NS/RclDalpn6Vw0DazfGm629hXAedFRDdgP+BMSd2A4cC0iNgNmJbmoXAT4m5pqgCug0KQB0YA+wI9gRFrA31DOGibWb40UdCOiDcjYm76/C4wH+gE9AfGpGpjgAHpc39gbBTMBNpJ2gnoA0yNiCURsRSYCjT4AXwO2maWL/UI2pIqJM0pmio2uEmpC/AFYBbQMSLWvrnrLaBj+twJeL1otQWprKbyBvGFSDPLmdIvREbEKApv5ap5a9IngLuB70TE8uJ35UZESIoGNrRB3NM2s3xpoguRhU2pNYWAfVtE/D4Vv53SHqSfi1L5QmDnotU7p7KayhvEQdvMckb1mGrZSqFLfRMwPyJ+U7RoMoXXLZJ+TioqH5xGkewHVKU0yhSgt6T26QJk71TWIE6PmFm+NN0dkQcCJwFPS5qXyn4A/AK4S9IpwKvAcWnZA8CRQCVQDQwDiIglki4FZqd6l0TEkoY2ShHNnI6pXrxR8z2WEbGmpVtgm6JtOjY64saiZ0qOOdpxz8zdieOetpnlTObicL04aJtZvviBUWZmWeKgbWaWHe5pm5lliB/NamaWJe5pm5llh9MjZmZZku+gne/kj5lZzrinbWa5IqdHzMwyxKNHzMyyxD1tM7PscHrEzCxLHLTNzLLDPW0zsyxx0DYzyw6PHjEzy5B8d7QdtM0sb/IdtR20zSxffCHSzCxLHLTNzLIj5xciFVHy2+atkSRVRMSolm6HbVp8Xlh95PsradNT0dINsE2SzwsrmYO2mVmGOGibmWWIg/bG5bylbYjPCyuZL0SamWWIe9pmZhnioG1mliG+uaYJSNoemJZm/wNYDSxO8z0j4sMWaZi1CEmrgaeLigZExCs11P1XRHxiozTMcsE57SYm6WLgXxHxq6KyVhGxquVaZRtTfQKxg7bVl9MjzUTSLZJ+J2kWcLmkiyWdX7T8GUld0ucTJT0uaZ6k6yWVt1S7relJ+oSkaZLmSnpaUv8N1NlJ0sPpHHhG0pdSeW9Jj6V1J0hygN/MOWg3r87AARFxbk0VJP038A3gwIjYm0Jq5YSN0zxrJlun4DtP0j3A+8AxEdEd+DLwa2m9R9F9E5iSzoHPA/MkdQAuAg5P684BajyXbPPgnHbzmhARq+uo0wvoAcxO/4+3BhY1d8OsWa1IwRcASa2ByyQdDKwBOgEdgbeK1pkNjE51742IeZIOAboBj6ZzYwvgsY1zCLapctBuXu8VfV7Fun/ZbJV+ChgTERdutFbZxnYCsAPQIyJWSnqFf//+AYiIh1NQ/ypwi6TfAEuBqRFx/MZusG26nB7ZeF4BugNI6g50TeXTgIGSdkzLtpO0S4u00JpLW2BRCthfBtb7/abf+dsRcQNwI4VzZSZwoKTPpDrbSPqvjdhu2wS5p73x3A0MlvQsMAv4B0BEPCfpIuCPksqAlcCZwKst1lJrarcB90l6mkJe+vkN1DkUuEDSSuBfwOCIWCxpKHCHpC1TvYtI545tnjzkz8wsQ5weMTPLEAdtM7MMcdA2M8sQB20zswxx0DYzyxAHbTOzDHHQNjPLkP8HJW7kRvHU7cEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax = sn.heatmap(df_cm, cmap='Oranges', annot=True)\n",
    "cm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9881875"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy = accuracy_score(y_test, y_predict)     \n",
    "accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  -------------------------------Complement NaiveBayes--------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ComplementNB, tamamlayıcı saf Bayes (CNB) algoritmasını uygular. CNB, özellikle dengesiz veri kümeleri için uygun olan standart multinomial Naive Bayes (MNB) algoritmasının bir uyarlamasıdır. CNB, modelin ağırlıklarını hesaplamak için özellikle her sınıfın tamamlayıcısından elde edilen istatistikleri kullanır. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model oluşturma ve eğitim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ComplementNB()"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf2 = ComplementNB()\n",
    "clf2.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test Verisinin Tahmini"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 0, 0, 0], dtype=int64)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y2_predict = clf2.predict(x_test)\n",
    "y2_predict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[14756,   869],\n",
       "       [  112,   263]], dtype=int64)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD4CAYAAAAn3bdmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAhCElEQVR4nO3deZxXY//H8ddnJkuhXaG675JIui2V5MeNW1RUpghZKnSbB8KNcNvrzu522yuitFgrS1EkEffSougm4i6J9mgnLTN9fn98r8m3muU735mpzun9fDzOY865znXOuY6Oz1zzOdc5x9wdERGJhoyd3QAREUmdgraISIQoaIuIRIiCtohIhChoi4hESLmyPkDvhntoeIpsp/fk2Tu7CbIrqlzXSrqL4sSc3l9vKvHxdjT1tEVEIqTMe9oiIjtS5LrOxaSgLSKxkhnzqK2gLSKxYgraIiLREfOYraAtIvGinraISITEfUhc3M9PRHYzZqlPRe/LBpnZMjObmc+6nmbmZlY9LJuZPWFmc8zsczNrklS3m5nNDlO3pPKmZvZF2OYJs6JbpaAtIrGSYalPKRgMtNm20MzqAK2AH5KKzwAahCkb6B/qVgV6AccBzYFeZlYlbNMfuDxpu+2Otd35pdRsEZGIsGJMRXH3j4EV+ax6FLgZSH76MgsY6gmTgcpmdiDQGhjv7ivcfSUwHmgT1lV098me+LDBUKBDUW1STltEYiXFHnTazCwLWOju/90mm1ELmJ+0vCCUFVa+IJ/yQiloi0isFCdmm1k2iVRGngHuPqCQ+hWA20ikRnYKBW0RiZXiDPkLAbrAIJ2P+kA9IK+XXRv41MyaAwuBOkl1a4eyhcAp25RPDOW186lfKOW0RSRWMs1TnorL3b9w9xruXtfd65JIaTRx9yXAaKBrGEXSAljt7ouBcUArM6sSbkC2AsaFdWvMrEUYNdIVGFVUGxS0RSRWSvNGpJm9DEwCDjOzBWbWvZDqY4G5wBzgWeAqAHdfAdwNfBKmPqGMUOe5sM23wDtFtUnpERGJldK8D+nuFxSxvm7SvAM9Cqg3CBiUT/k0oHFx2qSgLSKxosfYRUQiJOYxW0FbROKlrMdp72wK2iISKwraIiIREvOYraAtIvGinraISITEPGYraItIvGjIn4hIhCg9IiISIXF/N4eCtojEitIjIiIREvOYraAtIvGinLaISITEPGYraItIvJSLedRW0BaRWNGNSBGRCNGQPxGRCFFPW0QkQjLS+GBvlChoi0isKD0iIhIhmTFPj8T9l5KI7GbMUp+K3pcNMrNlZjYzqezvZva1mX1uZm+YWeWkdbea2Rwz+8bMWieVtwllc8zslqTyemY2JZS/amZ7FtUmBW0RiZWMYkwpGAy02aZsPNDY3Y8E/gfcCmBmjYDOwBFhm35mlmlmmUBf4AygEXBBqAvwIPCoux8CrAS6p3J+IiKxkWGpT0Vx94+BFduUvefuOWFxMlA7zGcBr7j7Bnf/DpgDNA/THHef6+4bgVeALDMz4FRgZNh+CNChyPMrutkiItFRnPSImWWb2bSkKbuYh7sMeCfM1wLmJ61bEMoKKq8GrEr6BZBXXijdiBSRWCnOjUh3HwAMSOc4ZnY7kAO8mM726VLQFpFY2RHpAzO7BGgHtHT3vIHhC4E6SdVqhzIKKF8OVDazcqG3nVy/QEqPiEislGZOOz9m1ga4GTjL3dclrRoNdDazvcysHtAAmAp8AjQII0X2JHGzcnQI9h8CncL23YBRRZ5fes0WEdk1WTGmIvdl9jIwCTjMzBaYWXfgKWA/YLyZzTCzpwHc/UtgOPAV8C7Qw91zQy/6amAcMAsYHuoC/BW4wczmkMhxDyyqTUqPiEislOZHENz9gnyKCwys7n4vcG8+5WOBsfmUzyUxuiRlCtoiEisxfyBSQVtE4qVchl4YJSISGeppi4hEiD7suxvLuvdZDj3lTH5Zvox+Zx2z3fq6zU+ic9/XWbVgHgCzxr/BR/22uwdRLJl77EnHB5/noCOasG7VCkbecCGrFn6/ZX2lA+vQ4+3Pmdi3D/8Z9GiJjiXpGfzy64wY9Q5mxqH163H/nT3Za6/f3vNz36NPM2X6fwFYv34Dy1euYtqE10t0zFWr13D9HfexcNFSah1Uk8fuvZ1KFfdj9Lsf8Oyw4eDOPhXK0/vma2h4aP0SHSvqYh6zNeSvMDPeGMILl7crtM4P0//F0x2b8XTHZsUK2JVr/Z5Lhr6/XXmTTpexfs0qnmh9OJOHPM5pPe/ban3rW/7O7H++m/JxpHQtXfYTQ199k9cGP8XbLw8gd3MuY8ZP3KrObddfwagX+jPqhf5cfF4Wp59yQsr7nzL9v9zS5+HtygcMHc7xzY7hvdee5/hmxzBg6KsA1D6oJi/0/ztvvfQMV152EXc+8HiJzi8Oynqc9s6moF2I76f9i19Xryi6Yj6ObH8hlw//D1e8MY12f+uHZaT2n/qwlu2Z8eYwAL4a9xoHH3/qlnUNW57FygXz+HHOV2m1SUpHbm4u6zdsICcnl/XrN1CjerUC645570PatTply/Jzw0ZwziXX0P6iK3hiwNCUjznh40l0aHsaAB3ansb7H00CoMmRR1Cp4n4AHN24IUuW/ZTGGcVLKb/lb5eTUrvN7CEzq2hme5jZBDP70cwuLuvGRUHto1twxZvTuWjAW+x/SOJti9UPbsgRZ57LwAtP4umOzfDcXI5sf2FK+6tY4yDWLE68W2Zzbi7r166mQuVq7FlhH064/CY+6nt3mZ2LFK1mjepcdlEn/pTVhRPbXsC+++7DiS2a5lt34eKlLFi0lBbNjgbgX5On8/38hYx8/glGDevHl1/P5pPPvkjpuMtXrNzyy2H/alVZvmLldnVGjn6Xk44/Nr0Ti5HMjNSnKEo1p93K3W82s47APOBs4GPghfwqhzdlZQO0q5lB08oR/a9ThMVffsZjp9Zn47pfaHBSGzo/NZIn2zTi4ONP5aAjmpA9YjIA5fbem19WLAPg/CdHUKV2PTL32INKB/6OK96YBsDkYU8y4/UhBR7rlKvvYvLgx9m47peyPzEp0Oo1a5nw8SQmvDGE/fbbl7/ceg+j3plA1hktt6s7ZvxEWp96IpmZmQD8e8p0/j31Uzp0uQqAdb/+yrz5Czn2mD9w7mXXsnHjJtb9+iur16wl6+IrAbjx6u78sUWzrfZrZtg2b/CfPG0GI98ax0sDHimL046UiGY9UpZq0M6r1xYY4e6rt71okiW/Oat3wz1iO2hywy9rt8zP/vhd2vZ6kgqVq4EZM94cxoRH7thum1evORdI5LQ73D+QwV1P22r9mmWLqHhgHdYsXUhGZiZ771eJdauWU+vI5jRqfTan33Q/e+9XGd+8mZwNG5j6Yr+yPUnZyn8++YzaBx1A1SqVAWj1pxP47Iuv8g3aY8d/xF039diy7DjZXc+n89ltt6s7YtATQCKn/caY8Txw141bra9WtQrLflpOjerVWPbT8i3HB/h69lzuuO8xnn3sHqpUqlgKZxlthcWmOEi1C/y2mX0NNAUmmNn+wPqya1Y07Fu95pb5Wn84FrMM1q1azneTPqBRq7PZp+r+AJSvVIVKB/0upX1+88HbHN2hCwCNWp/Dd5M/BOD5i//EYy0b8FjLBkwe+gT/HPCAAvZOcFDNGvx35ix+Xb8ed2fSJzOoX3f7f9tv5/3AmrU/c8wfGm0pO/G4Zrz29jh+WfcrkLipuXzFqpSOe+ofW/DmmMSN6zfHvE/Lk44HYNGSZVxzSx8e6n0T9X5Xu7Bd7DZK83Nju6KUetrufouZPQSsdvdcM/uFxFcaYu2cfwyj7rEnU6FKdW6Y+B0fPtmHzHJ7ADDt1QE0an0OzTpnszk3l5z1vzKyZyLN/+O3s/jg8V50GfgOlpFBbs4mxva5ltWLfijymJ+NHETHhwZz7bhZ/Lp6JSNvuKhMz1GK56jGDWl96h/p2LUH5TIzOfzQQzi/wxk8/swQGh9+6JZgOnb8R5x5+slb9fpObNGUb+f9QOc/XwdAhfLl+fvfbqZa1cpFHje72/lcd9u9jBz9LgcdWIPH7r0dgL4DX2TV6rX87aGnAMjMzOT1IU+V7klHTVSjcYrst1fBFlLJbA/gSuCkUPQR8LS7bypq2zinRyR9vSfP3tlNkF1R5boljrhzz0o95hw8elPkInyqOe3+wB5A3t/jXULZn8uiUSIi6Yp7TjvVoH2sux+VtPyBmf23LBokIlIi8RystkWqp5drZluejTWzg4HcsmmSiEj68oZEpjJFUao97ZuAD81sbliuC1xaJi0SESmBiMbilKXa0/438AywGVgR5ieVVaNERNKlnnbCUGANkPcM9YXAMODcsmiUiEi6LKpvgkpRqkG7sbs3Slr+0Mz01iIR2eVEtAOdslTTI5+aWYu8BTM7DphWNk0SEUlf3NMjqQbtpsB/zGyemc0jkc8+1sy+MLPPy6x1IiLFVYrPsZvZIDNbZmYzk8qqmtl4M5sdflYJ5WZmT5jZHDP73MyaJG3TLdSfbWbdksqbhjg6J2xbZKNSDdptgHrAyWGqF8raAe1T3IeISJkr5XePDCYR65LdAkxw9wbAhLAMcAbQIEzZJB5AxMyqAr2A44DmQK+8QB/qXJ603bbH2k6q7x75vuhaIiI7X2mmPdz9YzOru01xFnBKmB8CTAT+GsqHeuLdIJPNrLKZHRjqjnf3FaF944E2ZjYRqOjuk0P5UKAD8E5hbdI3IkUkVoozeiT53f/BgPBq6cLUdPfFYX4JkPe6z1rA/KR6C0JZYeUL8ikvlIK2iMRKcTraye/+T4e7u5nt0JfixfwpfRHZ3eyA0SNLQ9qD8HNZKF8I1EmqVzuUFVZeO5/yQiloi0is7ICgPRrIGwHSDRiVVN41jCJpQeL7A4uBcUArM6sSbkC2AsaFdWvMrEUYNdI1aV8FUnpERGKlNIdfm9nLJG4kVjezBSRGgTwADDez7sD3wHmh+ljgTGAOsI7wfiZ3X2FmdwOfhHp98m5KAleRGKFSnsQNyEJvQoKCtojETGk+xu7uFxSwaruPgoZRIz3yqYu7DwIG5VM+DWhcnDYpaItIrET1ScdUKWiLSKzEPGYraItIzMQ8aitoi0isKD0iIhIhMY/ZCtoiEi/6CIKISIRYRryfGVTQFpF4iXl+REFbROJFQVtEJDrMlB4REYkO5bRFRKJD47RFRKJE6RERkejQOG0RkShRekREJEKUHhERiQ49ESkiEiVKj4iIRIcerhERiRL1tEVEoiPuQ/7i/XeEiOx+LCP1qahdmV1vZl+a2Uwze9nM9jazemY2xczmmNmrZrZnqLtXWJ4T1tdN2s+tofwbM2tdktNT0BaRWLGMjJSnQvdjVgu4Fmjm7o2BTKAz8CDwqLsfAqwEuodNugMrQ/mjoR5m1ihsdwTQBuhnZpnpnp+CtojEi1nqU9HKAeXNrBxQAVgMnAqMDOuHAB3CfFZYJqxvaYkXoWQBr7j7Bnf/DpgDNE/39BS0RSReipEeMbNsM5uWNGXn7cbdFwIPAz+QCNargenAKnfPCdUWALXCfC1gftg2J9SvllyezzbFphuRIhIrxXnLn7sPAAYUsJ8qJHrJ9YBVwAgS6Y2dSj1tEYmXDEt9KtxpwHfu/qO7bwJeB04AKod0CUBtYGGYXwjUAQjrKwHLk8vz2ab4p5fuhiIiuyLLyEx5KsIPQAszqxBy0y2Br4APgU6hTjdgVJgfHZYJ6z9wdw/lncPoknpAA2Bquuen9IiIxEspPVzj7lPMbCTwKZADfEYilTIGeMXM7gllA8MmA4FhZjYHWEFixAju/qWZDScR8HOAHu6em267FLRFJFZK88s17t4L6LVN8VzyGf3h7uuBcwvYz73AvaXRJgVtEYkXvXtERCRC9O6Rkun96eKyPoREkW/e2S2QmNKHfUVEoiQz7SfEI0FBW0TiRTltEZEIUXpERCRC1NMWEYkQ9bRFRCJEPW0RkQgp+p0ikaagLSLxovSIiEiEKGiLiESIctoiIhGinraISIToRqSISIQoPSIiEiFKj4iIRIh62iIiEaKetohIhMS8px3vsxOR3U9GZupTEcysspmNNLOvzWyWmR1vZlXNbLyZzQ4/q4S6ZmZPmNkcM/vczJok7adbqD/bzLqV6PRKsrGIyC7HLPWpaI8D77p7Q+AoYBZwCzDB3RsAE8IywBlAgzBlA/0TzbGqJL7ofhyJr7j3ygv06VDQFpF4sYzUp8J2Y1YJOAkYCODuG919FZAFDAnVhgAdwnwWMNQTJgOVzexAoDUw3t1XuPtKYDzQJt3TU9AWkXgpRk/bzLLNbFrSlJ20p3rAj8DzZvaZmT1nZvsANd0974vlS4CaYb4WMD9p+wWhrKDytOhGpIjESzFuRLr7AGBAAavLAU2Aa9x9ipk9zm+pkLzt3cw83aamQz1tEYmX0stpLwAWuPuUsDySRBBfGtIehJ/LwvqFQJ2k7WuHsoLK06KgLSLxkpmZ+lQId18CzDezw0JRS+ArYDSQNwKkGzAqzI8GuoZRJC2A1SGNMg5oZWZVwg3IVqEsLUqPiEi8lO7DNdcAL5rZnsBc4FISnd3hZtYd+B44L9QdC5wJzAHWhbq4+wozuxv4JNTr4+4r0m2QuZdxOmbdTzs03yMR4Zt3dgtkV7RPjRJH3M2vX5VyzMk4u1/kHp9UT1tE4kWPsYuIREjMH2NX0BaReMmId1iL99mJyO5H6RERkQhRekREJEIUtEVEIkTpERGRCFFPW0QkQjR6REQkQpQeERGJEKVHREQiREFbRCRCMhS0RUSiI4WvrEeZgraIxIvSIyIiEaLRIyIiEaKetohIhChoi4hEiIK2iEiEaPSIiEiExLynHe+zE5Hdj1nqU0q7s0wz+8zM3g7L9cxsipnNMbNXzWzPUL5XWJ4T1tdN2setofwbM2tdktNT0BaReLGM1KfU/AWYlbT8IPCoux8CrAS6h/LuwMpQ/mioh5k1AjoDRwBtgH5mlnYOR0G7ELf2vo/jT21Lu04X57v+2+++5/yu2TRufgoDh75UKsfcuHEj1/31Tk4/6zzO7XI5CxYtBuDfk6dy9oWX0f7cLpx94WVMmjq9VI4nxbN4yVK6ZF/LmedcTNtOXRjy0oh8602Z9hlZnS+lbacuXPznq0t83MR10YvTz+rMuV2zk66LTzj7wu60P68bZ1/YXdcFlGrQNrPaQFvgubBswKnAyFBlCNAhzGeFZcL6lqF+FvCKu29w9++AOUDzdE9PQbsQZ7c/k+f6PlLg+sqVKnL7X6+ne9cLir3vBYsW0yWf/5lHvPk2Fffbj/Gjh3PJRefz8OP9AKhSuTL9H3uQt0YM44E+d3DzHX2KfUwpuczMTG65vgdjX3uBV4c8w0vDX2fO3O+2qrNm7Vr+dv8/6P/oA4wZOYzHH7o75f0vWLSYLpdfs135iDfHULHifowf/QqXXHQeDz/+NABVKlei/+MP8tbwITzQ53ZuvvOekp1gHBQjaJtZtplNS5qyt9nbY8DNwOawXA1Y5e45YXkBUCvM1wLmA4T1q0P9LeX5bFNsCtqFOLbp0VSqVLHA9dWqVuHIIw6nXLnt7+eOGjOOThf/mazzu3HXPQ+Rm5ub0jE/mPhPOrY/E4DWp53CpKnTcXcaNTyUmjX2B6BB/Xps2LCBjRs3pnFWUhI19q/OEYcfBsC++1Tg4Hp1Wbrsp63qvPXO+5x+6skcdGBNIHGd5Bk1ZhydumST1flS7rrn78W7Ltq1AaB1y1OY9EnSdbF/dUDXxRaZmSlP7j7A3ZslTQPydmNm7YBl7r5L/fmioF0Gvp07j3fem8DLzz/NqFeHkJGRwVtj30tp26XLfuTAA2oAUK5cOfbbdx9Wrlq9VZ1x70+kUcPD2HPPPUu97ZK6BYsWM+ub/3FU40Zblc/7fj5r1qyly+XXcPaF3Xnz7XeBvOviA14e1I9RrzxPRmYGb70zPqVjLf3xp6KviwkTadTwUF0XpZceOQE4y8zmAa+QSIs8DlQ2s7yeWm1gYZhfCNQBCOsrAcuTy/PZpthSGvJnZmsB36Z4NTAN6Onuc7epnw1kAzzz5D/Ivqxruu2LpElTpzHzq6/pdHHi/sT6DRu29LZ63HArCxYuYtOmHBYvWUrW+d0A6HrheZyT1bbIfc/+di4PP9GPQf0eLbsTkCL9sm4d1954B7f1vJZ9991nq3W5ubl8OesbBj/zGOvXb6DzJVdy1B8aMWnqdGbO+oZOXS4HwnVRJVwXPW9jwcLFbNq0icVLlpHV+VIAul7QKcXr4jsefuJpBhWSztttlNKQP3e/FbgVwMxOAW5094vMbATQiUQg7waMCpuMDsuTwvoP3N3NbDTwkpk9AhwENACmptuuVMdpP0YiD/MSYCTuhNYHPgUGAackVw5/YiT+zFj307bBPvbcnY7tz6DntVdut67vI/cDiV7arXfdy7Dnntpqfc0a+7N4yTIOqFmDnJwc1v78C1UqVwJgydJlXH3DbTx49538rk7tsj8RydemTTlce+MdtD/zdFq1PHm79QfU3J/KlSpSoXx5KpQvT7MmR/H1/77FcTq2b0PPa67Ybpu+/7gPCNdFr/sY9uyTW62vuX/1wq+LnrfxYJ/b+V2dtFOl8VH2L4z6K/CKmd0DfAYMDOUDgWFmNgdYQSJO4u5fmtlw4CsgB+jh7qnlxfKR6q+ks9z9GXdf6+5rQlBu7e6vAlWK2nh3c3zzZox7fyLLV6wEYNXqNSxctCSlbU89+UTeeGsskEiDtDi2KWbGmrVryb7mJnpeewVNjz6yzNouhXN3bu/zAAfXq8ulF3fOt07Lk09k+ozPycnJ4ddf1/P5zK+oX+/3HN+8KePe/yj96yKkWcZNmEiLY5v8dl1cezM9r9F18RsrxpQad5/o7u3C/Fx3b+7uh7j7ue6+IZSvD8uHhPVzk7a/193ru/th7v5OSc4u1Z72OjM7j9+GuXQC1ue1pyQN2JXdcEsvpk7/jJWrVnFS6w5cc0V3cnISN40vOLcjP/60nHMu6s7Pv/xChmUw5MXhjH3tRQ6pX4/relzOZVdex2Z39ihXjrtuuYFaBx1Q5DE7dWjHTXfczelnnUelihV59IG/AfDCK6/xw/wF9B3wPH0HPA/AoP6PbXWTS8re9BlfMGrMOA495OAtKYwbrs5m0ZKlAFzQqQP1D67LH//vOM46/xIyMjLo1KEdhx5yMADXXfVnLrvqBjZv3lzM66ItN915D6ef1ZlKlSry6P29AXjh1df5Yf5C+j47mL7PDgZgUL9Hdu/rIuaPsZt70THXzA4mkYA/nkSQngxcTyKZ3tTd/1XgxrthekRS4JuLriO7n31qlDi3sXnm8JRjTkbj8yL38u2Uetqhm9++gNUFB2wRkR0t5h9BSCmnbWaHmtkEM5sZlo80szvKtmkiImko5XeP7GpSvRH5LImhL5sA3P1zwp1REZFdS0YxpuhJ9UZkBXefalv/ZsopqLKIyE4T0R50qlIN2j+ZWX3CSBEz6wQsLrNWiYikK/0X6EVCqkG7B4mHZRqa2ULgO+CiMmuViEi61NMGEkP7ngc+BKoCa0g8rqlXzYnIriXmX65JNWiPAlaReGx9UZm1RkSkxNTTBqjt7m3KtCUiIqUh5umRVP+O+I+Z/aFMWyIiUhpK/3Nju5RUe9onApeY2XfABhJ/f7i76w01IrJLKcHnFyMh1aB9Rpm2QkSktMQ8PZLqu0e+L+uGiIiUCgVtEZEoiWauOlUK2iISL+ppi4hEiG5EiohEiHraIiIREtHx16lS0BaRmIl3Tzvev5JEZPdTSl+uMbM6ZvahmX1lZl+a2V9CeVUzG29ms8PPKqHczOwJM5tjZp+bWZOkfXUL9WebWbeSnJ6CtojES+k9xp4D9HT3RkALoIeZNQJuASa4ewNgQliGxEOIDcKUDfSHRJAHegHHAc2BXnmBPh0K2iISL6UUtN19sbt/GubXArOAWkAWMCRUGwJ0CPNZwFBPmAxUNrMDgdbAeHdf4e4rgfFA2i/gU9AWkXgpRtA2s2wzm5Y0Zee7S7O6wDHAFKCmu+d9uWsJUDPM1wLmJ222IJQVVJ4W3YgUkZhJ/Uakuw8g8VWugvdmti/wGnCdu69J/lauu7uZeZoNTYt62iISL6V0IzKxK9uDRMB+0d1fD8VLQ9qD8HNZKF8I1EnavHYoK6g8LQraIhIzVoypkL0kutQDgVnu/kjSqtEkPrdI+DkqqbxrGEXSAlgd0ijjgFZmViXcgGwVytKi9IiIxEvpPRF5AtAF+MLMZoSy24AHgOFm1h34HjgvrBsLnAnMAdYBlwK4+wozuxv4JNTr4+4r0m2UuZdxOmbdTzs03yMR4Zt3dgtkV7RPjRJHXF82M+WYYzUaR+5JHPW0RSRmIheHi0VBW0TiRS+MEhGJEgVtEZHoUE9bRCRC9GpWEZEoUU9bRCQ6lB4REYmSeAfteCd/RERiRj1tEYkVU3pERCRCNHpERCRK1NMWEYkOpUdERKJEQVtEJDrU0xYRiRIFbRGR6NDoERGRCIl3R1tBW0TiJt5RW0FbROJFNyJFRKJEQVtEJDpifiPS3FP+2ryUkJllu/uAnd0O2bXoupDiiPevpF1P9s5ugOySdF1IyhS0RUQiREFbRCRCFLR3LOUtJT+6LiRluhEpIhIh6mmLiESIgraISITo4ZpSYGbVgAlh8QAgF/gxLDd39407pWGyU5hZLvBFUlEHd59XQN2f3X3fHdIwiQXltEuZmfUGfnb3h5PKyrl7zs5rlexIxQnECtpSXEqPlBEzG2xmT5vZFOAhM+ttZjcmrZ9pZnXD/MVmNtXMZpjZM2aWubPaLaXPzPY1swlm9qmZfWFmWfnUOdDMPg7XwEwz+2Mob2Vmk8K2I8xMAX43p6BdtmoD/+fuNxRUwcwOB84HTnD3o0mkVi7aMc2TMlI+BN8ZZvYGsB7o6O5NgD8B/zDb7lV0FwLjwjVwFDDDzKoDdwCnhW2nAQVeS7J7UE67bI1w99wi6rQEmgKfhP+PywPLyrphUqZ+DcEXADPbA7jPzE4CNgO1gJrAkqRtPgEGhbpvuvsMMzsZaAT8O1wbewKTdswpyK5KQbts/ZI0n8PWf9nsHX4aMMTdb91hrZId7SJgf6Cpu28ys3n89u8PgLt/HIJ6W2CwmT0CrATGu/sFO7rBsutSemTHmQc0ATCzJkC9UD4B6GRmNcK6qmb2+53SQikrlYBlIWD/Cdju3zf8my9192eB50hcK5OBE8zskFBnHzM7dAe2W3ZB6mnvOK8BXc3sS2AK8D8Ad//KzO4A3jOzDGAT0AP4fqe1VErbi8BbZvYFibz01/nUOQW4ycw2AT8DXd39RzO7BHjZzPYK9e4gXDuye9KQPxGRCFF6REQkQhS0RUQiREFbRCRCFLRFRCJEQVtEJEIUtEVEIkRBW0QkQv4fj3fMjdexQi8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm2 = confusion_matrix(y_test, y2_predict)\n",
    "df_cm2 = pd.DataFrame(cm2, index=['pos', 'neg'], columns=['True', 'False'])\n",
    "ax = sn.heatmap(df_cm2, cmap='Oranges', annot=True)\n",
    "cm2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9386875"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy2 = accuracy_score(y_test, y2_predict)\n",
    "accuracy2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  ---------------------------------KNeighbors Classifier----------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "K-NN ( K-Nearest Neighbor) algoritması en basit ve en çok kullanılan sınıflandırma algoritmasından biridir. K-NN non-parametric ( parametrik olmayan ), lazy ( tembel ) bir öğrenme algoritmasıdır. lazy kavramını anlamaya çalışırsak eager learning aksine lazy learning’in bir eğitim aşaması yoktur. Eğitim verilerini öğrenmez, bunun yerine eğitim veri kümesini “ezberler”. Bir tahmin yapmak istediğimizde, tüm veri setinde en yakın komşuları arar."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model oluşturma ve eğitim"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier(n_neighbors=3)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf3 = KNeighborsClassifier(n_neighbors=3)\n",
    "clf3.fit(x_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test Verisinin Tahmini"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 0, 0, 0], dtype=int64)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y3_predict = clf3.predict(x_test)\n",
    "y3_predict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[15571,    54],\n",
       "       [  178,   197]], dtype=int64)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD4CAYAAAAn3bdmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAey0lEQVR4nO3dd5gV1f3H8fdnFyygdEFgjWBEjRpNMKL+NMZIAogF7F0scR+VqLGXaDCWGDXR2CMKCHbEAvYQNLHSVGI3bkRkEVikKouR8v39cQdyFZa9u+yyzPh5Pc88e+ecM3PPyPV7z/3OmRlFBGZmlg5FDd0BMzMrnIO2mVmKOGibmaWIg7aZWYo4aJuZpUij+n6Dy7Zp7OkptpLL3pje0F2wdVGTNlrTXdQk5lz2weI1fr+1zSNtM7MUqfeRtpnZ2pS6oXMNOWibWaYUZzxqO2ibWabIQdvMLD0yHrMdtM0sWzzSNjNLkaxPiXPQNrNM8UjbzCxFihy0zczSI+Mx20HbzLLFI20zsxTJeMx20DazbPGJSDOzFClWtm8s6qBtZpmS8YG2g7aZZYuDtplZijinbWaWIhmP2Zm/TN/MvmOKVPhSHUmDJVVIemcVdedICkltknVJuklSmaS3JHXNa9tP0kfJ0i+vfCdJbyfb3CRV/zvBQdvMMqUugzZwN9Dr24WSNgN6AJ/mFe8DdEmWUuD2pG0rYACwC9ANGCCpZbLN7cDJedut9F4rHV9B3TYzSwnVYKlORLwIzFlF1Q3A+UD+/MI+wLDIGQu0kNQe6AmMjog5ETEXGA30SuqaRcTYiAhgGNC3uj45aJtZptRkpC2pVNLEvKW0uv1L6gNMi4h/fauqIzA1b708KVtdefkqylfLJyLNLFNqciIyIgYCAwvet9QEuJhcaqRBeKRtZpkiFb7UwveBzsC/JH0ClABvSNoUmAZslte2JClbXXnJKspXy0HbzDKljk9EfkNEvB0RbSOiU0R0IpfS6BoRM4BRwHHJLJJdgfkRMR14DughqWVyArIH8FxSt0DSrsmskeOAkdX1wekRM8uUuhyJSnoA2AtoI6kcGBARg6po/jTQGygDKoETACJijqQrgAlJu8sjYvnJzdPIzVDZEHgmWVbLQdvMMqUur4iMiCOrqe+U9zqA/lW0GwwMXkX5RGD7mvTJQdvMMiXrV0Q6aJtZpvjJNWZmKZLxmO2gbWbZ0ijjUdtB28wyxbdmNTNLkaxffOKgbWaZ4pG2mVmKFPnBvmZm6eH0iJlZihQ7PWJmlh7OaZuZpYjTI2ZmKeLL2M3MUsTpETOzFPGJSDOzFHFO28wsRZzTNjNLkYzHbAdtM8uWrI+0s57+MbPvGNVgqXZf0mBJFZLeySu7TtIHkt6S9JikFnl1F0kqk/ShpJ555b2SsjJJF+aVd5Y0Lil/SNJ61fXJQdvMMqVRURS8FOBuoNe3ykYD20fEDsC/gYsAJG0LHAFsl2xzm6RiScXArcA+wLbAkUlbgGuAGyJiS2AucFJ1HXLQNrNMqcuRdkS8CMz5VtnfImJJsjoWKEle9wEejIj/RsRkoAzolixlEfFxRHwNPAj0kSRgb2BEsv1QoG91fXLQNrNMKVLhi6RSSRPzltIavt2JwDPJ647A1Ly68qSsqvLWwLy8L4Dl5avlE5Gr0eeqO9lqr94snF3BbQf8eJVtOnXbk14XXU9Ro0ZUzpvN3cd2X6P3LG68HgdeM4QO23Wlct4cRpx9FPOmTVlR37z9ZvR/8i3+cevlvDr4hjV6L2sYe/c+mKZNm1BUVERxcTGP3j94Rd3gYQ9wzQ238NrzT9GqZYuG62SK1eQ8ZEQMBAbW6n2k3wJLgPtqs31tOWivxqTHhjL+vts48I+DV1m/wcbN2fd3N3Pvyfsxf/pUmrbapOB9t+i4OX2vHsTdx/3iG+VdDzmRrxbM46aeP2D73ofxi3P+wIizj15R3/PC6/jopWdrd0C2zhg68OaVgvL0GTN5Zex4OmzarmE6lRFrY/aIpOOB/YDuEbE8OT4N2CyvWUlSRhXls4EWkholo+389lVyemQ1pkx8mUXz51RZ/8P9juT90Y8zf3rul8/CObNW1O2w/1GcPPxVTnlsIvv9/jZUVNh/6q2778+kx+8B4L3nHmGL3fZeUbdN9wOYW/4Js8req83h2Dru6j/dxHlnnoayfvOMelZUg6U2JPUCzgcOiIjKvKpRwBGS1pfUGegCjAcmAF2SmSLrkTtZOSoJ9i8AhyTb9wNGFnJ8hXTyWknNJDWWNEbSLEnHFHqQWdW6Uxc2aNaS44f9ndJHxrFjn9x/kjZbbMN2vQ9l0FF78tcDf0IsXcoO+x9V0D6bte3AguRLYNnSpXz1xXyatGjNek2asvvJ5/HPW6+ot+OxtUTipNPO4qCjTuShR3L/j/79hZdo23YTttm6SwN3Lv2KiwpfqiPpAeA1YGtJ5ZJOAm4BNgZGS5ok6a8AEfEuMBx4D3gW6B8RS5NR9K+B54D3geFJW4ALgLMllZHLcQ+qrk+Fpkd6RMT5kg4EPgEOAl4E7q3iQEuBUoD92hWxU4tsDuiLGjWiw3ZdGXpCDxqvvyEnPfgS5f8axxa77U2H7bpS+vBYABptsAEL51QAcPjND9OypDPFjRvTvP33OOWxiQCMvedmJj06tMr32uvXv2Ps3TfydeXC+j8wq1cPDLmddm03YfacuZxwym/YotPm3DF4GINv8zmKulCXv1Mi4shVFFcZWCPiKuCqVZQ/DTy9ivKPyc0uKVihQXt5u32BhyNi/up+wuUn9y/bpnFmn7K5YEY5lfNms3hRJYsXVTJl4su023oHkJj0+D2Muf6SlbZ56PRDgapz2gsqPqNZ+81YMHMaRcXFbLBxcyrnzabjDt3YtudB/PK8q9lg4xbEsmUs+e9/GX/fbWvlWK3utGubO/fRulVLfrn3nox//U3Kp31Gn8P7ATCjYhYHHXUiD99zJ5u0ad2QXU2lrKeXCg3aT0r6AFgEnCppE+Cr+utWOnww5gl6X3ojRcXFFDdej5Iddmbs0BuZVfYeR9z6CGPvvpGFc2axYfOWrNd0Y+Z/9mm1+/zw+Sf5Ud9jKZ80lm17HszksS8AMOSYn69os9evL+Xryi8dsFOoctEili1bxkZNm1K5aBGvvDae00pP4LXnn1rRZu/eBzPivkGePVJLGY/ZhQXtiLhQ0rXA/IhYKmkhuYnkmXbwn++h084/o0nLNpz9j8m8cPPlFDdqDMDEhwby+ccfUPbSc5w68g1i2TLeGDGEio9yqarnbxzAsYOeQUVFLF2ymKcvP6OgoP3miMEceO3dnPHc+yyaP/cbM0cs/WbPnkP/sy8GYOnSJey3Tw/23H3XBu5VxmQ8aut/s1VW00hqDJwK7JkU/RP4a0Qsrm7bLKdHrPYue2N6Q3fB1kVN2qxxxP34gMJjzhajFqcuwheaHrkdaAws/z1+bFL2q/rolJlZbTmnnbNzROyYt/68pH/VR4fMzNZINierrVDo4S2V9P3lK5K2AJbWT5fMzGpPUsFLGhU60j4PeEHSx8l6J+CEeumRmdkaSGksLlihI+1XgDuAZeRuU3gHuauEzMzWKR5p5wwDFgDLr6E+CrgHOLQ+OmVmVlvK+PPGCg3a20fEtnnrL0jyXYvMbJ2T0gF0wQpNj7whacUVAJJ2ASbWT5fMzGrP6ZGcnYBXJS2/pO97wIeS3gYieVaamVnDS2kwLlShQfvbD7Y0M1snZTxmF3zvkSnVtzIza3hpTXsUyo8bM7NM8ewRM7MUyfhA20HbzLLF6REzsxTJetDO+P2wzOy7Rip8qX5fGiypQtI7eWWtJI2W9FHyt2VSLkk3SSqT9Jakrnnb9EvafySpX175TpLeTra5SQV84zhom1mmqEgFLwW4m5WnPF8IjImILsCYZB1gH6BLspSSe+YAkloBA4BdyD3Ed8DyQJ+0OTlvu2qnVztom1mm1OUVkRHxIrmb5OXrAwxNXg8F+uaVD4ucsUALSe2BnsDoiJgTEXOB0UCvpK5ZRIyN3CPEhuXtq0oO2maWKTVJj0gqlTQxbykt4C3aRcTy5+XNANolrzsCU/PalSdlqysvX0X5avlEpJllSw1OREbEQGBgbd8qIkLSWn0OrkfaZpYpa+GGUTOT1AbJ34qkfBqwWV67kqRsdeUlqyhfLQdtM8uUupw9UoVRwPIZIP2AkXnlxyWzSHYF5idplOeAHpJaJicgewDPJXULJO2azBo5Lm9fVXJ6xMwypS4vY5f0ALAX0EZSOblZIH8Ehks6CZgCHJY0fxroDZQBlSSPZIyIOZKuACYk7S6PiOUnN08jN0NlQ+CZZFktB20zyxQV1V0CISKOrKKq+yraBtC/iv0MBgavonwisH1N+uSgbWbZkvErIh20zSxbHLTNzNJDyvb8CgdtM8uWOsxpr4sctM0sU7J+lz8HbTPLFqdHzMzSw48bMzNLE6dHzMxSxOkRM7P0qMsrItdFDtpmli1Oj5iZpYcvrjEzSxOPtM3M0sNT/szM0sTpETOz9PDsETOzNHFO28wsRZweMTNLj6zf5S/bX0lm9t1TpMKXakg6S9K7kt6R9ICkDSR1ljROUpmkhyStl7RdP1kvS+o75e3noqT8Q0k91+jw1mRjM7N1jYqKC15Wux+pI3AG8JOI2B4oBo4ArgFuiIgtgbnASckmJwFzk/IbknZI2jbZbjugF3CbpNW/+Wo4aJtZtkiFL9VrBGwoqRHQBJgO7A2MSOqHAn2T132SdZL67srlavoAD0bEfyNiMlAGdKvt4Tlom1mmSKrJUippYt5Sunw/ETEN+BPwKblgPR94HZgXEUuSZuVAx+R1R2Bqsu2SpH3r/PJVbFNjPhFpZtlSg9kjETEQGLjK3UgtyY2SOwPzgIfJpTcalEfaZpYtdZce+QUwOSJmRcRi4FFgd6BFki4BKAGmJa+nAZvluqBGQHNgdn75KrapsXofaV/2eq37ZlkWyxq6B5ZRdTjl71NgV0lNgEVAd2Ai8AJwCPAg0A8YmbQflay/ltQ/HxEhaRRwv6TrgQ5AF2B8bTvl9IiZZUtxrSdmfENEjJM0AngDWAK8SS6V8hTwoKQrk7JBySaDgHsklQFzyM0YISLelTQceC/ZT/+IWFrbfikiarttYRZW1PMbmFlmNG27xsPkZTf+vOCYU3TmC6m7EscjbTPLloxfEemgbWbZ4nuPmJmliEfaZmYp4pG2mVmKVHNPkbRz0DazbHF6xMwsRRy0zcxSxDltM7MU8UjbzCxFfCLSzCxFnB4xM0sRp0fMzFLEI20zsxTxSNvMLEU80jYzSxHPHjEzSxGnR8zMUsTpETOzFMn4SDvbX0lm9t2josKX6nYltZA0QtIHkt6XtJukVpJGS/oo+dsyaStJN0kqk/SWpK55++mXtP9IUr81OTwHbTPLFqnwpXo3As9GxDbAjsD7wIXAmIjoAoxJ1gH2AbokSylwe647agUMAHYBugEDlgf62nDQNrNsKS4ufFkNSc2BPYFBABHxdUTMA/oAQ5NmQ4G+yes+wLDIGQu0kNQe6AmMjog5ETEXGA30qu3hOWibWbbUYKQtqVTSxLylNG9PnYFZwBBJb0q6S1JToF1ETE/azADaJa87AlPzti9PyqoqrxWfiDSzbKnB7JGIGAgMrKK6EdAVOD0ixkm6kf+lQpZvH5Kitl2tDY+0zSxb6i6nXQ6UR8S4ZH0EuSA+M0l7kPytSOqnAZvlbV+SlFVVXisO2maWLXU0eyQiZgBTJW2dFHUH3gNGActngPQDRiavRwHHJbNIdgXmJ2mU54AeklomJyB7JGW14vSImWVLUZ2GtdOB+yStB3wMnEBusDtc0knAFOCwpO3TQG+gDKhM2hIRcyRdAUxI2l0eEXNq2yFF1HM6ZmHFWs33mFmKNW27xlfGLHvi7IJjTtH+16fuShyPtM0sW3wZu5lZijhom5mlSMbvPeKgbWbZ4pG2mVmK1O3skXVOto/OzL57nB4xM0sRp0fMzFLEQdvMLEWKHLTNzNLDT2M3M0sRp0fMzFLEs0fMzFLEI20zsxRx0DYzSxEHbTOzFPHsETOzFPFI28wsRTx7xMwsRTzS/u666LKr+cdLr9K6VUuefHjYSvVffPEl511yBZ/NmMnSpUs58dgjOLjPvmv0nvPmL+CsCwcw7bMZdOywKX+55nKaN9uYUU//jTvvvg+Apk2acNnF57DNVluu0XtZ3Zg+Yybn/+4qZs+egyQOO+gA+h116Brt87EnnuH2u3KfuVN/dRwH7r8PixZ9xZkXXMqn5Z9RXFTEz/fcnXPPOKUuDiFb6jhoSyoGJgLTImI/SZ2BB4HWwOvAsRHxtaT1gWHATsBs4PCI+CTZx0XAScBS4IyIqPXT2LP9lbSGDtp/H+665U9V1t83/FG+v0UnRj10N/fceRPX3HArXy9eXNC+x018kwsHXLVS+cAh97Jbt53428gH2K3bTgwcci8AJR3bc+9dt/DE8KGcenI/Lr3y2todlNW54uJiLjyrP08/ci8PDb2D+4c/StnHkwva9tiTT6f8s+nfKJs3fwG3DBzC8GF38PA9A7ll4BDmL/gCgBOPPZJnH72Pxx4YzBuT3uafr4yt8+NJPRUVvhTmTOD9vPVrgBsiYktgLrlgTPJ3blJ+Q9IOSdsCRwDbAb2A25Ivglpx0F6NnXf6Ec2bN6uyXhILKyuJCBZWLqJ5s2Y0Ks79W9w19H4OPuZk9j+sHzfdPqjg9xzzz5fpu18vAPru14u//+MlALru+EOaN9sYgB/9cDtmzJxV28OyOtZ2kzZs94OtAdioaRO26NyJmRWf8+nUaZzU/xwOOuokjjqxP/+ZPKWg/b382nh232VnWjRvRvNmG7P7Ljvz0qvj2HDDDdh1564ArNe4Mdv+YCtmzqyot+NKreLiwpdqSCoB9gXuStYF7A2MSJoMBfomr/sk6yT13ZP2fYAHI+K/ETEZKAO61fbwHLTXwNGHH8x/Jk/hpz37csBhx/Pb886gqKiIl18bz5RPyxlxz0BGPjiEd9//kAmvTypon7Nnz6XtJm0A2KRNa2bPnrtSmxGPP8meu+9Sl4didaT8s+m8/+G/2XH7bbn0ymu59ILf8Oj9g7jgrNP4/dXXF7SPmRWz2HTTtivW27XbhJkV3/ySXvDFF7zw4ivs1u0nddr/TKjbkfZfgPOBZcl6a2BeRCxJ1suBjsnrjsBUgKR+ftJ+RfkqtqmxgnLakr4A4lvF88nlec6JiI+/1b4UKAW446brKD3xuNr2b5328mvj+MFWWzLsjhv5dOo0TjjtbH7y4x15ZewEXhk7gb5HnghAZeUiPplazs47/YhDjyvl668XU1m5iPkLFtDniBMAOPeMU/jp/30zEEta6UT42AlvMOLxp7h/8K1r5RitcAsrKznj3Eu4+JwzUJF48613OPP8362o//rrXOrskZFPMeyB3EDt06nTKD39PBo3bkxJx/bc+uc/VPs+S5Ys4eyLfs+xRxzCZiUd6udg0qwGOe38WJUYGBEDk7r9gIqIeF3SXnXZxTVR6InIv5D7drgfELn8zPeBN4DBwF75jZODHgjAwopvB/vMeHTU05QefwyS2Px7JZR0aM/Hn0whIig94RiOOKTPSts8PCz3n2XcxDd57Imn+ePvf/uN+tatW1Ix63PabtKGilmf06pVyxV1H/y7jEuuuIY7b76Oli2a1+/BWY0sXryEM869hP17/5Ie3X/Gl18upNnGGzHywSErtT24z74rTlgfe/LpXP37iynp0H5Ffbu2mzB+4psr1mfOnEW3n/x4xfqlV15Hp++VcPzRh9XjEaVYDab8fSNWrWx34ABJvYENgGbAjUALSY2S0XQJMC1pPw3YDCiX1AhoTu6E5PLy5fK3qbFCv5IOiIg7IuKLiFiQHGjPiHgIaFndxlnVftN2vDb+dQA+nz2HyVM+paRjB/bYrRuPjHqKhZWVQO7n7uw5K6c5VmXvPXfn8SefBeDxJ5+l+8/2AOCz6TM5/dxLuPaKS+i8+ffq4WistiKC317+R7bo3IkTjjkCgI02akpJhw48M/qFFW0++HdZQfvbY7duvDx2AvMXfMH8BV/w8tgJ7LFbLgV6w6138uWXC7n43DPq52AyQTVYqhYRF0VESUR0IjdQfT4ijgZeAA5JmvUDRiavRyXrJPXPR0Qk5UdIWj+ZedIFGF/boyt0pF0p6TD+l3w/BPgqeZ3ZkfTZF13G+NffZO68+ezZ6yBOP+VElizJpbKOPKQvp518PBcN+AP7H9aPiODcM06hVcsW7LFbN/4zeQpHHH8qAE023JDrrryU1q2q/34rPeEYfnPB7xjx+FN0aN+Ov1xzOQC33jmEefPnr8iLFhcX8+h9d9XTkVtNvD7pbUY+9RxbbbnFinTX2b8u5bqrLuWyP/yZ2+8aypIlS+jds3tB0zRbNG/Gab/qxyHHnAxA/5P70aJ5M2bMrOCvg4axRafNOfCo3ISFYw4/iEMP3L/+Di6N6v8y9guAByVdCbwJLJ9pMAi4R1IZMIdcoCci3pU0HHgPWAL0j4iltX1z5b4IqmkkbUHuZ8Fu5IL0WOAsckP8nSLi5So3znB6xMzqWNO2a3w547J3hhccc4q2Pyx1l08WNNJOTjRW9XVedcA2M1vbMn4Ze0E5bUlbSRoj6Z1kfQdJl9Rv18zMakEqfEmhQk9E3glcBCwGiIi3SPI1ZmbrlqIaLOlT6InIJhExXt/8ZlpSVWMzswaT0hF0oQoN2p9L+j7JTBFJhwDTV7+JmVkDqP1tPVKh0KDdn9wE9G0kTQMmA0fXW6/MzGrLI20gN7VvCLlJ5a2ABeQmkV9eT/0yM6sd308byF3xM4/cZeuf1VtvzMzWmEfaACUR0atee2JmVhcynh4p9HfEq5J+WK89MTOrC3X/EIR1SqEj7T2A4yVNBv5L7vdHRMQO9dYzM7NaWIOHwqRCoUF7n3rthZlZXcl4eqTQe48U9pwkM7OG5qBtZpYm6cxVF8pB28yyxSNtM7MU8YlIM7MU8UjbzCxFUjr/ulAO2maWMdkeaWf7K8nMvnvq6Mk1kjaT9IKk9yS9K+nMpLyVpNGSPkr+tkzKJekmSWWS3pLUNW9f/ZL2H0nqV9V7FsJB28yype4uY18CnBMR2wK7Av0lbQtcCIyJiC7AmGQdchchdkmWUuB2yAV5YACwC9ANGLA80NeGg7aZZUsdBe2ImB4RbySvvwDeBzoCfYChSbOhQN/kdR9gWOSMBVpIag/0BEZHxJyImAuMBmp9Az7ntM0sW+rhRKSkTsCPgXFAu4hY/uSuGUC75HVHYGreZuVJWVXlteKRtplljApeJJVKmpi3lK60N2kj4BHgNxGxIL8uIoLkMYxri0faZpYtNZinHREDyT1KsYpdqTG5gH1fRDyaFM+U1D4ipifpj4qkfBqwWd7mJUnZNGCvb5X/o+BOfotH2maWMYWPtFe7F0nAIOD9iLg+r2oUucctkvwdmVd+XDKLZFdgfpJGeQ7oIallcgKyR1JWKx5pm1m21N0VkbsDxwJvS5qUlF0M/BEYLukkYApwWFL3NNAbKAMqgRMAImKOpCuACUm7yyNiTm07pVxKph4trFir+R4zS7Gmbdc44kbFOwXHHLXdPnVX4nikbWYZk7o4XCMO2maWLb5hlJlZmjhom5mlh0faZmYp4luzmpmliUfaZmbp4fSImVmaZDtoZzv5Y2aWMR5pm1mmyOkRM7MU8ewRM7M08UjbzCw9nB4xM0sTB20zs/TwSNvMLE0ctM3M0sOzR8zMUiTbA20HbTPLmmxHbQdtM8sWn4g0M0sTB20zs/TI+IlIRRT8tHlbQ5JKI2JgQ/fD1i3+XFhNZPsrad1T2tAdsHWSPxdWMAdtM7MUcdA2M0sRB+21y3lLWxV/LqxgPhFpZpYiHmmbmaWIg7aZWYr44po6IKk1MCZZ3RRYCsxK1rtFxNcN0jFrEJKWAm/nFfWNiE+qaPtlRGy0VjpmmeCcdh2TdBnwZUT8Ka+sUUQsabhe2dpUk0DsoG015fRIPZF0t6S/ShoHXCvpMknn5tW/I6lT8voYSeMlTZJ0h6Tihuq31T1JG0kaI+kNSW9L6rOKNu0lvZh8Bt6R9NOkvIek15JtH5bkAP8d56Bdv0qA/4uIs6tqIOkHwOHA7hHxI3KplaPXTvesnmyYBN9Jkh4DvgIOjIiuwM+BP0sr3YruKOC55DOwIzBJUhvgEuAXybYTgSo/S/bd4Jx2/Xo4IpZW06Y7sBMwIfn/eEOgor47ZvVqURJ8AZDUGPiDpD2BZUBHoB0wI2+bCcDgpO3jETFJ0s+AbYFXks/GesBra+cQbF3loF2/Fua9XsI3f9lskPwVMDQiLlprvbK17WhgE2CniFgs6RP+9+8PQES8mAT1fYG7JV0PzAVGR8SRa7vDtu5yemTt+QToCiCpK9A5KR8DHCKpbVLXStLmDdJDqy/NgYokYP8cWOnfN/k3nxkRdwJ3kfusjAV2l7Rl0qappK3WYr9tHeSR9trzCHCcpHeBccC/ASLiPUmXAH+TVAQsBvoDUxqsp1bX7gOekPQ2ubz0B6tosxdwnqTFwJfAcRExS9LxwAOS1k/aXULy2bHvJk/5MzNLEadHzMxSxEHbzCxFHLTNzFLEQdvMLEUctM3MUsRB28wsRRy0zcxS5P8BeBXPidWlxawAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm3 = confusion_matrix(y_test, y3_predict)\n",
    "df_cm3 = pd.DataFrame(cm3, index=['pos', 'neg'], columns=['True', 'False'])\n",
    "ax = sn.heatmap(df_cm3, cmap='Oranges', annot=True)\n",
    "cm3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9855"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy3 = accuracy_score(y_test, y3_predict)\n",
    "accuracy3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# -----------------------Sınıflandırma Algoritmalarını karşılaştırma-----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SGDClassifier, Complement NaiveBayes ve KNeighbors olmak üzere 3 tane sınıflandırma algoritması çalıştırdık. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy olarak bakarsak."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.9881875, 0.9386875, 0.9855)"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy, accuracy2, accuracy3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdUAAAE/CAYAAAAQZlkTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXGklEQVR4nO3de5RlZX3m8e8DDYKCONqtYQHayLRijxLEHjTidTQZIArJeIPoUlyMPa4R7xrJGJGgceGYUReCIhpFiYp4I61igBAIBgW7CdeGgE2D0miw8UJERW6/+WO/RR+Lqq7T8DZd1Xw/a9WqfXn3ft86Z9d+9rv3PvukqpAkSffdFpu6AZIkbS4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOpm3qSqeP39+LVy4cFNVL0nSvXLhhRfeVFULppq3yUJ14cKFrFixYlNVL0nSvZLkB9PN8/SvJEmdGKqSJHViqEqS1ImhKklSJ4aqJEmdzBiqST6V5CdJLp9mfpIck2RVkkuT7NW/mZIkzX7j9FRPBPZdz/z9gEXtZynwsfveLEmS5p4ZQ7WqzgV+tp4iBwKfrcH5wMOS7NirgZIkzRU9rqnuBFw/Mr6mTbuHJEuTrEiyYu3atR2qliRp9rhfb1SqqhOqaklVLVmwYMonPEmSNGf1CNUbgF1Gxndu0yRJekDpEarLgFe2u4CfBtxcVT/usF5JkuaUGR+on+QLwHOA+UnWAO8GtgKoquOB04D9gVXAr4FXb6zGrs/Cw7+5KapVJ9cd/cebugnStNy/zG335/5lxlCtqoNnmF/A67q1SJKkOconKkmS1ImhKklSJ4aqJEmdGKqSJHViqEqS1ImhKklSJ4aqJEmdGKqSJHViqEqS1ImhKklSJ4aqJEmdzPjsX2lz5APS5za/gEGzlT1VSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSepkrFBNsm+Sq5KsSnL4FPMfneTsJBcluTTJ/v2bKknS7DZjqCbZEjgO2A9YDBycZPGkYn8JnFJVTwYOAj7au6GSJM124/RU9wZWVdXqqroNOBk4cFKZAh7ahncAftSviZIkzQ3zxiizE3D9yPga4KmTyhwJnJHk9cBDgOd3aZ0kSXNIrxuVDgZOrKqdgf2Bk5LcY91JliZZkWTF2rVrO1UtSdLsME6o3gDsMjK+c5s26lDgFICq+i6wDTB/8oqq6oSqWlJVSxYsWHDvWixJ0iw1TqguBxYl2TXJ1gw3Ii2bVOaHwPMAkjyBIVTtikqSHlBmDNWqugM4DDgduJLhLt+VSY5KckAr9lbgNUkuAb4AHFJVtbEaLUnSbDTOjUpU1WnAaZOmHTEyfAWwT9+mSZI0t/hEJUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKmTsUI1yb5JrkqyKsnh05R5aZIrkqxM8vm+zZQkafabN1OBJFsCxwF/CKwBlidZVlVXjJRZBPwFsE9V/TzJIzdWgyVJmq3G6anuDayqqtVVdRtwMnDgpDKvAY6rqp8DVNVP+jZTkqTZb5xQ3Qm4fmR8TZs26nHA45Kcl+T8JPtOtaIkS5OsSLJi7dq1967FkiTNUr1uVJoHLAKeAxwMfCLJwyYXqqoTqmpJVS1ZsGBBp6olSZodxgnVG4BdRsZ3btNGrQGWVdXtVXUtcDVDyEqS9IAxTqguBxYl2TXJ1sBBwLJJZU5l6KWSZD7D6eDV/ZopSdLsN2OoVtUdwGHA6cCVwClVtTLJUUkOaMVOB36a5ArgbODtVfXTjdVoSZJmoxk/UgNQVacBp02adsTIcAFvaT+SJD0g+UQlSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSerEUJUkqRNDVZKkTgxVSZI6MVQlSepkrFBNsm+Sq5KsSnL4esq9KEklWdKviZIkzQ0zhmqSLYHjgP2AxcDBSRZPUW574I3ABb0bKUnSXDBOT3VvYFVVra6q24CTgQOnKPce4P3ArR3bJ0nSnDFOqO4EXD8yvqZNu1uSvYBdquqbHdsmSdKccp9vVEqyBfBB4K1jlF2aZEWSFWvXrr2vVUuSNKuME6o3ALuMjO/cpk3YHngicE6S64CnAcumulmpqk6oqiVVtWTBggX3vtWSJM1C44TqcmBRkl2TbA0cBCybmFlVN1fV/KpaWFULgfOBA6pqxUZpsSRJs9SMoVpVdwCHAacDVwKnVNXKJEclOWBjN1CSpLli3jiFquo04LRJ046Ypuxz7nuzJEmae3yikiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktSJoSpJUieGqiRJnRiqkiR1YqhKktTJWKGaZN8kVyVZleTwKea/JckVSS5NclaSx/RvqiRJs9uMoZpkS+A4YD9gMXBwksWTil0ELKmqPYAvA/+3d0MlSZrtxump7g2sqqrVVXUbcDJw4GiBqjq7qn7dRs8Hdu7bTEmSZr9xQnUn4PqR8TVt2nQOBb411YwkS5OsSLJi7dq147dSkqQ5oOuNSkleASwBPjDV/Ko6oaqWVNWSBQsW9KxakqRNbt4YZW4AdhkZ37lN+x1Jng+8E3h2Vf22T/MkSZo7xumpLgcWJdk1ydbAQcCy0QJJngx8HDigqn7Sv5mSJM1+M4ZqVd0BHAacDlwJnFJVK5McleSAVuwDwHbAl5JcnGTZNKuTJGmzNc7pX6rqNOC0SdOOGBl+fud2SZI05/hEJUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqxFCVJKkTQ1WSpE4MVUmSOjFUJUnqZKxQTbJvkquSrEpy+BTzH5Tki23+BUkWdm+pJEmz3IyhmmRL4DhgP2AxcHCSxZOKHQr8vKr+M/Ah4P29GypJ0mw3Tk91b2BVVa2uqtuAk4EDJ5U5EPhMG/4y8Lwk6ddMSZJmv3FCdSfg+pHxNW3alGWq6g7gZuARPRooSdJcMe/+rCzJUmBpG70lyVX3Z/1z3Hzgpk3diI0lXjDoze1FG8LtZcM8ZroZ44TqDcAuI+M7t2lTlVmTZB6wA/DTySuqqhOAE8aoU5MkWVFVSzZ1OzQ3uL1oQ7i99DPO6d/lwKIkuybZGjgIWDapzDLgVW34xcA/VVX1a6YkSbPfjD3VqrojyWHA6cCWwKeqamWSo4AVVbUM+FvgpCSrgJ8xBK8kSQ8osUM5NyRZ2k6fSzNye9GGcHvpx1CVJKkTH1MoSVInhuqYkrwzycoklya5OMlTk8xL8r4k32/TLk7yzpFl7mzTVia5JMlbk2wxMn/vJOe2R0BelOSTSR6c5JAkx3Zs+2lJHtaG35DkyiSfS3LAVI+d3Jwk+b0kJye5JsmF7bV43Eas7zlJvrGx1j9D3QuT/Nl65lWS149MOzbJITOs87VJXnkv23Nikmvb/8C/JXn3vVnPA1WSW0aG909ydZLHJDkyya+TPHKqsutZ3937gfWUOSfJPe4C7r1P2pzdr59TnauS/AHwAmCvqvptkvnA1sB7gd8DnlRVtybZHnjryKK/qao92zoeCXweeCjw7iSPAr4EHFRV321lXgxs37v9VbX/yOj/Bp5fVWva+OQ7uaeVZF57uMec0J7q9TXgM1V1UJv2+8CjgKs3Zds2koXAnzFsZ1P5CfDGJB9vT0ebUVUdfx/b9Paq+nKSbYArkny2qq69j+t8QEnyPOAY4L9X1Q/aw+puYtjXvGPc9UzaD9xv2v9hququTVH//c2e6nh2BG6qqt8CVNVNwC+A1wCvr6pb2/RfVtWRU62gqn7C8OCLw9pG9jqGnf13R8p8uapuHF0uyQvblxRclOQfWxiT5NkjveOLkmyfZMfW8704yeVJntnKXpdkfpLjgccC30ry5tGjzyQLknwlyfL2s0+bfmSSk5KcB5zU6fW8vzwXuH00GKrqkqr6dgYfaK/TZUleBnf3NP85yd8nWZ3k6CQvT/K9Vm63Vu7EJMcnWdF6EC+YXHmShyT5VFv2oiQHtumHJDk1yZntvTksyVtamfOTPLyV2y3JP7Qe9reT7D5S9zFJvtPa+OJW5dHAM9v7/+YpXo+1wFms+/jbaFtf0973S9p28OA2/cgkb0uye5LvjZRfmOSyNvyU9ppdmOT0JDtOUfc27fev2jJHtPouT3JCez92S/KvI3Usmhifro4MZ16uyHAG6eQp6p3TkjwL+ATwgqq6ZmTWp4CXTWwrk5Z5RdvmLk7y8QzPb797P9CG35XhDNm/JPlCkreNrOIlbfmrJ/YhzS4ZerLfz8hZh7btXt5+3tSmLWzr/yxweVv2xJH/t6m2z81DVfkzww+wHXAxQ+/mo8CzgT2Ai2ZY7pYppv2Coaf0VeDAaZY7BDi2Df8n1t1Q9j+B/9eGvw7sM9K+eQxHru9s07YEtm/D1wHzpxgerefzwDPa8KOBK9vwkcCFwLab+n24F+/bG4APTTPvRcCZ7XV6FPBDhoOn57T3aEfgQQwPNvmrtswbgQ+34ROBf2A4MF3E8PjObdry32hl3ge8og0/rG0/D2mv+yqGsxILGB7r+dpW7kPAm9rwWcCiNvxUhs9/T9T9pVb3YoZnczNa9xR/70KGndtjgava330scEib/4iRsu9lOFiceP/f1oYvBnZtw+8A/hLYCvgOsKBNfxnDx+4m2nltW+4W4H0jdTx8ZPgk4IVt+Gxgz5HX7/Uz1PEj4EETr/Gm3uY6b7+3M3xEcY9J048E3gYcMbJt3tJ+P4Fh37BVG/8o8Mo2fB3Dk5P+a3tPtmnb4PdH3uNzWLeP2R/4xzZ8CPBjhsfPbtu2pSXAU4DLGLbr7YCVwJPb9nYX8LS2/FOAM0f+hs3qvRr98fTvGKrqliRPAZ7J0Pv5IsM//N2SvJphp/sI4OlVdf09VnTv7Ax8sR2Zb82wkwI4D/hgks8BX62qNUmWA59KshVwalVdvAH1PB9YnHXfg/DQJNu14WVV9Zv7+ofMMs8AvlBVdwI3Jvlnhp3NfwDLq+rHAEmuAc5oy1zG8P5POKWGU1rfT7Ia2H1SHX8EHDDSC9iG4YAF4Oyq+iXwyyQ3M+wIJ+rYo732Twe+NPKePGhk3ae2uq9IO3sxjqpaneQChtPEo56Y5L0M4b8dw+fSJzuFIdCObr9fBjweeCJwZmvnlgw73wkTp3+3A85K8vSq+g7w3CR/DjwYeDjDzvjrwCeBVyd5S1v/3jPUcSnwuSSnAqeO+zrMEbczHEwcyrBvmewY4OIkfzMy7XkMAba8vVbbMpz2H7UP8Pc1nGG7NcnXJ83/avt9IUM4Tjizqn4KkOSrDP9DBXytqn41Mv2ZDJeVflBV57dlVwOPTfIR4Jus+5/a7Hj6d0xVdWdVnVNV7wYOA14IPDrDdVSq6tM1XD+9meGf/h6SPBa4k2EjX8mw8c/kIwy9yScB/4t2Gq2qjmbouW4LnJdk96o6F3gWQ+/qxGzYDSZbMBxV7tl+dqqqiZsffrUB65lNxn2NJ/vtyPBdI+N38bv3IUz+PNrk8QAvGnlNH11VV45ZxxbAL0aW3bOqnjBNGzf0G6Hex9DTHF3uROCwtp39FetO1476IvDSDDd6VVV9v61j5Ugbn1RVfzR5wbYtnQM8I8P11Y8CL271fWKkvq8wfM3kC4AL2058fXX8McNXU+7FECSbU0fhLuClwN5J/s/kmVX1C4YzTK8bmRyGy0oTr9Xja5pLUusxsW3dyYZt75Pdvd+oqp8Dv8+wDbyW4eBps2SojiHJ45MsGpm0J8MptL8Fjm07iYnvnt16mnUsAI5nCMhiOPX2qiRPHSnzP6bodezAumctv2qk7G5VdVlVvZ/hUZK7J3kMcGNVfYJho91rA/7MMxhOtU2sf88NWHa2+ifgQRm+yAGAJHu060TfZrgmtWV7b54FfG+a9UznJUm2yHCddeK06qjTgdendRmSPHncFVfVfwDXJnlJWzYZbrJan18yxo1uVfVvwBUMB4YTtgd+3M5yvHya5a5h2NG+iyFgYfibF2S4mY8kWyX5L5OXbWH3VOAa1gXoTa0HO3FNmNZ7Oh34GPDp9dWR4U76XarqbIaDhB0Yetmbjar6NcOBw8uTHDpFkQ8yHGxPhN9ZwIvT7gxO8vC2Xxh1HvDCJNu01/8e9wNM4w/b+rYF/qSt59vAn2T41MJDgD9t035Hu5a7RVV9heGywYbsm+aUzemobmPaDvhIhtvR72C4HraUoVf6HuDyJL8EfsPwvbI/asttm+RihmtCdzBcO/ogQFXdmOQg4G/aP8BdwLkM1+lGHclwCvDnDCGxa5v+piTPbcutBL7F8HjItye5neEa1ob0VN8AHJfkUobt4lyGI8o5q6oqyZ8CH07yDuBWhutKbwL+BfgD4BKGI+4/r6p/T7sZaEw/ZAjihzJcE701v/s1wu8BPgxc2gLgWsbfgcEQbh9LMnHt8uTW3ulcCtyZ5BLgxKr60HrK/jVw0cj4u4ALGG5muoDpw/mLwAdo22FV3ZbhRqljkuzAsO18mGGbBPhAa//WDDv8r7b35RMM1+X+neGgcNTnGHbOZ8xQx9XA37VpAY5pvbfNSlX9LMm+wLlJ1k6ad1OSrwFvbuNXtNf7jLbN3c7Qk/3ByDLLkyxj2F5uZLjkcPMYTfkew5mEnYG/q6oVMNw4x7oD0k9W1UVJFk5adifg01n3kcK/GOuPn4N8opJ0L7QdyTeq6subui2bm3YNeoeqetembsvmKsl27V6RBzMcQC+tqn+daTnNzJ6qpFmj9bp2A/7bpm7LZu6EJIsZTsV/xkDtx56qJEmdeKOSJEmdGKqSJHViqEqS1ImhKklSJ4aqJEmdGKqSJHXy/wFIhzAIt4Fh2AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_axes([0,0,1,1])\n",
    "ax.bar(['SGDClassifier', 'Complement NaiveBayes', 'KNeighbors'],[accuracy, accuracy2, accuracy3])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SGDClassifier > KNeighbors > Complement NaiveBayes olarak sıralanmaktadır. Ama veri setimizde 59000 negatif kısma 1000 tane pozitif kısma ait veri olduğu için hepsini negatif seçsek bile accuracy yüksekmiş gibi durucak. Bu yüzden accuracy den çok veri setimizin açıklamasında da bahsedilen cost denklemini kullanalım."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Total cost = NTx500 + PNx10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix olarak bakarsak."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Her yöntemin costunu hesaplayacağız. Ne kadar az ise o kadar iyi sonuç elde etmiş demektir."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SGDClassifier'ın costu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "72940"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost1 = cm[0][1]*10 + cm[1][0]*500\n",
    "cost1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Complement NaiveBayes'ın costu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "64690"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost2 = cm2[0][1]*10 + cm2[1][0]*500\n",
    "cost2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "KNeighbors'ın costu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "89540"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost3 = cm3[0][1]*10 + cm3[1][0]*500\n",
    "cost3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cost değerleri. Az olan daha iyi."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeQAAAE/CAYAAACXV7AVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZaElEQVR4nO3debRlZX3m8e8DJZPIJBVCF8RCUy3BCaEaUNREsRFxgHRQMaYpXLS0S3CKRjG2gcRh4dIWg2NQEDBEQERBRbGCEKcwFIJAgUrJIEUjFjIIKsrw6z/2e+F4uVX3ljXct6q+n7XOunu/+333fs85++7nvHvve26qCkmSNL3Wm+4OSJIkA1mSpC4YyJIkdcBAliSpAwayJEkdMJAlSerAjOnuwB9q6623rtmzZ093NyRJWi6XXnrpbVU1c3z5GhvIs2fPZsGCBdPdDUmSlkuSGycq95S1JEkdMJAlSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjqwxv5zCUlaU8w+4qvT3QWtgBuOftFq2Y4jZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR0wkCVJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdcBAliSpAwayJEkdMJAlSeqAgSxJUgemFMhJ3pxkYZKrknwuyUZJdkhyUZJFSU5LskGru2GbX9SWzx5Zzzta+Y+SvGCkfJ9WtijJESv9WUqS1LlJAznJLOANwNyqejKwPnAg8H7gmKr6U+AO4JDW5BDgjlZ+TKtHkp1auycB+wAfT7J+kvWBjwEvBHYCXtnqSpK0zpjqKesZwMZJZgCbALcAzwPOaMtPAvZv0/u1edryvZKklZ9aVb+tquuBRcBu7bGoqq6rqt8Bp7a6kiStMyYN5Kq6Gfgg8FOGIL4LuBS4s6rub9UWA7Pa9Czgptb2/lb/saPl49osrVySpHXGVE5Zb8kwYt0B+C/AoxlOOa92SQ5NsiDJgiVLlkxHFyRJWiWmcsr6+cD1VbWkqu4DzgT2BLZop7ABtgNubtM3A9sDtOWbA78YLR/XZmnlj1BVx1XV3KqaO3PmzCl0XZKkNcNUAvmnwB5JNmnXgvcCrgbOBw5odeYBZ7Xps9s8bfk3q6pa+YHtLuwdgDnAxcAlwJx21/YGDDd+nb3iT02SpDXHjMkqVNVFSc4Avg/cD1wGHAd8FTg1yXta2fGtyfHAZ5MsAm5nCFiqamGS0xnC/H7gsKp6ACDJ4cC5DHdwn1BVC1feU5QkqX+TBjJAVR0JHDmu+DqGO6TH170XeNlS1vNe4L0TlJ8DnDOVvkiStDbym7okSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR2YMd0d6MHsI7463V3QCrjh6BdNdxckaYU5QpYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR0wkCVJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdcBAliSpAwayJEkdMJAlSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMzprsD0ppm9hFfne4uaAXdcPSLprsL0iM4QpYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDUwrkJFskOSPJD5Nck+QZSbZKMj/Jte3nlq1ukhybZFGSK5LsMrKeea3+tUnmjZTvmuTK1ubYJFn5T1WSpH5NdYT8z8DXq2pH4GnANcARwHlVNQc4r80DvBCY0x6HAp8ASLIVcCSwO7AbcORYiLc6rxlpt8+KPS1JktYskwZyks2B5wDHA1TV76rqTmA/4KRW7SRg/za9H3ByDS4EtkiyLfACYH5V3V5VdwDzgX3ass2q6sKqKuDkkXVJkrROmMoIeQdgCfCZJJcl+XSSRwPbVNUtrc7PgG3a9CzgppH2i1vZssoXT1AuSdI6YyqBPAPYBfhEVT0d+BUPn54GoI1sa+V37/clOTTJgiQLlixZsqo3J0nSajOVQF4MLK6qi9r8GQwBfWs73Uz7+fO2/GZg+5H227WyZZVvN0H5I1TVcVU1t6rmzpw5cwpdlyRpzTBpIFfVz4CbkjyxFe0FXA2cDYzdKT0POKtNnw0c1O623gO4q53aPhfYO8mW7WauvYFz27JfJtmj3V190Mi6JElaJ8yYYr3XA6ck2QC4Dng1Q5ifnuQQ4Ebg5a3uOcC+wCLg160uVXV7kncDl7R6/1RVt7fp1wEnAhsDX2sPSZLWGVMK5Kq6HJg7waK9JqhbwGFLWc8JwAkTlC8AnjyVvkiStDbym7okSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR0wkCVJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdcBAliSpAwayJEkdMJAlSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR0wkCVJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdcBAliSpAwayJEkdMJAlSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOTDmQk6yf5LIkX2nzOyS5KMmiJKcl2aCVb9jmF7Xls0fW8Y5W/qMkLxgp36eVLUpyxEp8fpIkrRGWZ4T8RuCakfn3A8dU1Z8CdwCHtPJDgDta+TGtHkl2Ag4EngTsA3y8hfz6wMeAFwI7Aa9sdSVJWmdMKZCTbAe8CPh0mw/wPOCMVuUkYP82vV+bpy3fq9XfDzi1qn5bVdcDi4Dd2mNRVV1XVb8DTm11JUlaZ0x1hPxh4G3Ag23+scCdVXV/m18MzGrTs4CbANryu1r9h8rHtVla+SMkOTTJgiQLlixZMsWuS5LUv0kDOcmLgZ9X1aWroT/LVFXHVdXcqpo7c+bM6e6OJEkrzYwp1NkTeGmSfYGNgM2Afwa2SDKjjYK3A25u9W8GtgcWJ5kBbA78YqR8zGibpZVLkrROmHSEXFXvqKrtqmo2w01Z36yqVwHnAwe0avOAs9r02W2etvybVVWt/MB2F/YOwBzgYuASYE67a3uDto2zV8qzkyRpDTGVEfLSvB04Ncl7gMuA41v58cBnkywCbmcIWKpqYZLTgauB+4HDquoBgCSHA+cC6wMnVNXCFeiXJElrnOUK5Kq6ALigTV/HcIf0+Dr3Ai9bSvv3Au+doPwc4Jzl6YskSWsTv6lLkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR0wkCVJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdcBAliSpAwayJEkdMJAlSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR0wkCVJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdcBAliSpAwayJEkdMJAlSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHZg0kJNsn+T8JFcnWZjkja18qyTzk1zbfm7ZypPk2CSLklyRZJeRdc1r9a9NMm+kfNckV7Y2xybJqniykiT1aioj5PuBt1TVTsAewGFJdgKOAM6rqjnAeW0e4IXAnPY4FPgEDAEOHAnsDuwGHDkW4q3Oa0ba7bPiT02SpDXHpIFcVbdU1ffb9N3ANcAsYD/gpFbtJGD/Nr0fcHINLgS2SLIt8AJgflXdXlV3APOBfdqyzarqwqoq4OSRdUmStE5YrmvISWYDTwcuArapqlvaop8B27TpWcBNI80Wt7JllS+eoFySpHXGlAM5yabAF4A3VdUvR5e1kW2t5L5N1IdDkyxIsmDJkiWrenOSJK02UwrkJI9iCONTqurMVnxrO91M+/nzVn4zsP1I8+1a2bLKt5ug/BGq6riqmltVc2fOnDmVrkuStEaYyl3WAY4HrqmqD40sOhsYu1N6HnDWSPlB7W7rPYC72qntc4G9k2zZbubaGzi3Lftlkj3atg4aWZckSeuEGVOosyfwP4Erk1zeyv4eOBo4PckhwI3Ay9uyc4B9gUXAr4FXA1TV7UneDVzS6v1TVd3epl8HnAhsDHytPSRJWmdMGshV9R1gaX8XvNcE9Qs4bCnrOgE4YYLyBcCTJ+uLJElrK7+pS5KkDhjIkiR1wECWJKkDBrIkSR0wkCVJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdcBAliSpAwayJEkdMJAlSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR0wkCVJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdcBAliSpAwayJEkdMJAlSeqAgSxJUgcMZEmSOmAgS5LUAQNZkqQOGMiSJHXAQJYkqQMGsiRJHTCQJUnqgIEsSVIHDGRJkjpgIEuS1AEDWZKkDhjIkiR1wECWJKkDBrIkSR3oJpCT7JPkR0kWJTliuvsjSdLq1EUgJ1kf+BjwQmAn4JVJdpreXkmStPp0EcjAbsCiqrquqn4HnArsN819kiRpteklkGcBN43ML25lkiStE2ZMdweWR5JDgUPb7D1JfjSd/VmDbA3cNt2dWFXy/unuwVrJfUbLw/1l+TxuosJeAvlmYPuR+e1a2e+pquOA41ZXp9YWSRZU1dzp7ofWHO4zWh7uLytHL6esLwHmJNkhyQbAgcDZ09wnSZJWmy5GyFV1f5LDgXOB9YETqmrhNHdLkqTVpotABqiqc4BzprsfaylP82t5uc9oebi/rASpqunugyRJ67xeriFLkrROM5BXgyTvTLIwyRVJLk+ye5IZSd6X5NpWdnmSd460eaCVLUzygyRvSbLeyPLdknyrfd3oZUk+nWSTJAcn+ehK7Ps5SbZo029Ick2SU5K8dF34itMkf5zk1CQ/SXJpez3+6yrc3l8k+cqqWv8k256d5K+XsaySvH6k7KNJDp5kna9NctAf2J8Tk1zffg9+mOTIP2Q966Ik94xM75vkx0kel+SoJL9O8kcT1V3G+h46DiyjzgVJHnGn9co+Jq3NurmGvLZK8gzgxcAuVfXbJFsDGwDvAf4YeEpV3ZvkMcBbRpr+pqp2buv4I+DfgM2AI5NsA3weOLCq/rPVOQB4zMruf1XtOzL7OuD5VbW4zU/5TvgkM6rq/pXauVUsSYAvAidV1YGt7GnANsCPp7Nvq8hs4K8Z9rWJ/Bx4Y5J/ad+oN6mq+uQK9unvquqMJBsBVyc5uaquX8F1rjOS7AUcC7ygqm4cdmluYzjWvH2q6xl3HFht2u9gqurB6dj+6uYIedXbFritqn4LUFW3AXcCrwFeX1X3tvK7q+qoiVZQVT9n+EKUw9sOehhDSPznSJ0zqurW0XZJXpLkojaC/vcW5CT585FR+WVJHpNk2zbivjzJVUme3erekGTrJJ8EHg98LcmbRz/1JpmZ5AtJLmmPPVv5UUk+m+S7wGdX0uu5Oj0XuG80VKrqB1X17Qw+0F6rK5O8Ah4a4f5HkrOSXJfk6CSvSnJxq/eEVu/EJJ9MsqCNXl48fuNJHp3khNb2siT7tfKDk3wpyfz2/hye5G9bnQuTbNXqPSHJ19vI/ttJdhzZ9rFJvtf6eEDb5NHAs9s+8OYJXo8lwHnAvAn6+pr23v+g7QubtPKjkrw1yY5JLh6pPzvJlW161/aaXZrk3CTbTrDtjdrPX7U2/9C2d1WS49r78YQk3x/Zxpyx+aVtI8NZn6sznL06dYLtrrGSPAf4FPDiqvrJyKITgFeM7Sfj2vxN298uT/IvGf7PwEPHgTb9rgxn5r6T5HNJ3jqyipe19j8eO4Y022cYQV+bkTMdbb+9qj3e1Mpmt/WfDFzV2p448rs20b65dqgqH6vwAWwKXM4wovo48OfAU4HLJml3zwRldzKMzs4E9ltKu4OBj7bpLXn4xr3/BfzfNv1lYM+R/s1g+MT8zla2PvCYNn0DsPUE06Pb+TfgWW36T4Br2vRRwKXAxtP9PvyB790bgGOWsuyvgPnttdoG+CnDh6+/aO/TtsCGDF9w84+tzRuBD7fpE4GvM3wonsPwdbEbtfZfaXXeB/xNm96i7UOPbq/9IoYzIjOBu4DXtnrHAG9q0+cBc9r07sA3R7b9+bbtnRi+R57RbU/wfGczHBwfD/yoPe+PAge35Y8dqfsehg+bY/vAW9v05cAObfrtwP8BHgV8D5jZyl/B8GePY/28vrW7B3jfyDa2Gpn+LPCSNn0+sPPI6/f6Sbbx/4ANx17j6d7nVuK+ex9wO/DUceVHAW8F/mFkv7yn/fwzhmPDo9r8x4GD2vQNDN/G9d/a+7FR2/+uHXl/L+DhY8y+wL+36YOBW4DHAhu3/WgusCtwJcM+vSmwEHh629ceBPZo7XcF5o88h7XmfRr/8JT1KlZV9yTZFXg2w4jrNIYDxUOSvJrhYP1Y4JlVddMjVvSH2Q44rY0GNmA4uAF8F/hQklOAM6tqcZJLgBOSPAr4UlVdvhzbeT6wU4bTYQCbJdm0TZ9dVb9Z0SfSoWcBn6uqB4Bbk/wHw8Hql8AlVXULQJKfAN9oba5k2AfGnF7Dqbhrk1wH7DhuG3sDLx0ZgWzE8IEH4Pyquhu4O8ldDAfSsW08tb3+zwQ+P/K+bDiy7i+1bV+dduZkKqrquiQXMZzaHvXkJO9h+OCwKcN3Cox3OkMYHt1+vgJ4IvBkYH7r5/oMB+8xY6esNwXOS/LMqvoe8NwkbwM2AbZiOJh/Gfg08Ookf9vWv9sk27gCOCXJl4AvTfV1WAPcx/Ah5BCGY8t4xwKXJ/ngSNleDOF3SXudNma4TDFqT+CsGs7s3Zvky+OWn9l+XsoQrGPmV9UvAJKcyfD7U8AXq+pXI+XPZrgUdmNVXdjaXgc8PslHgK/y8O/TWsdT1qtBVT1QVRdU1ZHA4cBLgD/JcN2YqvpMDdeL72I4WDxCkscDDzD8gixk+MWZzEcYRrFPAf437bRfVR3NMGLeGPhukh2r6lvAcxhGdCdm+W7EWY/h0+zO7TGrqsZuFPnVcqynN1N9ncf77cj0gyPzD/L7922M/5vD8fMB/mrkdf2TqrpmittYD7hzpO3OVfVnS+ljWD7vYxjhjrY7ETi87Wv/yMOnmEedBrw8w01xVVXXtnUsHOnjU6pq7/EN2/50AfCsDNeTPw4c0Lb3qZHtfYHh37i+GLi0hcCytvEihn/9ugtDEK0tg5QHgZcDuyX5+/ELq+pOhjNbh40Uh+FS2Njr9MRaymW0ZRjbrx5g+fb18R46blTVHcDTGN7/1zJ86ForGcirWJInJpkzUrQzwym/44GPtoPL2P+E3mAp65gJfJIhXIvhVOG8JLuP1PkfE4x0Nufh7wSfN1L3CVV1ZVW9n+FrS3dM8jjg1qr6FMMOv8tyPM1vMJwaHFv/zsvRtmffBDbM8E9NAEjy1HZt7NsM1+HWb+/Pc4CLl7KepXlZkvUyXFceOxU86lzg9WnDlSRPn+qKq+qXwPVJXtbaJsMNactyN1O4MbCqfghczfDBcsxjgFvaGZZXLaXdTxgO1O9iCGcYnvPMDDc/kuRRSZ40vm0Lyt2Bn/Bw+N7WRs5j18BpI7dzgU8An1nWNjL81cL2VXU+wweMzRlG92uFqvo1wweOVyU5ZIIqH2L4oD4WnOcBB6TdgZ1kq3ZcGPVd4CVJNmqv/SPufViK/97WtzGwf1vPt4H9M/x1yKOBv2xlv6ddu16vqr7AcJljeY5Na5S15dNgzzYFPpLhTwbuZ7j2dyjDaPjdwFVJ7gZ+A5zEcE0LYOMklzNc/7qf4TrZhwCq6tYkBwIfbL88DwLfYrgmOeoohlOWdzCEyw6t/E1JntvaLQS+xvD94X+X5D6G63XLM0J+A/CxJFcw7FPfYvgku0arqkryl8CHk7wduJfhWtqbgO8AzwB+wPBp/21V9bO0G6em6KcMIb4ZwzXge0dOL8Owf3wYuKKFx/VM/QAIQzB+IsnYtdpTW3+X5grggSQ/AE6sqmOWUfe9wGUj8+8CLmK48esilh7spwEfoO2LVfW7DDeVHZtkc4b958MM+yXAB1r/N2AIjDPb+/IphmuRP2P4UDnqFIaD+zcm2caPgX9tZQGObSPHtUZV3Z5kH+BbSZaMW3Zbki8Cb27zV7fX+httf7uPYQR940ibS5KczbCv3MpwieSuKXTlYoazF9sB/1pVC2C4wZCHP8h+uqouSzJ7XNtZwGfy8J99vmNKT34N5Dd1SdOgHYi+UlVnTHdf1jbtmvvmVfWu6e7L2ijJpu3emE0YPnwfWlXfn6ydJucIWdJao434ngA8b7r7shY7LslODJcOTjKMVx5HyJIkdcCbuiRJ6oCBLElSBwxkSZI6YCBLktQBA1mSpA4YyJIkdeD/A4t/ROOPY+O4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_axes([0,0,1,1])\n",
    "ax.bar(['SGDClassifier', 'Complement NaiveBayes', 'KNeighbors'],[cost1, cost2, cost3])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Precision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'SGDClassifier Precision-Recall curve ')"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA5mklEQVR4nO3dd3wU5fb48c9JaGnUBAQSCB0BMUhoggi2i4D1qmAvWK/Y67Viuz+s33u5FsCuV8CuqCg2qggCGlCqlGhCbyFACGnn98dM4hKS7KZsNps979drX9mZeWbmzG52z87zzDyPqCrGGGNCV1igAzDGGBNYlgiMMSbEWSIwxpgQZ4nAGGNCnCUCY4wJcZYIjDEmxFkiMGUSkXEi8j8/bn+FiAxxn4uIvC4ie0TkJxE5QUTW+Gvf/iAibURkv4iEeyl3sYh8XV1x+ZuIvCEij7vPh4hIeqBjMr6zRFBDicggEVkgIntFZLeI/CAifTyWtxSRl0Vks/vFs8H9MHZ1lyeKiLrL9ovINhH5XEROLWFfF4nIErfcFhH5UkQGVcdxqmp3VZ3tTg4CTgXiVbWvqs5T1S5VtS/39clxj3O3iHxT+HpVFVX9U1WjVTXfS7l3VPW0qtx3IRGZLSLZ7nHuFJGPRKSlP/ZlagdLBDWQiDQEPgf+CzQFWgOPAIfc5c2ABUAkcAIQAxwHzMH5IvXUWFWjgWOBb4CPReQKj33dDvwb+BfQAmgDvAic5ZeDK1tbIFVVD1R2QyJSp5RFT7mvRzywHXijhHVFRIL9szHWPc6OQDTwTIDjqVJlvL+mIlTVHjXsASQDGWUsfxxYBoSVUSYRUKBOsfl3AttwfgQ0AvYD55exnXHA/zym3we2AnuBuUB3j2XDgZXAPmATcKc7PxYnsWUAu4F5hbEDqcApwBggG8h3Y3oEGAKke2y/FfAhsAPYCNxcLM4PgP8BmcDVJRzLG8DjHtMjgP3u89nAE8APwEGcL9CuOMlzN7AGuMBj3QjgWeAP97WY78477HUHrgA2uK/JRuBij/nzPbZ3PLDY3dZi4HiPZbOBx9zY9gFfA7FlvGezPY8f+AewwmO63Mflw3tf9NoWf99KiK+7x/63AfeV8v4Uf/9TgXuA5Tg/iu4BPii27f8AE9znjYBXgS04/4+PA+GB/nzXxEew/+qprdYC+SLypoicLiJNii0/BfhYVQsqsO2PgOZAF2AA0AD4uBzrfwl0crfxM/COx7JXgetUNQboAXzvzr8DSAficM467sP5siyiqq8C1wM/qlO18rDncvcX+mc4CbA1cDJwq4j8zaPYWTjJoHGxuI4gItHAxcAvHrMvBa7FOcPagfNlNcU91tHAiyLSzS37DNAb5wu8KXA3cNj7ISJRwATgdPc1OR5IKSGWpsAXbtlmwHPAF+6ZX6GLgCvdWOrhJHSv3G2cC6zziKmix1XWe+8TEYkBvgW+wknsHYHvyrGJC3ESeGNgGjDc3SZuu8wF7rGBk1jy3H30Ak4Dri5vzKHAEkENpKqZOPXlCrwM7BCR6SLSwi0Si/PLDAAROVNEMkRknw8NkJvdv01xvnR2qmpeOWJ7TVX3qeohnF/hx4pII3dxLtBNRBqq6h5V/dljfkugrarmqlP3X95OrvoAcar6qKrmqOoGnNdmtEeZH1X1E1UtUNWDpWznThHJwPlijMb5ZV7oDVVd4b4ew3CqqV5X1TxV/QXnbOR8NyldBdyiqptUNV9VF7ivSXEFQA8RiVDVLaq6ooQyI4DfVfVtd19TgdXAGR5lXlfVte5xvQcklfFaAUwQkb3ATpz/l5vc+SMrelxe3ntfjQS2quqzqprtbm9ROdafoKppqnpQVf/ASUjnuMtOArJUdaH7WRkO3KqqB1R1O/B/HP7/YlyWCGooVV2lqleoajzOr+tWOHX5ALtwvlgLy05X1cbAbTi/FsvS2v27291OrK/1rSISLiLjRWS9iGTinKqD80UD8HecD98fIjJHRAa485/G+eL92m3UvteX/RXTFmjlJrwM98v8PpwzjEJpPmznGVVtrKpHqeqZqrq+lPXbAv2K7e9i4Cic420AeK57BHXaOkbhnOlsEZEvSmmcboVTFePpD/56r8Aj8QNZOEkMEZnocUHAfR5lblbVRkBPoAlOm0iFj8uH995XCSVtvxyKv8dTcM4SwDlrKjwbaAvUxXndC49zEs7ZjCnGEkEQUNXVOKe5PdxZ3wFnV7BB8xycRtI1wI84da1n+7juRTjVL6fg1L8muvPFjXOxqp6F82H7BOeXK+6vvjtUtT1wJnC7iJxczrjTgI3ul3jhI0ZVh3uUqWxXup7rpwFziu0vWlVvwPmVnQ108LpB1ZmqeipO4l6NcxZT3GacLy5PbXDqtb1t/3o3rmhV/VcJy3/FqRt/QUSkEsdV5ntfDmlA+1KWHcC5AKLQUSWUKf4evw8MEZF4nP/twkSQhvO/HetxnA1VtXs54w0JlghqIBHpKiJ3uP/ciEgCzq+ehW6R53B+5b0tIh3cq1xiKKO6QERaiMhY4GHgn271yV7gIZwvibNFJFJE6rrtEk+VsJkYnA/XLpwP7L88tl9PnGvjG6lqLk6DbYG7bKSIdHS/iPbiNAiXt33jJ2CfiNwjIhHuL9Qe4nFJbRX7HOgsIpe6r0ldEekjIke7bTOvAc+JSCs3lgEiUt9zA+5rfpZbL38IpxG8pOOe4e7rIhGpIyKjgG5uDFXhTZwzpzMrcVylvvfl9DnQUkRuFZH6IhIjIv3cZSk4df5NReQo4FZvG1PVHTiN46/j/FBY5c7fgtOo/qyINBSRMPezcmIF467VLBHUTPuAfsAiETmAkwB+w2l0RVV3Av1xfr3Nd8un4HxYbyi2rQx3G7/iVNucr6qvFS5U1WeB24EHcBpI04CxOL/oi3sLp8piE87VQQuLLb8USHWrDq7HqXIAp4HxW5wvwh+BF1V1lq8vhhtnPk79chLO1Tc7gVdwfp1WOVXdh9O4OBrnF/tW4Emg8Mv+TpzXdDFONduTHPl5CsN5bTe7ZU7kyPcHVd2Fc2x34HzR3g2MdN/nqjiWHJyraR6sxHF5e+99jWUfziXOZ7j7/h0Y6i5+G+digFScL/F3fdzsFJwzlSnF5l+GU1W6EtiDcyGB3U9RAil/m50xxpjaxM4IjDEmxFkiMMaYEGeJwBhjQpwlAmOMCXFB13FTbGysJiYmBjoMY4wJKkuXLt2pqnElLQu6RJCYmMiSJUsCHYYxxgQVESl+93oRqxoyxpgQZ4nAGGNCnCUCY4wJcZYIjDEmxFkiMMaYEOe3RCAir4nIdhH5rZTlIiITRGSdiCwXkeP8FYsxxpjS+fOM4A2cUZ5KczpOr5SdcIYHfMmPsRhjjCmF3+4jUNW5IpJYRpGzgLfcIQsXikhjEWnp9iNe5Ran7mbe2h3+2HSZIuvX4YrjE2lQN7za922MMb4I5A1lrTl82Ll0d94RiUBErsU5a6BNmzYV2tnPf+zhv7PWVWjdiirs4fvY+MYM6NCs7MLGGBMgQXFnsapOBiYDJCcnV2gAhetO7MB1J3odWbBK/bRxNxdM+pECG/PBGFODBfKqoU04A1kXiseHMVqNMcZUrUAmgunAZe7VQ/2Bvf5qHzDGGFM6v1UNichUYAgQKyLpOIOm1wVQ1Yk4A3YPB9YBWcCV/ool0F6cvY73l6Sxbsd+2jaL4uGR3WjesEGgwzLGGMC/Vw1d6GW5Ajf6a/81QfqeLAB+WLeraN5vmzI5o2dLhvUo/xjaefkFbMo4SNrug/y+fR/rtu+nQGHpH7uJrFeHFg3r89LFvQkLkyo7BmNM7RcUjcXB6qyk1kxbnMaVxyfSu20Tdu7PYfiEeWWuk1+gpO46wPzfd7Jm2z6WpWWQkZXLpoyDpa7TslEDNu48QEqacjA3n6j6R76tuw/k8MeuA2zLzGb11n3sz84jbU8WqTuzqF83jJy8Asad2Z3+7e3qJmNCjSUCPwoPE967bkDR9K4DOYctP5SXz5LUPXy+fAvL0jJYuSWz1G31TWxKs+h6JCc2JTa6Hu1jo0mMjSS6fh1EhJdmr+fJr1bzacpmVm7Zy5aMbHZn5bAsLYOCMi5aatWoAXsPwtbMbH7+cw/JbZuw60AOW/dms2brPjbvPUh2bgGH8vI5r3c83Vs1qvTrYoypWSwRVKPCq0iv/9/PiPw17al9XBTdWzVicKdYerdtQrvYKES8V/XMXLEVgPs+/rVoXkTdcE7oFEd4mND1qBjaNoukeUwDEmOjiG8SQd1w51qB7Nx8uj74FU99tYanvlpT6j7yC5RHz7JEYExtY4mgGu09mFv0XBXax0bRr30zzujZkt6JTahfp+J3H0+8pDfjv1zFqD5t6NQimtjo+j6vW7+OkxCi69fhBDcBRdQLJ7FZFImxUcRG16PLA1/x1o9/8NPG3Uy5pj9No+pVOFZjTM1iiaAaDejQjCuOT2TMoHYkNI2s0m0f1agB/x7dq0Lrigip40f4VHb11n28vySNM45tRavGERXanzGmZhENsrtek5OT1cYsDoyJc9Yz/svVAByb0JhPbxxIQYGy71AeEXXDqVfHejU3pqYSkaWqmlzSMjsjMD67amA7Fm/czXert7MsLYPEe78oWlaYGMBpc1i5JZM9B3JYnLqHtN1Z7D+Ux+LU3dQNDyM8TPjkHwNp0ywSVSW/QKkTbknEmECxMwJTbr0f+4ZdB3KIqhdOu7goftvkXO1Ur45zGWpp+rVryoadB9ix71DRPBGoEyZ8/I+BHN2yIdv3ZZOTV0CbppFlNpJn5+YDWK+uxviorDMCSwSm0pIf/4ad+51LY+Ni6tM+Noo+iU05NqExXY+KIb5JRNGX+tI/9vD3lxYAcFLX5qTtzuL37fuP2ObzF/UioUkkq7dmsmPfIdZs28/arfsQcdopwGncnnnbYPYcyGHF5r38uTuLA4fyWfLHbiLqhhMmwsRLetPEGraNsURgaq5tmdn0+9d3AFw2oC079x9ixq9bSyzbLKoeDeqGk5zYhE9TNpe6zeYx9dnucdYRXb8OYwa147ze8azfsZ/s3HyOiW9Ma2vsNiHEEoEJGvkFSof7ZvD34+LpelQMPeMbkRgbRbOoeoe1I6zYvJcRE+Zz97AuRNYNp3OLGDo2j6ZJVD3qhoexZus+/vbvucTF1D+sKsrTzw+eapfBmpBhicCEtBvf+ZmFG3Zx+fGJxDeJ4Pb3lhUt69uuKbn5BRzMyWfsSR0Z2bNVACM1xn/sqiET0l64+LjDpgd1jKWvWx3108bdxDSoQ3ZuPgs37LJEYEKSJQITcpo3bMAvD55K3TphRLsd9CXe+wX/W/gnPVo14qSuza2bcBNSLBGYkFT8SqLY6Prs3H+Iez9y+mp6YMTRDO3anA5x0YEIz5hqZXfxGAMseeAU+iY2LZp+/ItVvDBrXQAjMqb6WCIwxvXe9QP47ZG/8erlTnvaRz9vYnl6Bnn5pd8kZ0xtYFVDxniIrl+Hk49uUTR95vM/AHBp/7ak7ckivkkEt53SmbwCPeKSVmOClV0+akwJVJV2/5zhtdzdw7pwcd+2NKgXxta92cQ3iSTchgo1NZDdR2BMBe3NyuXFOes4o2crEppEcuyjX3NUwwZszcwudZ2+7ZrSo1UjHhx5tE+DChlTHSwRGFPF0vdksW77fq54fTHg3Jswf93Ow8pcO7g9qTsPkJKWwfZ9h6hXJ4wxg9pxz7CugQjZhDhLBMZUg+37nLOEm6f+wsINu0stV69OGCjk5Bdw7eD2NIqoy6g+Cew9mMvy9AzCw8IY0iWOhg3qVlfoJgRYIjCmGm3PzOaxL1Zxfu94esY3onGkc8+C5/gNvvjHkA5kZueyYccBmkXXZ1j3o9iccZANOw8QG12PywYkEhfj+5CkJrRZIjCmhtmccZB3Fv3BsO4tOeP5+UTXr8PIni05NqEx/3RvavNF22aR9GjViAv6JHBi5zg/RmyCnSUCY4KIqnLF64s5K6kVxyY0JrFZFEOfmc3QLnEM7BhLt1YNGfTkrCPW696qIXXCw3j18mRio+1MwRzOEoExtdDv2/bRplkkXR746ohlr16ezE+pu8k6lM9xbRtzTq/4AERoahJLBMbUYoXDdmbl5HPcY9+UWObzmwbRo3Wj6gzL1DBlJQK7LdKYINegbjgN6obTNKoeEy85jtaNI/jn6V2Zek1/mrmd6z362coAR2lqMp+6mBCR5sBAoBVwEPgNWKKq1gmLMTXIsB4tGdajZdH0D/eexMDx37N8Uwa5+QXUtS4xTAnKTAQiMhS4F2gK/AJsBxoAZwMdROQD4FlVzfRznMaYCmhQN5xdB3IA6HT/lwzpEkdevpK66wDbMrO5cWhHbjqpk3WLEeK8nREMB65R1T+LLxCROsBI4FTgQz/EZoypAq9clszVbzntarPX7Dhs2b+//Z2BHWPp49EFtwk9ZSYCVb2rjGV5wCdlrS8iw4D/AOHAK6o6vtjyNsCbQGO3zL2q6r2nL2OMz07p1oLU8SN4df5Gjm4ZQ6+EJkTUC+f/fbmKSXM28NumvZYIQlyFrxoSkStV9fUylocDa3HOGNKBxcCFqrrSo8xk4BdVfUlEugEzVDWxrP3aVUPGVI0vlm/hxik/HzbvqIYNODOpFUO6xDGgfTPrNK8W8ddVQ494Wd4XWKeqG1Q1B5gGnFWsjAIN3eeNgM2ViMcYUw7DjzmKxpGH92e0NTObyXM3cNHLi7j3Q9/vcDbBzVtj8fLSFgEtSllWqDWQ5jGdDvQrVmYc8LWI3AREAad42aYxpoqICCkPnXbYvP/7Zi0HDuXxyvyNvLskjRm/buGuYV24tH9bOzuoxbw1FrcA/gbsKTZfgAVVsP8LgTdU9VkRGQC8LSI9il+WKiLXAtcCtGnTpgp2a4wpyW2ndgbglfkbAdh3KI+HPl3BQ5+u4N1r+9OjdSMi6oYTZlcZ1SreEsHnQLSqphRfICKzvay7CUjwmI5353kaAwwDUNUfRaQBEItzmWoRVZ0MTAanjcDLfo0xlZQ6fgQAk+eu518zVgMwavJCAMLDhG9uG0z7uOiAxWeqVpltBKo6RlXnl7LsIi/bXgx0EpF2IlIPGA1ML1bmT+BkABE5GucehR0YY2qEawd3YPrYgbRtFlk0L79A2bq39BHaTPDx222G7uWlY4GZwCrgPVVdISKPisiZbrE7gGtEZBkwFbhCg63zI2NquZ7xjZlz11BSx49g0qW9Abjj/WUUFNhHtbbwqYuJinLvCZhRbN5DHs9X4nRdYYwJAu1iowDYsjebvv/6jgX3nuSMuGaCmr2DxhifdW4Rw9tj+gKwc/8h7v/YLjGtDSwRGGPK5YROcUXJ4P2l6Xz565YAR2Qqy+dE4N4FXOq0MSZ0nNApjuM7NANg1dZ9AY7GVFZ5zggmeZk2xoSQKdf0D3QIpor4nAhUdWlZ08aY0DThu9/5bdPeQIdhKsFbFxOf4fQHVCJVPbO0ZcaY0JG+56ANhRnEvF0++ky1RGGMCUozbj6B4RPmBToMU0nexiOYU/hcRCKANqq6xu9RGWOCgroVBh8sTefEznFE1AsPcESmInxqIxCRM4AU4Ct3OklEincXYYwJMdm5+QB8u2obvR//JsDRmIrytbF4HM74AhkAbid07fwSkTEmaPRu25R/nt4VgKycfF7/YSPrd+zHeooJLr4mglxVLX5ZgL3TxhiuO7EDfd2hLh/5bCUnPzuHb1dt97KWqUl8TQQrROQiIFxEOonIf6ma8QiMMbXAe9cPYMygvyoJrnt7CVMW/RnAiEx5+JoIbgK6A4dwegnNBG71U0zGmCD04MhuzLt7KAAFCvd9/CszV2wNcFTGFz4lAlXNUtX7ccYOGKqq96uqdUhujDlMQtNI3h7Tl57xzj0F1729lMWpuwMclfHG16uG+ojIr8By4FcRWSYivf0bmjEmGJ3QKY7pYwcVTb+7OK2M0qYm8LVq6FXgH6qaqKqJwI3A636LyhgT9FLHj6B144hAh2F84GsiyFfVotsH3eEr8/wTkjGmttiUcZAPlqYz//edgQ7FlKHMRCAix4nIccAcEZkkIkNE5EQReRGYXS0RGmOCVpg4fy95dRGpOw8ENhhTKm99DT1bbPphj+d2H4Expkzr/zWcdv90RqvdlplNojvUpalZvPU1NLS6AjHG1D4iwpSr+3HRK4sCHYopg8+D14vICJx7CRoUzlPVR/0RlDHGmOrj6+WjE4FRODeWCXA+0NaPcRljjKkmvl41dLyqXgbsUdVHgAFAZ/+FZYypLQ66PZS+/kNqYAMxpfI1ERx0/2aJSCsgF2jpn5CMMbWJuFcOfbViK/sP2VXnNZGvieBzEWkMPA38DKTi9DlkjDFlOqlrC+51u6q27qlrJl/7GnpMVTNU9UOctoGuqvqgf0MzxtQWP67fBcAt01ICG4gpkbfB688tYxmq+lHVh2SMqW1G9UlgztodrNm6L9ChmBJ4u3z0jDKWKWCJwBjj1fBjWnJur9b8lLqbnfsPkXUonzbNIgMdlnF5u6HsyuoKxBhTu330yyYAkh//tmjeyV2b88z5x9Ikql6gwjL43lhsjDGVcmq3FgC0j/urm4nvVm+nzxPfsi87N1BhGcpxZ7ExxlTGy5clHza9YvNeRkyYT16B8vLcDdx+WpcARWbsjMAYExDdWzVi2rX9AZjw/Tp27T8U4IhCl69dTESKyIMi8rI73UlERvqw3jARWSMi60Tk3lLKXCAiK0VkhYhMKV/4xphg1r99s6Ln82zMgoDx9YzgdZyB6we405uAx8taQUTCgReA04FuwIUi0q1YmU7AP4GBqtoduNXnyI0xtcIXNzvDWt76bgp7DuQEOJrQ5Gsi6KCqT+F0LYGqZuF0PleWvsA6Vd2gqjnANOCsYmWuAV5Q1T3udrf7HLkxplZo5zFGQa/HvglgJKHL10SQIyIRuIPRiEgHnDOEsrQGPEetTnfneeoMdBaRH0RkoYgMK2lDInKtiCwRkSU7duzwMWRjTDCIrFeH+ffY0CeB5GsiGAd8BSSIyDvAd8DdVbD/OkAnYAhwIfCy26fRYVR1sqomq2pyXFxcFezWGFOTxDeJZFDHWACmLPozwNGEHl/7GvoaOBe4AqezuWRVne1ltU1Agsd0vDvPUzowXVVzVXUjsBYnMRhjQkyzaOemsqk/WSKobr5eNfQZcBowW1U/V1VfmvcXA51EpJ2I1ANGA9OLlfkE52wAEYnFqSra4Fvoxpja5D+jezGkSxx5BdZDaXXztWroGeAEYKWIfCAi54lIg7JWUNU8YCwwE1gFvKeqK0TkURE50y02E9glIiuBWcBdqrqrQkdijAl6s9fsYNWWTCbOWR/oUEKKT3cWq+ocYI57SehJOFf7vAY09LLeDGBGsXkPeTxX4Hb3YYwJcdcObs/kuRtI3Xkg0KGEFJ/vLHavGvo7cD3QB3jTX0EZY0LTfcOPBmDa4jQueWURWTk2oll18LWN4D2c6p2TgOdx7iu4yZ+BGWNC2/x1O+n20Ezyrc3A73ztdO5V4EJVzfdnMMYYkzp+BGm7szjhqVkA5OQVEFEvPMBR1W5lnhGIyEnu0yjgLBE51/Ph//CMMaEooWlk0TjHxv+8nRGcCHxPySOV2QhlxhhTC3gboexh9+mj7g1fRUSknd+iMsYYU218vWrowxLmfVCVgRhjjAmMMs8IRKQr0B1oVKxNoCFQ5g1lxhhjgoO3NoIuwEigMYe3E+zDuanMGGNMkPPWRvAp8KmIDFDVH6spJmOMMdXIW9XQ3e6ANBeJyIXFl6vqzX6LzBhjTLXwVjW0yv27xN+BGGOMCQxvVUOfuX+L+hUSkTAgWlUz/RybMcaYauBrX0NTRKShiEQBv+F0R32Xf0MzxhhTHXy9j6CbewZwNvAl0A641F9BGWOMqT6+JoK6IlIXJxFMV9Vc3IHsjTHGBDdfE8EkIBWn87m5ItIWsDYCY4ypBXwdoWwCMMFj1h8iMtQ/IRljjKlOvjYWNxKR50Rkift4FufswBhj/OKHdTsBOPqhr3h1/kYvpU1l+Fo19BpOtxIXuI9M4HV/BWWMMSOOaVn0/Oc/9gQwktrP10TQQVUfVtUN7uMRoL0/AzPGhLbRfduQOn4EHZtHBzqUWs/XRHBQRAYVTojIQOCgf0IyxhhTnXwds/h64C0RaeRO7wEu909Ixhjzlz93ZZG2O4usnDwi6/n6lWXKw+sZgYgkAZ2A0UBPoKeq9lLV5X6OzRhjyMkv4FBeAQ99ugJVu33JH7wNXv8Q8B7wd+ALYJT1MWSMqU4vX5YMwAdL01lijcZ+4e2MYBSQpKoXAn2Aa/0fkjHG/OXUbi1IbtsEgP3ZeQGOpnbylggOqWoWgKru8qG8McZUuQdGdgt0CLWat5aX9iIy3X0uQAePaVT1TL9FZowxroysHACufGMxvz9xOnXD7TdpVfKWCM4qNv2MvwIxxpjSeN5L8PeXFvDcBUl2f0EV8jYwzZzqCsQYY0oT3ySS0X0SmLY4jeXpeznluTl8desJdD2qYaBDqxW8XTX0mYic4XZBXXxZexF5VESu8l94xhjjeOKcY/j0xoFF05/8sjmA0dQu3irargFOAFaLyGIRmSEi34vIBpyuqZeq6mulrSwiw0RkjYisE5F7yyj3dxFREUmu0FEYY2q98DDh2ITGpI4fQf06YagNiVJlvFUNbQXuBu4WkUSgJU7XEmsLryYqjYiEAy8ApwLpwGIRma6qK4uViwFuARZV9CCMMcZUnM9N76qaqqo/qmqKtyTg6gusczupywGmcWTjM8BjwJNAtq+xGGOMqTr+vAarNZDmMZ3uzisiIscBCar6RVkbEpFrC8dC2LFjR9VHaowxISxgF+OKSBjwHHCHt7KqOllVk1U1OS4uzv/BGWNMCPFnItgEJHhMx7vzCsUAPYDZIpIK9AemW4OxMcZUL5/6dHXHHxgHtHXXEUBVtazBaRYDnUSkHU4CGA1cVLhQVfcCsR77mA3cqapLyncIxhhjKsPXM4JXcapxBuF0Ppfs/i2VquYBY4GZwCrgPVVd4d57YF1TGGMq7FBeAZPmbGBZWkagQ6kVfB3lYa+qflnejavqDGBGsXkPlVJ2SHm3b4wJbam7DnBsQuNAhxH0fD0jmCUiT4vIABE5rvDh18iMMaYU391xYqBDqFV8PSPo5/71bMhV4KSqDccYY0x18ykRqOpQfwdijDEmMHyqGhKRRiLyXOFNXSLyrMdA9sYYExBb9lqHBFXB1zaC14B9wAXuIxN43V9BGWNMWQ4ccoasHP/larZnWjKoLF/bCDqo6t89ph8RkRQ/xGOMMV71jG9c9DwzO4/mNixBpfh6RnBQRAYVTrg3mB30T0jGGOPdhAt7BTqEWsPXM4IbgDfddgEBdgNX+CsoY4wx1cfXq4ZSgGNFpKE7nenPoIwxxlSfMhOBiFyiqv8TkduLzQdAVZ/zY2zGGGOqgbczgij3b4y/AzHGGBMY3oaqnOT+faR6wjHGGFPdfL2h7CkRaSgidUXkOxHZISKX+Ds4Y4wx/ufr5aOnuQ3EI4FUoCNwl7+CMsYYU318TQSFVUgjgPfdQWWMMcbUAr7eR/C5iKzGuYnsBhGJA+y+bmOMqQV8OiNQ1XuB44FkVc0FDgBn+TMwY4wx1cPbfQQnqer3InKuxzzPIh/5KzBjjDHVw1vV0InA98AZJSxTLBEYY0zQ83YfwcPu3yurJxxjjDHVzdf7CP4lIo09ppuIyON+i8oYY7zYkuF0gPzMzDUBjiT4+Xr56OmqmlE4oap7gOF+icgYY3yQ0DQSgK9WbA1wJMHP10QQLiL1CydEJAKoX0Z5Y4zxq+HHtOT6EztQr46vX2OmNL7eR/AO8J2IFA5PeSXwpn9CMsYYU518HY/gSRFZBpziznpMVWf6LyxjjDHVxdczAoBVQJ6qfisikSISo6r7/BWYMcaY6uHrVUPXAB8Ak9xZrYFP/BSTMcaYauRrK8uNwEAgE0BVfwea+ysoY4wx1cfXRHBIVXMKJ0SkDs6dxcYYY4Kcr4lgjojcB0SIyKnA+8Bn/gvLGGNMdfE1EdwD7AB+Ba4DZgAP+CsoY4wx1cfrVUMiEg6sUNWuwMv+D8kYY0x18npGoKr5wBoRaVPejYvIMBFZIyLrROTeEpbfLiIrRWS5OxZy2/LuwxhjTOX4eh9BE2CFiPyEMygNAKp6ZmkruGcSLwCnAunAYhGZrqorPYr9gjPYTZaI3AA8BYwq5zEYY4ypBF8TwYMV2HZfYJ2qbgAQkWk4o5oVJQJVneVRfiFwSQX2Y4wxphK8jVDWALge6IjTUPyqqub5uO3WQJrHdDrQr4zyY4AvS4njWuBagDZtyl1DZYwxpgze2gjeBJJxksDpwLP+CEJELnH383RJy1V1sqomq2pyXFycP0IwxpiQ5a1qqJuqHgMgIq8CP5Vj25uABI/peHfeYUTkFOB+4ERVPVSO7RtjjKkC3s4IcguflKNKqNBioJOItBOResBoYLpnARHphdN/0Zmqur2c2zfGGFMFvCWCY0Uk033sA3oWPheRzLJWdBPHWGAmTs+l76nqChF5VEQKrzZ6GogG3heRFBGZXsrmjDGmRDl5Bcz/fWegwwhq3gavD6/MxlV1Bs5dyJ7zHvJ4fsoRKxljjI+6HBUNwHPfrGFQp9gARxO8bIw3Y0zQOqdXPIM7x1FgXWBWiiUCY4wJcZYIjDFBLyUtg49+TufAofJe02LAEoExJsjFRtUD4Pb3ljHj1y0BjiY4WSIwxgS1J8/rySc3DgQgJ78gwNEEJ0sExpigVjc8jNaNIwC4/+Pf2JaZHeCIgo8lAmNM0IuNrscJ7uWjXyy36qHyskRgjAl6IsLtp3YG4NHPV5JrVUTlYonAGFMr9GrThIv6Ob0TF6jdWFAelgiMMbVGYVuBKR9LBMaYWmfl5jK7QjPFWCIwxtQacdH1Abj3w18DHElwsURgjKk1LuiTQJ/EJtZYXE6WCIwxtUrzhg3YsPMAC9bvRK3R2CeWCIwxtUrLhg0AuOjlRbyxIDWwwQQJSwTGmFrl/hFH8/aYvgA88tlKXpq9PsAR1XyWCIwxtYqIcEKnOB4a2Q2AJ79azd6sXC9rhTZvg9cHhdzcXNLT08nOtj5GTGA1aNCA+Ph46tatG+hQQt5Vg9px4FAez36zltwCazwuS61IBOnp6cTExJCYmIiIBDocE6JUlV27dpGenk67du0CHY4BGkdaQvZFragays7OplmzZpYETECJCM2aNbMzUxN0akUiACwJmBrB/g9NMKo1icAYY0zFWCKoIk888QTdu3enZ8+eJCUlsWjRIgDy8vK477776NSpE0lJSSQlJfHEE08UrRceHk5SUhLdu3fn2GOP5dlnn6XAo2Hrp59+YvDgwXTp0oVevXpx9dVXk5WVxRtvvMHYsWOrLP7hw4eTkZEBwIQJEzj66KO5+OKLmT59OuPHj6/Utrds2cLIkSMPm3frrbfSunXrw471jTfeIC4ujqSkJLp168bLL79cqf0CbNy4kX79+tGxY0dGjRpFTk7OEWXeeeedovcmKSmJsLAwUlJSALj//vtJSEggOjr6sHWef/55XnvttUrHZ0yNoKpB9ejdu7cWt3LlyiPmVacFCxZo//79NTs7W1VVd+zYoZs2bVJV1XvuuUcvv/xyPXjwoKqqZmZm6sMPP1y0blRUVNHzbdu26cknn6wPPfSQqqpu3bpV27RpowsWLCgq8/777+vWrVv19ddf1xtvvNEvx9OlSxdNS0ur0Lq5ublHzLvzzjv1k08+KZrOz8/XNm3aaL9+/fT7778vmu95TNu2bdPY2FjdunVrheIodP755+vUqVNVVfW6667TF198sczyy5cv1/bt2xdN//jjj7p58+bD3idV1QMHDmhSUlKJ2wj0/6P5y/8Wpmrbez7Xtvd8rh/9XLH/6doCWKKlfK/WiquGPD3y2Yoq73mwW6uGPHxG91KXb9myhdjYWOrXdzq8io11RkrKysri5ZdfJjU1lQYNnLsdY2JiGDduXInbad68OZMnT6ZPnz6MGzeOF154gcsvv5wBAwYUlTnvvPOOWO+zzz7j8ccfJycnh2bNmvHOO+/QokUL5syZwy233AI4dddz585l//79jBo1iszMTPLy8njppZc44YQTSExMZMmSJTzwwANs2LCB008/nauuuoomTZqwZMkSnn/+eXbs2MH111/Pn3/+CcC///1vBg4cyLhx41i/fj0bNmygTZs2TJ069bD4PvzwQx5//PGi6dmzZ9O9e3dGjRrF1KlTGTp0aImvRYcOHfjjjz9o0aJFqa99WVSV77//nilTpgBw+eWXM27cOG644YZS15k6dSqjR48umu7fv3+J5SIjI0lMTOSnn36ib9++FYrP+N+w7kfx9YptzFm7g+kpmzmnV3ygQ6qRrGqoCpx22mmkpaXRuXNn/vGPfzBnzhwA1q1bR5s2bYiJifF5W+3btyc/P5/t27fz22+/0bt3b6/rDBo0iIULF/LLL78wevRonnrqKQCeeeYZXnjhBVJSUpg3bx4RERFMmTKFv/3tb6SkpLBs2TKSkpIO29bEiRNp1aoVs2bN4rbbbjts2S233MJtt93G4sWL+fDDD7n66quLlq1cuZJvv/32iCSwceNGmjRpUpQkwfmyvfDCCznnnHP44osvyM098mafDRs2sGHDBjp27HjY/DVr1hxWjeP5KKzaKrRr1y4aN25MnTrO7534+Hg2bdpU5mv57rvvcuGFF5ZZplBycjLz5s3zqawJjGbR9Xnzqr70jG8U6FBqtFp3RlDWL3d/iY6OZunSpcybN49Zs2YxatQoxo8fz3HHHXdYuddff53//Oc/7Nq1iwULFpCQkFAl+09PT2fUqFFs2bKFnJycomvYBw4cyO23387FF1/MueeeS3x8PH369OGqq64iNzeXs88++4hEUJZvv/2WlStXFk1nZmayf/9+AM4880wiIo4cFGTLli3ExcUVTefk5DBjxgyee+45YmJi6NevHzNnzixqQ3j33XeZP38+9evXZ9KkSTRt2vSw7XXp0qWo/r6qLVq0iMjISHr06OFT+ebNm7N69Wq/xGJMdbIzgioSHh7OkCFDeOSRR3j++ef58MMP6dixI3/++Sf79u0D4MorryQlJYVGjRqRn59f4nY2bNhAeHg4zZs3p3v37ixdutTrvm+66SbGjh3Lr7/+yqRJk4quY7/33nt55ZVXOHjwIAMHDmT16tUMHjyYuXPn0rp1a6644greeustn4+xoKCAhQsXkpKSQkpKCps2bSpqRI2KiipxnYiIiMOuq585cyYZGRkcc8wxJCYmMn/+/MPOIkaNGkVKSgqLFi3inHPOOWJ75TkjaNasGRkZGeTl5QFOwmzdunWpxzdt2jSfzwbAuX+lpORnah5VmLVmB89//zvbM7M5lFfy5y9U1bozgkBYs2YNYWFhdOrUCYCUlBTatm1LZGQkY8aMYezYsUyaNIkGDRqQn59f4pUrQFEd/NixYxERxo4dS9++fRkxYgT9+vUD4KOPPmLgwIGHrbd3796iL7g333yzaP769es55phjOOaYY1i8eDGrV68mIiKC+Ph4rrnmGg4dOsTPP//MZZdd5tNxnnbaafz3v//lrrvuKjpOb2cUnTt3JjU1tWh66tSpvPLKK0VfuAcOHKBdu3ZkZWX5FEN5zghEhKFDh/LBBx8wevRo3nzzTc4666wSyxYUFPDee++Vq6pn7dq1R7wXpmYa3TeBXz/eyzNfr+WZr9fSPi6K7+8Y4vf9qioFCvkFSoEWPtxpd16+Ktk5BYg4Yy0Xls0vwOO5Uy6hSSRxMfW977icLBFUgf3793PTTTeRkZFBnTp16NixI5MnTwacy0offPBBevToQUxMDBEREVx++eW0atUKgIMHD5KUlERubi516tTh0ksv5fbbbwegRYsWTJs2jTvvvJPt27cTFhbG4MGDGTZs2GH7HzduHOeffz5NmjThpJNOYuPGjYDTmDtr1izCwsLo3r07p59+OtOmTePpp5+mbt26REdHl+uMYMKECdx444307NmTvLw8Bg8ezMSJE8tcJyoqig4dOrBu3TpatWrFV199ddg6UVFRDBo0iM8++8znOMrjySefZPTo0TzwwAP06tWLMWPGADB9+nSWLFnCo48+CsDcuXNJSEigffv2h61/9913M2XKFLKysoiPj+fqq68uauz/4YcfSm34NzXLxf3a0q9dM2av2c6ctTuY9/tOej/2Dc79f4IICFB4P6C48zIP5nIgxzl7aNGwPgX615d7gSpa7G/hFz0KOX4YHOfxs3twSf+2Vb5d0SAbuCE5OVmXLFly2LxVq1Zx9NFHBygi483HH3/M0qVLD7tyKNj98ssvPPfcc7z99ttHLLP/x5pt7bZ9/G/hH+QXKIpTbQRK4VehKqg7nZNfwKcpmzmpa3Oax9RHRAgTCBMnUXj+LZyPx/TOfTm0bhLhLAsTwkQId9cJDxPCwwQRYc6aHfSMb0TrxhGEhwlhYU658DBnW4XzOreIoXXjilVHishSVU0uaZlfzwhEZBjwHyAceEVVxxdbXh94C+gN7AJGqWqqP2My1e+cc85h165dgQ6jSu3cuZPHHnss0GGYCujcIoZHz/LtggCA/4zu5cdoHJf64Vd+efgtEYhIOPACcCqQDiwWkemqutKj2Bhgj6p2FJHRwJPAKH/FZALH81LT2uDUU08NdAjGVBl/XjXUF1inqhtUNQeYBhRvqTsLKGzd/AA4WSrYa1ewVXGZ2sn+D00w8mciaA2keUynu/NKLKOqecBeoFnxDYnItSKyRESW7Nix44gdNWjQgF27dtmH0ASUuuMRFN5FbkywCIqrhlR1MjAZnMbi4svj4+NJT0+npCRhTHUqHKHMmGDiz0SwCfC8dTbenVdSmXQRqQM0wmk0Lpe6devaiFDGGFNB/qwaWgx0EpF2IlIPGA1ML1ZmOnC5+/w84Hu1+h1jjKlWfjsjUNU8ERkLzMS5fPQ1VV0hIo/idIc6HXgVeFtE1gG7cZKFMcaYauTXNgJVnQHMKDbvIY/n2cD5/ozBGGNM2YLuzmIR2QH8UcHVY4GdVRhOMLBjDg12zKGhMsfcVlXjSloQdImgMkRkSWm3WNdWdsyhwY45NPjrmK0bamOMCXGWCIwxJsSFWiKYHOgAAsCOOTTYMYcGvxxzSLURGGOMOVKonREYY4wpxhKBMcaEuFqZCERkmIisEZF1InJvCcvri8i77vJFIpIYgDCrlA/HfLuIrBSR5SLynYgEdiSMKuDtmD3K/V1EVESC/lJDX45ZRC5w3+sVIjKlumOsaj78b7cRkVki8ov7/z08EHFWFRF5TUS2i8hvpSwXEZngvh7LReS4Su9UVWvVA6c7i/VAe6AesAzoVqzMP4CJ7vPRwLuBjrsajnkoEOk+vyEUjtktFwPMBRYCyYGOuxre507AL0ATd7p5oOOuhmOeDNzgPu8GpAY67koe82DgOOC3UpYPB77EGWa5P7CosvusjWcE1TogTg3h9ZhVdZaqZrmTC3F6gw1mvrzPAI/hjHyXXZ3B+Ykvx3wN8IKq7gFQ1e3VHGNV8+WYFWjoPm8EbK7G+Kqcqs7F6XutNGcBb6ljIdBYRFpWZp+1MRFU2YA4QcSXY/Y0BucXRTDzeszuKXOCqn5RnYH5kS/vc2egs4j8ICIL3XHDg5kvxzwOuERE0nH6NrupekILmPJ+3r0KioFpTNURkUuAZODEQMfiTyISBjwHXBHgUKpbHZzqoSE4Z31zReQYVc0IZFB+diHwhqo+KyIDcHo07qGqBYEOLFjUxjOC8gyIQ2UGxKlBfDlmROQU4H7gTFU9VE2x+Yu3Y44BegCzRSQVpy51epA3GPvyPqcD01U1V1U3AmtxEkOw8uWYxwDvAajqj0ADnM7ZaiufPu/lURsTQSgOiOP1mEWkFzAJJwkEe70xeDlmVd2rqrGqmqiqiTjtImeq6pLAhFslfPnf/gTnbAARicWpKtpQjTFWNV+O+U/gZAARORonEdTmcWunA5e5Vw/1B/aq6pbKbLDWVQ1pCA6I4+MxPw1EA++77eJ/quqZAQu6knw85lrFx2OeCZwmIiuBfOAuVQ3as10fj/kO4GURuQ2n4fiKYP5hJyJTcZJ5rNvu8TBQF0BVJ+K0gwwH1gFZwJWV3mcQv17GGGOqQG2sGjLGGFMOlgiMMSbEWSIwxpgQZ4nAGGNCnCUCY4wJcZYIjN+JSL6IpIjIbyLymYg0ruLtp7rXzCMi+0spEyEic0QkXEQSReSgG9NKEZno3olcnn0mi8gE9/kQETneY9n1InJZZY7J3c44EbnTS5k3ROS8cmwzsbReLYuVe0JE0oq/niIyVkSu8nV/JjhYIjDV4aCqJqlqD5z7Nm4MQAxXAR+par47vV5Vk4CeOD1Wnl2ejanqElW92Z0cAhzvsWyiqr5V2YAD7DOcDt+Ke43a35dPyLFEYKrbj7gdZIlIBxH5SkSWisg8Eenqzm8hIh+LyDL3cbw7/xO37AoRubac+70Y+LT4TLfTwQVAR/fX8vfy15gNbdz9nu+ezSwTkbnuvCEi8rk4Y1lcD9zmnmGcUPhLXkS6ishPhftyt/+r+7y3e4ayVERmipfeI0XkGhFZ7MbwoYhEeiw+RUSWiMhaERnplg8XkafddZaLyHXlebFUdWFJd6u6PdimikhJScIEKUsEptqISDhOVwCFd/1OBm5S1d7AncCL7vwJwBxVPRanX/YV7vyr3LLJwM0i4lOPsW7XBO1VNbWEZZFuTL8C/wXeVNWewDtuHAAPAX9z4znsbmx3mxOB/3PPeuZ5LFsN1BORdu6sUcC7IlLX3dd57vG8Bjzh5TA+UtU+bgyrcPrXKZSI8+t9BDBRRBq4y/eqah+gD3CNRxyFx95KRGZ42W9JlgAnVGA9U0PVui4mTI0UISIpOGcCq4BvRCQapzqlsMsLgPru35OAywDcqpy97vybReQc93kCTmdqvnSfEAtkFJvXwY1JgU9V9UsReRs4113+NvCU+/wH4A0ReQ/4yIf9eXoPJwGMd/+OArrgdIj3jXvs4YC3vmJ6iMjjQGOcrkJmeu7D7WnzdxHZAHQFTgN6erQfNMJ5vdYWrqSqm3G6Kiiv7e4+TC1hicBUh4OqmuT++p6J00bwBpDh1tN7JSJDgFOAAaqaJSKzcToX82n/JZRd7+u+VfV6EemH84t7qYj09nG/AO/iJLuPnE3p7yJyDLBCVQeUYztvAGer6jIRuQK3Y7nCEIuHjDN61U2q6pkwkKoZlrUBzmtqagmrGjLVxq1fvhmnk7AsYKOInA9F47Ae6xb9Dmc4zcK67kY4v2j3uEmgK0630r7udw8Q7laZlGUBf3VAeDEwz42hg6ouUtWHcHq1TCi23j6cbq9L2vd6nM7fHsRJCgBrgDhx+s5HROqKSHcvscUAW9xqpYuLLTtfRMJEpAPOkI5rcBLuDW55RKSziER52YevOgNerzwywcMSgalWqvoLsBxnMJGLgTEisgynHaBwCMJbgKFuw+pSnKt6vgLqiMgqnGqWheXc9dfAIC9lbgKuFJHlwKVuHABPi8iv7mWXC3DGzfX0GXBOYWNxCdt9F7iEv/rMz8Hp/vxJ99hT8LjqqBQPAotwqqlWF1v2J/ATzqhz16tqNvAKsBL42Y17EsVqAMpqIxCRp8Tp+TJSRNJFZJzH4oHAN17iNUHEeh81IUGcYStvU9VLAx1LMBNnXIvb7XWsXeyMwIQEVf0ZmOVeuWQqLhbn7MTUInZGYIwxIc7OCIwxJsRZIjDGmBBnicAYY0KcJQJjjAlxlgiMMSbE/X8aQEXfQlnMowAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "disp = plot_precision_recall_curve(clf, x_test, y_test)\n",
    "disp.ax_.set_title('SGDClassifier Precision-Recall curve ')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'Complement NaiveBayes Precision-Recall curve')"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAtI0lEQVR4nO3deZgU1dn+8e8NooCyGMHEsAjivgAqrtGIxgUVxQUFNa/BuESTaNSo8RejotE30RizGN+gJmo0iLiDu0bBfQFkUcBdFBAjMSAoi4DP74+qHptmZrpgpnuc6ftzXXNNLaeqnuqu7qfPObUoIjAzs8rVrKEDMDOzhuVEYGZW4ZwIzMwqnBOBmVmFcyIwM6twTgRmZhXOieBrTtIMSfs2dBzlJuk4SY81dBxNhaRfSvpbhnIPS/pBOWIqB0khadN0+GZJlzV0TF9HTgQ1kHSspPGSPpM0J/2A7NHQcX0dSBoi6dkiZcZKWiKpS960fSXNyLKNiBgeEfvXMVQkDZW0LH0fP5M0XdKRdV1vfUq/rD5P45st6WpJzetzGxHxvxFxUoZyB0bEP+pz2wCSuqX7mXsfZkg6v763Y2vGiaAaks4G/gj8L/BNoCvwf8CABgyrMfocuLChgwBGRsR6EbEecCbwT0nfbOCYCvVK4/secCxwcmEBSWuVPar61z7dz4HAhZL2a+iA6lNjfY+cCApIagdcCvwkIu6JiM8jYllE3B8R56Zl1pH0R0kfpn9/lLROOq+vpFmSzpP0cVqbOEzSQZLelPRfSb/M295QSXdJGilpoaRXJPWqIbZmks6X9I6kTyTdIekb6bzcL64TJM2UNE/SqZJ2kjRF0nxJfylY3w/TX8jzJD0qaeO8eZEu/1a67LVKbAUMA3ZLf9nNr+Xl/DNwjKQeNexPbl8WSpom6fC8eVW1Dkl/lXRVwbKj0oSNpG9LulvSXEnvSTqjpoAi4lFgIdAjXXZ9SQ+ky85Lhzun846SNKFgu2dLGpUOryPpKkkfSPq3pGGSWqXzOqTrmp++589IKvp5i4jXgWeAbfPe0xMlfQA8ma67tvdtG0mPp9v8d+5YS4+zf6bDLSX9Mz2G5ksal0uMSmpyJ6XDzST9StL76bF8S/r5yD/efpDu/38kXVBs//L2czwwFeidF/ua7NfOkl5I92OOpL9IWjtrHPkknZxuP3c87pBOr2peSsermpj01ef9F5I+Am5K19E/r/xa6fGVW9+ukp5PY54sqe+axFuvIsJ/eX9AP2A5sFYtZS4FXgQ2BDoCzwO/Tuf1TZe/CGhB8stuLnAb0AbYBlgMdE/LDwWWkfxCagGcA7wHtEjnzwD2TYd/lm63M7AOcB0wIp3XDQiSL+mWwP7AEuC+NM5OwMfAXmn5AcDbwFbAWsCvgOfz9jGAB4D2JDWiuUC/dN4Q4Nkir+NY4CTgauCf6bR9gRl5ZY4Cvk3yg2QQSQ1io8JtAN8FZgJKx9dPX8PcshPS13ttYBPgXeCAvNc3t30BBwPzSX6ZAmwAHAm0Tt+fO4H70nnrAP8FtsqLeSJwZDr8B2A08I102fuB36TzfpO+Fy3Svz1z8VfzWgWwaTq8NfARcGLee3oLsC7Qqrb3LY1hDvDz9BhoA+xSzevwozTW1kBzYEegbf77lg7/MN3WJsB6wD3ArQXH2w1pXL2ApfmvVcE+5sqvlY7vCiwCDi92PBbZrx3Tda2VbmM6cGYNr+3NwGU1xHcUMBvYieQ42RTYuHAdhevhq8/7FSTHSyuSY3F4XvmDgenpcCfgE+AgkmN3v3S8Y4N+7zXkxr+Of8BxwEdFyrwDHJQ3fgDpF1x6YCwGmucdxJE7cNNpE4DD0uGhwIt585qlB/2e6fgMvkoE04Hv5ZXdiCSJ5D4EAXTKm/8JMChv/O7chwR4GDixYLuLCg7+PfLm3wGcnw4PIXsi6Ah8SpIAV0oE1SwzCRhQuI30g/kB8N10/GTgyXR4F+CDgvX8P+CmvNf3C5Iv/8+BFcB5tcTQG5iXN/5X4PJ0eBtgHskHXun6euSV3Q14Lx2+FBhF3hdILdsMYEG67neAy9L3I/eebpJXtsb3DTgGmFjDNobyVSL4IcmPl541vW/p8BPAj/PmbVHN8dY5b/7LwOAatp8rP5/k8xHAVXyV3Ndov6rZzpnAvQWvbZZE8Cjws1ren9oSwRdAy7z5m5LUOlun48OBi9LhX5Am04Jt/yDL/pXqz01Dq/oE6KDa2/q+DbyfN/5+Oq1qHRGxIh1enP7/d978xSS/sHJm5gYi4ktgVsH6cjYG7k2rlPNJEsMKkn6MnMLt1LTdjYE/5a3rvyRfbp3yyn+UN7yoIOZMImIu8BeSL8aVSDpe0qS8GLYFOlSzjgBuJ/lCgKQNfXjefnw7t450Pb9k5dfkjohoHxHrkjQJHS/pR2kMrSVdlzZ/LACeBtrrq87afwDHShLwP+m6lpIkuNbAhLztPpJOB/gdyS/cxyS9q+IdoztExPoR0SMifpUeBzkz84Zre9+6kCSSYm4l+fK5XUnT5pWSWlRTrrrjfC1Wfm2rPUb0VafwZ5K65pXpkJb5OcmXaG67a7RfkjZPm+A+St+//6WaYyiDrK9ddeZGxJLcSES8TfLZPERSa+BQkhYBSPbzqILjdQ+SH3UNxolgVS+QVHEPq6XMhyRvaE7XdNqayj+zphlJ009165sJHJh+qeX+WkbE7DXY5kzgRwXrahURz2dYNlZzW78D9iapxgOQtv/eAPwU2CAi2gOvkXz4qzMCGJgutwtJ7Sa3H+8V7EebiDio2sAjZpD8+jwknfRzkl+6u0REW5JmKHJxRMSLJL/49iRJQLem8/9Dkli3ydtuu0g6QomIhRHx84jYhOSL4GxJ38vwWlUbdt5wbe/bTJJmnNpXlvR5XRIRWwO7A/2B46spWt1xvpyVf1zUtI318v4+KJi3IiKuJmm6/HEd9+uvwOvAZun790tqPoZqM5O036gai0iSfs63CuZX93kYQfLDZQAwLU0Oue3cWrCf60bEb9cg5nrjRFAgIj4laeO7Vkknb2tJLSQdKOnKtNgI4FeSOkrqkJb/Zx02u6OkI9JayJkkiejFasoNAy7PdaKl2x+whtscBvw/Sduk62on6aiMy/4b6Jy1Uy4i5gO/B87Lm7wuyQdobrr9E0hqBDWtYyLJl+/fgEfTdULSHLEw7axrJam5pG0l7VTdepR0BPcj6aiEpOluMTBfScf7xdUsdgtJrWZZRDybxvMlSSL7g6QN03V3knRAOtxf0qZpTeJTkprbl9Wse3XV9r49AGwk6UwlHdltJO1SzWuwt6Tt0lrPApLmnupiGwGcJam7pPVIfm2PjIjl9bAfAL8FzpPUsg771Sbdh88kbQmctoax/A04R9KOSmyqrzqrJ5HUCptL6gfslWF9t5P0053GV7UBSL4nDpF0QLq+lko6nDuvYdz1womgGhHxe+Bskg6ruSRZ/KckHa+QtOGOB6YArwKvpNPW1CiSztJ5JM0PR0TEsmrK/Ymkc/IxSQtJksUqH/QsIuJekg6u29Mq9WvAgRkXf5Lki/QjSf/JuMyfSL4Mc9ufRpIcXiBJLNsBzxVZx20k/QxVH6y0Ca4/Sdv+e3yVLNrlLTco10wBjEu3c0k6748kHXz/IXk9H6lmu7eSJKnCZP8LkuafF9PX8F8ktQuAzdLxz9J9/L+IGFNk/4qq7X2LiIUknY+HkDTZvEVSEyv0LeAuki/Q6cBTfFXTyXdjOv1pktd2CXB6Xfchz4Mkx/zJddivc0hqagtJEvPINQkkIu4ELic5thaSfNa/kc7+Wbrt+SR9iPdlWN8ckvd99/yYImImSS3hl3z13XIuDfxdnOuosQYiaShJR9T3GzoWq56SU0I/JmnHf6uh4zGrb64RmBV3GjDOScCaqkZ5FZxZuSi5JYao/eQBs0bNTUNmZhXOTUNmZhWu0TUNdejQIbp169bQYZiZNSoTJkz4T0R0rG5eo0sE3bp1Y/z48Q0dhplZoyLp/ZrmuWnIzKzCORGYmVU4JwIzswrnRGBmVuGcCMzMKlzJEoGkG5U83u61GuZL0p8lva3kUYo7lCoWMzOrWSlrBDeT3O63JgeS3KFxM+AUkvuKm5lZmZUsEUTE0yRPGarJAOCWSLxI8lSokj2l55L7p3LJ/VOLFzQzqzANeUFZJ1Z+BN+sdNqcwoKSTiGpNdC1a9fC2ZlM+3DBGi1nZtbUNYrO4oi4PiL6RESfjh2rvULazMzWUEMmgtnkPauX5Dm9a/LsXTMzq4OGTASjgePTs4d2BT5NH+9mZmZlVLI+AkkjgL5AB0mzSB4K3gIgIoYBDwEHkTzzdRFwQqliMTOzmpUsEUTEMUXmB/CTUm3fzMyyaRSdxWZmVjpOBGZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCqcE4GZWYVzIjAzq3BOBGZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCqcE4GZWYVzIjAzq3BOBGZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCqcE4GZWYVzIjAzq3BOBGZmFc6JwMyswjkRmJlVuLWyFJK0IfAd4NvAYuA1YHxEfFnC2MzMrAxqTQSS9gbOB74BTAQ+BloChwE9JN0F/D4iFpQ4TjMzK5FiNYKDgJMj4oPCGZLWAvoD+wF3lyA2MzMrg1oTQUScW8u85cB99R2QmZmV1xp3Fks6IUOZfpLekPS2pPOrmd9V0hhJEyVNkXTQmsZjZmZrpi5nDV1S20xJzYFrgQOBrYFjJG1dUOxXwB0RsT0wGPi/OsRjZmZroFhn8ZSaZgHfLLLunYG3I+LddF23AwOAaXllAmibDrcDPiwWsJmZ1a9incXfBA4A5hVMF/B8kWU7ATPzxmcBuxSUGQo8Jul0YF1g3+pWJOkU4BSArl27FtmsmZmtjmJNQw8A60XE+wV/M4Cx9bD9Y4CbI6IzyRlKt0paJaaIuD4i+kREn44dO9bDZs3MLKfYWUMn1jLv2CLrng10yRvvnE7LdyLQL13fC5JaAh1IrlcwM7MyKOUtJsYBm0nqLmltks7g0QVlPgC+ByBpK5KL1eaWMCYzMytQskSQXmfwU+BRYDrJ2UFTJV0q6dC02M+BkyVNBkYAQyIiShWTmZmtKtO9htZURDwEPFQw7aK84Wkk9zAyM7MG4ruPmplVuMyJQNL1tY2bmVnjtDo1guuKjJuZWSOUORFExITaxs3MrHEqdouJ+0luA1GtiDi0pnlmZtY4FDtr6KqyRGFmZg2m2JXFT+WGJbUCukbEGyWPyszMyiZTH4GkQ4BJwCPpeG9JhVcJm5lZI5S1s3goyW2l5wNExCSge0kiMjOzssqaCJZFxKcF03wrCDOzJiDrLSamSjoWaC5pM+AMij+PwMzMGoGsNYLTgW2ApSQ3h1sAnFmimMzMrIwy1QgiYhFwgaQrktFYWNqwzMysXLKeNbSTpFeBKcCrkiZL2rG0oZmZWTlk7SP4O/DjiHgGQNIewE1Az1IFZmZm5ZG1j2BFLgkARMSzwPLShGRmZuVU7F5DO6SDT0m6jqSjOIBB1M/D683MrIEVaxr6fcH4xXnDvo7AzKwJKHavob3LFYiZmTWMzM8slnQwybUELXPTIuLSUgRlZmblk/X00WEk/QKnAwKOAjYuYVxmZlYmWc8a2j0ijgfmRcQlwG7A5qULy8zMyiVrIlic/l8k6dvAMmCj0oRkZmbllLWP4AFJ7YHfAa+QnDH0t1IFZWZm5ZP1XkO/TgfvlvQA0LKa21KbmVkjVOyCsiNqmUdE3FP/IZmZWTkVqxEcUsu8AJwIzMwauWIXlJ1QrkDMzKxhZD1ryMzMmignAjOzCudEYGZW4bLeYqK1pAsl3ZCObyapf4bl+kl6Q9Lbks6voczRkqZJmirpttUL38zM6irrBWU3ARNIbi0BMBu4E3igpgUkNQeuBfYDZgHjJI2OiGl5ZTYD/h/wnYiYJ2nD1d8FMzOri6xNQz0i4kqSW0vkHmavIsvsDLwdEe9GxBfA7cCAgjInA9dGxLx0vR9njtzMzOpF1kTwhaRWpA+jkdQDWFpkmU7AzLzxWem0fJsDm0t6TtKLkvpVtyJJp0gaL2n83LlzM4ZsZmZZZE0EQ4FHgC6ShgNPAOfVw/bXAjYD+gLHADek9zRaSURcHxF9IqJPx44d62GzZmaWk/VeQ49JmgDsStIk9LOI+E+RxWYDXfLGO6fT8s0CXoqIZcB7kt4kSQzjssRlZmZ1l/WsofuB/YGxEfFAhiQAyZf5ZpK6S1obGAyMLihzH0ltAEkdSJqK3s0WupmZ1YesTUNXAXsC0yTdJWmgpJa1LRARy4GfAo8C04E7ImKqpEslHZoWexT4RNI0YAxwbkR8skZ7YmZmayRr09BTwFPpKaH7kJztcyPQtshyDwEPFUy7KG84gLPTPzMzawCr8/D6ViR3Ix0E7AD8o1RBmZlZ+WRKBJLuILku4BHgL8BTEfFlKQMzM7PyyFoj+DtwTESsKGUwZmZWfsWeULZPRDwJrAsMkFa+mNhPKDMza/yK1Qj2Ap6k+ieV+QllZmZNQLEnlF2cDl4aEe/lz5PUvWRRmZlZ2WS9juDuaqbdVZ+BmJlZwyjWR7AlsA3QTtIRebPaArVeUGZmZo1DsT6CLYD+QHtW7idYSHJRmZmZNXLF+ghGAaMk7RYRL5QpJjMzK6NiTUPnpQ+kOVbSMYXzI+KMkkVmZmZlUaxpaHr6f3ypAzEzs4ZRrGno/vR/1X2FJDUD1ouIBSWOzczMyiDr8whuk9RW0rrAayS3oz63tKGZmVk5ZL2OYOu0BnAY8DDQHfifUgVlZmblkzURtJDUgiQRjE4fLRkli8rMzMomayK4DphBcvO5pyVtDLiPwMysCcj6hLI/A3/Om/S+pL1LE5KZmZVT1s7idpKuljQ+/fs9Se3AzMwauaxNQzeS3Fbi6PRvAXBTqYIyM7PyyfqEsh4RcWTe+CWSJpUgHjMzK7OsNYLFkvbIjUj6DrC4NCGZmVk5Za0RnArcIqldOj4P+EFpQjIzs3Iqmggk9QY2BQYDswF8ewkzs6aj1qYhSRcBdwBHAg8Cg5wEzMyalmI1gkFA74hYJGkD4BHghtKHZWZm5VKss3hpRCwCiIhPMpQ3M7NGpliNYBNJo9NhAT3yxomIQ0sWmZmZlUWxRDCgYPyqUgViZmYNo9iDaZ4qVyBmZtYwip01dL+kQ9JbUBfO20TSpZJ+WLrwzMys1Io1DZ0MnA38UdJ/gblAS6Ab8A7wl4gYVdIIzcyspIo1DX0EnAecJ6kbsBHJrSXezJ1NVBtJ/YA/Ac2Bv0XEb2sodyRwF7BTRIxfrT0wM7M6yXqLCSJiBsnDaTKR1By4FtgPmAWMkzQ6IqYVlGsD/Ax4Keu6zcys/pTyuoCdgbcj4t2I+AK4nVXPQgL4NXAFsKSEsZiZWQ1KmQg6ATPzxmel06pI2gHoEhEP1rYiSafkHoozd+7c+o/UzKyCNdiVwpKaAVcDPy9WNiKuj4g+EdGnY8eOpQ/OzKyCZOojSJ8/MBTYOF1GQETEJrUsNhvokjfeOZ2W0wbYFhgrCeBbwGhJh7rD2MysfLJ2Fv8dOAuYAKzIuMw4YDNJ3UkSwGDg2NzMiPgU6JAblzQWOMdJwMysvLImgk8j4uHVWXFELJf0U+BRktNHb4yIqZIuBcZHxOja12BmZuWQNRGMkfQ74B5gaW5iRLxS20IR8RDwUMG0i2oo2zdjLGZmVo+yJoJd0v998qYFsE/9hmNmZuWWKRFExN6lDsTMzBpGptNHJbWTdHXuXH5Jv897kL2ZmTViWa8juBFYCByd/i0AbipVUGZmVj5Z+wh6RMSReeOXSJpUgnjMzKzMstYIFkvaIzeSXmC2uDQhmZlZOWWtEZwG/CPtFxDwX2BIqYIyM7PyyXrW0CSgl6S26fiCUgZlZmblU2sikPT9iPinpLMLpgMQEVeXMDYzMyuDYjWCddP/bUodiJmZNYxij6q8Lv1/SXnCMTOzcst6QdmVktpKaiHpCUlzJX2/1MGZmVnpZT19dP+0g7g/yXOLNwXOLVVQZmZWPlkTQa4J6WDgzvRZAmZm1gRkvY7gAUmvk1xEdpqkjvhh82ZmTUKmGkFEnA/sDvSJiGXA58CAUgZmZmblUew6gn0i4klJR+RNyy9yT6kCMzOz8ijWNLQX8CRwSDXzAicCM7NGr9h1BBen/08oTzhmZlZuWa8j+F9J7fPG15d0WcmiMjOzssl6+uiBETE/NxIR84CDShKRmZmVVdZE0FzSOrkRSa2AdWopb2ZmjUTW6wiGA09Iyj2e8gTgH6UJyczMyinr8wiukDQZ2Ded9OuIeLR0YZmZWblkrREATAeWR8S/JLWW1CYiFpYqMDMzK4+sZw2dDNwFXJdO6gTcV6KYzMysjLJ2Fv8E+A6wACAi3gI2LFVQZmZWPlmbhpZGxBe520tIWovkyuJGZdqcBQy67oWGDsOsSRnQuxPH7tK1ocOwOsiaCJ6S9EuglaT9gB8D95curPo3oHenhg7BrMmZNmcBgBNBI5c1EfwCOAl4FfgR8BDwt1IFVQrH7tLVB6tZPXMNu2komggkNQemRsSWwA2lD8nMzMqpaGdxRKwA3pC02j+nJfWT9IaktyWdX838syVNkzQlfRbyxqu7DTMzq5usTUPrA1MlvUzyUBoAIuLQmhZIaxLXAvsBs4BxkkZHxLS8YhNJHnazSNJpwJXAoNXcBzMzq4OsieDCNVj3zsDbEfEugKTbSZ5qVpUIImJMXvkXge+vwXbMzKwOij2hrCVwKrApSUfx3yNiecZ1dwJm5o3PAnappfyJwMM1xHEKcApA167u8DUzq0/F+gj+AfQhSQIHAr8vRRCSvp9u53fVzY+I6yOiT0T06dixYylCMDOrWMWahraOiO0AJP0deHk11j0b6JI33jmdthJJ+wIXAHtFxNLVWL+ZmdWDYjWCZbmB1WgSyhkHbCapu6S1gcHA6PwCkrYnuX/RoRHx8Wqu38zM6kGxGkEvSQvSYZFcWbwgHY6IaFvTghGxXNJPgUeB5sCNETFV0qXA+IgYTdIUtB5wZ3r7ig9qOxPJzMzqX7GH1zevy8oj4iGSq5Dzp12UN7zvKguZmVlZZb37qJmZNVFOBGZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCqcE4GZWYVzIjAzq3BOBGZmFc6JwMyswjkRmJlVOCcCM7MK50RgZlbhnAjMzCqcE4GZWYVzIjAzq3DFHlXZKCxbtoxZs2axZMmShg7FLLOWLVvSuXNnWrRo0dChWIVrEolg1qxZtGnThm7dupE++9jsay0i+OSTT5g1axbdu3dv6HCswjWJpqElS5awwQYbOAlYoyGJDTbYwLVY+1poEokAcBKwRsfHrH1dNJlEYGZma8aJoJ589NFHDB48mB49erDjjjty0EEH8eabb5Zse2PHjqV///4lW39tZsyYwW233bZSLJK4//77q6b179+fsWPHAtC3b1+22GILevfuzVZbbcX1119f47oHDhzIu+++WzU+adIkJPHII4+sVK558+b07t2bbbfdlqOOOopFixbVaZ8igjPOOINNN92Unj178sorr1Rb7oILLqBLly6st956K00/66yz6N27N71792bzzTenffv2AMydO5d+/frVKTazUnMiqAcRweGHH07fvn155513mDBhAr/5zW/497//3dChlURhIgDo3Lkzl19+eY3LDB8+nEmTJvHcc8/xi1/8gi+++GKVMlOnTmXFihVssskmVdNGjBjBHnvswYgRI1Yq26pVKyZNmsRrr73G2muvzbBhw+q0Tw8//DBvvfUWb731Ftdffz2nnXZateUOOeQQXn755VWm/+EPf2DSpElMmjSJ008/nSOOOAKAjh07stFGG/Hcc8/VKT6zUmoSZw3lu+T+qUz7cEG9rnPrb7fl4kO2qXH+mDFjaNGiBaeeemrVtF69egFJkjjvvPN4+OGHkcSvfvUrBg0axNixY7n44otp3749r776KkcffTTbbbcdf/rTn1i8eDH33XcfPXr0YMiQIbRs2ZLx48ezYMECrr766lVqAp9//jmnn346r732GsuWLWPo0KEMGDCAm2++mfvuu4/PP/+ct956i3POOYcvvviCW2+9lXXWWYeHHnqIb3zjG7zzzjv85Cc/Ye7cubRu3ZobbriBLbfckiFDhtC2bVvGjx/PRx99xJVXXsnAgQM5//zzmT59Or179+YHP/gB22+/Pb169WLZsmU8/vjj7LfffjW+Vp999hnrrrsuzZs3X2Xe8OHDGTBgQNV4RHDnnXfy+OOPs+eee7JkyRJatmy5ynJ77rknU6ZMqfkNzGDUqFEcf/zxSGLXXXdl/vz5zJkzh4022milcrvuumvRdY0YMYJLLrmkavywww5j+PDhfOc736lTjGal4hpBPXjttdfYcccdq513zz33MGnSJCZPnsy//vUvzj33XObMmQPA5MmTGTZsGNOnT+fWW2/lzTff5OWXX+akk07immuuqVrHjBkzePnll3nwwQc59dRTVznT5PLLL2efffbh5ZdfZsyYMZx77rl8/vnnVbHdc889jBs3jgsuuIDWrVszceJEdtttN2655RYATjnlFK655homTJjAVVddxY9//OOqdc+ZM4dnn32WBx54gPPPPx+A3/72t+y5555MmjSJs846q6rsBRdcwGWXXVbt63DcccfRs2dPtthiCy688MJqE8Fzzz230uv4/PPP0717d3r06EHfvn158MEHV1lm+fLlPPzww2y33XarzBs0aFBVc03+X26/882ePZsuXbpUjXfu3JnZs2dXuy+1ef/993nvvffYZ599qqb16dOHZ555ZrXXZVYuTa5GUNsv94bw7LPPcswxx9C8eXO++c1vstdeezFu3Djatm3LTjvtVPWLs0ePHuy///4AbLfddowZM6ZqHUcffTTNmjVjs802Y5NNNuH1119faRuPPfYYo0eP5qqrrgKS02k/+OADAPbee2/atGlDmzZtaNeuHYccckjVNqZMmcJnn33G888/z1FHHVW1vqVLl1YNH3bYYTRr1oytt966aFPXd7/73ap9LjR8+HD69OnD3Llz2X333enXrx8bb7zxSmXmzJlDx44dq8ZHjBjB4MGDARg8eDC33HILRx55JACLFy+md+/eQFIjOPHEE1fZ5siRI2uNtxRuv/12Bg4cuFKi23DDDfnwww/LHotZVk0uETSEbbbZhrvuumu1l1tnnXWqhps1a1Y13qxZM5YvX141r/A0w8LxiODuu+9miy22WGn6Sy+9VHQbX375Je3bt2fSpElFY4yIovuUqxWstVb1h1bHjh3ZYYcdeOmll1ZJBK1ataqq7axYsYK7776bUaNGcfnll1ddgLVw4ULatGlT1UdQm0GDBvHGG2+sMv3ss8/m+OOPX2lap06dmDlzZtX4rFmz6NSpU9H9LXT77bdz7bXXrjRtyZIltGrVarXXZVYubhqqB/vssw9Lly5d6WyYKVOm8Mwzz7DnnnsycuRIVqxYwdy5c3n66afZeeedV2v9d955J19++SXvvPMO77777ipf+AcccADXXHNN1Rf1xIkTM6+7bdu2dO/enTvvvBNIvuwnT55c6zJt2rRh4cKF1c7bf//9mTdvXo1t9osWLWLixIn06NFjlXlbbbUVb7/9NgBPPPEEPXv2ZObMmcyYMYP333+fI488knvvvTfzvo0cObKqAzf/rzAJABx66KHccsstRAQvvvgi7dq1W6V/oJjXX3+defPmsdtuu600/c0332TbbbddrXWZlZNrBPVAEvfeey9nnnkmV1xxBS1btqRbt2788Y9/ZI899uCFF16gV69eSOLKK6/kW9/61irNO7Xp2rUrO++8MwsWLGDYsGGrdJheeOGFnHnmmfTs2ZMvv/yS7t2788ADD2Re//DhwznttNO47LLLWLZsGYMHD67q7K5Oz549ad68Ob169WLIkCFsv/32K82/4IILVur0haSPoFWrVixdupQhQ4ZU26dy8MEHM3bsWPbdd19GjBjB4YcfvtL8I488kr/+9a/VfpHX1UEHHcRDDz3EpptuSuvWrbnpppuq5vXu3buq9nHeeedx2223sWjRIjp37sxJJ53E0KFDgaQ2MHjw4FVqbGPGjOHggw+u95i/LqbNWcCg615o6DAqQrETV9aUslT3v0769OkT48ePX2na9OnT2WqrrRoootIaMmQI/fv3Z+DAgQ0dSsktXryYvffem+eee67azuTG6rvf/S6jRo1i/fXXX2VeYz92b3vpA0ZNWv1OdVszdUkEkiZERJ/q5rlGYF8brVq14pJLLmH27Nl07dq1ocOpF3PnzuXss8+uNgk0Bcfu0pVjd2ka71UlK2kikNQP+BPQHPhbRPy2YP46wC3AjsAnwKCImFHKmBqbm2++uaFDKKsDDjigoUOoVx07duSwww5r6DDMalWyzmJJzYFrgQOBrYFjJG1dUOxEYF5EbAr8AbhiTbfX2Jq4zHzM2tdFKc8a2hl4OyLejYgvgNuBAQVlBgD/SIfvAr6nNbglY8uWLfnkk0/8wbJGI3c6bHVXSpuVWymbhjoBM/PGZwG71FQmIpZL+hTYAPhPfiFJpwCnANW2HXfu3JlZs2Yxd+7cegverNRyTygza2iNorM4Iq4HrofkrKHC+S1atPBTnszM1lApm4ZmA13yxjun06otI2ktoB1Jp7GZmZVJKRPBOGAzSd0lrQ0MBkYXlBkN/CAdHgg8GW7oNzMrq5I1DaVt/j8FHiU5ffTGiJgq6VJgfESMBv4O3CrpbeC/JMnCzMzKqNFdWSxpLvD+Gi7egYKO6Argfa4M3ufKUJd93jgiOlY3o9ElgrqQNL6mS6ybKu9zZfA+V4ZS7bPvPmpmVuGcCMzMKlylJYLrixdpcrzPlcH7XBlKss8V1UdgZmarqrQagZmZFXAiMDOrcE0yEUjqJ+kNSW9LOr+a+etIGpnOf0lStwYIs15l2OezJU2TNEXSE5I2rm49jUmxfc4rd6SkkNToTzXMss+Sjk7f66mSbit3jPUtw7HdVdIYSRPT4/ughoizvki6UdLHkl6rYb4k/Tl9PaZI2qHOG42IJvVHchXzO8AmwNrAZGDrgjI/Boalw4OBkQ0ddxn2eW+gdTp8WiXsc1quDfA08CLQp6HjLsP7vBkwEVg/Hd+woeMuwz5fD5yWDm8NzGjouOu4z98FdgBeq2H+QcDDgIBdgZfqus2mWCMo23MQvkaK7nNEjImIRenoiyQ3AWzMsrzPAL8meeDRknIGVyJZ9vlk4NqImAcQER+XOcb6lmWfA2ibDrcDPixjfPUuIp4mueVOTQYAt0TiRaC9pI3qss2mmAiqew5Cp5rKRMRyIPcchMYqyz7nO5HkF0VjVnSf0ypzl4h4sJyBlVCW93lzYHNJz0l6MX1cbGOWZZ+HAt+XNAt4CDi9PKE1mNX9vBfVKJ5HYPVH0veBPsBeDR1LKUlqBlwNDGngUMptLZLmob4ktb6nJW0XEfMbMqgSOwa4OSJ+L2k3khtZbhsRXzZ0YI1FU6wRVOJzELLsM5L2BS4ADo2IpWWKrVSK7XMbYFtgrKQZJG2poxt5h3GW93kWMDoilkXEe8CbJImhscqyzycCdwBExAtAS5KbszVVmT7vq6MpJoJKfA5C0X2WtD1wHUkSaOztxlBknyPi04joEBHdIqIbSb/IoRExvmHCrRdZju37SGoDSOpA0lT0bhljrG9Z9vkD4HsAkrYiSQRN+bm1o4Hj07OHdgU+jYg5dVlhk2saigp8DkLGff4dsB5wZ9ov/kFEHNpgQddRxn1uUjLu86PA/pKmASuAcyOi0dZ2M+7zz4EbJJ1F0nE8pDH/sJM0giSZd0j7PS4GWgBExDCSfpCDgLeBRcAJdd5mI369zMysHjTFpiEzM1sNTgRmZhXOicDMrMI5EZiZVTgnAjOzCudEYCUnaYWkSZJek3S/pPb1vP4Z6TnzSPqshjKtJD0lqbmkbpIWpzFNkzQsvRJ5dbbZR9Kf0+G+knbPm3eqpOPrsk/peoZKOqdImZslDVyNdXar6a6WBeUulzSz8PWU9FNJP8y6PWscnAisHBZHRO+I2Jbkuo2fNEAMPwTuiYgV6fg7EdEb6Elyx8rDVmdlETE+Is5IR/sCu+fNGxYRt9Q14AZ2P8kN3wrdSNO/l0/FcSKwcnuB9AZZknpIekTSBEnPSNoynf5NSfdKmpz+7Z5Ovy8tO1XSKau53eOAUYUT05sOPg9smv5aflJfPbOha7rdo9LazGRJT6fT+kp6QMmzLE4FzkprGHvmfslL2lLSy7ltpet/NR3eMa2hTJD0qIrcPVLSyZLGpTHcLal13ux9JY2X9Kak/mn55pJ+ly4zRdKPVufFiogXq7taNb2D7QxJ1SUJa6ScCKxsJDUnuRVA7qrf64HTI2JH4Bzg/9LpfwaeioheJPdln5pO/2Fatg9whqRMd4xNb02wSUTMqGZe6zSmV4FrgH9ERE9geBoHwEXAAWk8K12Nna5zGPCHtNbzTN6814G1JXVPJw0CRkpqkW5rYLo/NwKXF9mNeyJipzSG6ST318npRvLr/WBgmKSW6fxPI2InYCfg5Lw4cvv+bUkPFdludcYDe67BcvY11eRuMWFfS60kTSKpCUwHHpe0HklzSu6WFwDrpP/3AY4HSJtyPk2nnyHp8HS4C8nN1LLcPqEDML9gWo80pgBGRcTDkm4Fjkjn3wpcmQ4/B9ws6Q7gngzby3cHSQL4bfp/ELAFyQ3xHk/3vTlQ7F4x20q6DGhPcquQR/O3kd5p8y1J7wJbAvsDPfP6D9qRvF5v5haKiA9JblWwuj5Ot2FNhBOBlcPiiOid/vp+lKSP4GZgftpOX5SkvsC+wG4RsUjSWJKbi2XafjVl38m67Yg4VdIuJL+4J0jaMeN2AUaSJLt7klXFW5K2A6ZGxG6rsZ6bgcMiYrKkIaQ3lsuFWBgyydOrTo+I/ISB6uexrC1JXlNrItw0ZGWTti+fQXKTsEXAe5KOgqrnsPZKiz5B8jjNXFt3O5JftPPSJLAlyW2ls253HtA8bTKpzfN8dQPC44Bn0hh6RMRLEXERyV0tuxQst5DkttfVbfsdkpu/XUiSFADeADoquXc+klpI2qZIbG2AOWmz0nEF846S1ExSD5JHOr5BknBPS8sjaXNJ6xbZRlabA0XPPLLGw4nAyioiJgJTSB4mchxwoqTJJP0AuUcQ/gzYO+1YnUByVs8jwFqSppM0s7y4mpt+DNijSJnTgRMkTQH+J40D4HeSXk1Pu3ye5Lm5+e4HDs91Flez3pHA9/nqnvlfkNz+/Ip03yeRd9ZRDS4EXiJppnq9YN4HwMskT507NSKWAH8DpgGvpHFfR0ELQG19BJKuVHLny9aSZkkamjf7O8DjReK1RsR3H7WKoOSxlWdFxP80dCyNmZLnWpzt17FpcY3AKkJEvAKMSc9csjXXgaR2Yk2IawRmZhXONQIzswrnRGBmVuGcCMzMKpwTgZlZhXMiMDOrcP8f1KgyGbLw6CMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "disp = plot_precision_recall_curve(clf2, x_test, y_test)\n",
    "disp.ax_.set_title('Complement NaiveBayes Precision-Recall curve')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'KNeighbors Precision-Recall curve')"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAsiElEQVR4nO3deZgU1fn28e8tooMgCIImsogawCAqIouKC4q4B1wSjTviEk1c3mg0JiYqaBb3JTFBokRNjBFNVDQouKHGuAARjGDcCCpIfiqLiAgKPu8fVTNpmpnpGpjucabvz3XNNV1Vp6qe6p7pp+qcqnMUEZiZWflar6EDMDOzhuVEYGZW5pwIzMzKnBOBmVmZcyIwMytzTgRmZmXOicCqJWmmpEEZy86RtG8NywZJmlufsX2ZSBot6acZymV+P7/sJHWVFJLWT6cnSzqloeOytedE0Ejlf/lK+rakRZL2yvlHnZC3zh8lXZpl+xGxXURMrt+oSyfnPVia/syRdGF97yciTo+IyzKUK8r7mSbaL9Jj/FjSa5JOqu/9WNPmRNAESDoRuAk4OCKeylk0QNJuDRRWvas8A62jTSKiFXA0cLGkA+ppu18m76XH2Br4PvA7ST0aOKZ61QQ+oy81J4JGTtJ3gGuA/SPiH3mLrwR+Vsu6h0iaLmmxpH9I2iFnWdUVh6QWkm5PrzhelXRBNdU9vSW9LOkjSXdLqsjb148lfZhu99ic+W0k3SHpA0lvS/qJpPXSZcMlPSvpOkkLgEslfU3SU+l+PpR0d5b3KSKeA2YCvSqrqyT9UNJ/gd9LWk/ShZLekrRA0jhJ7XLi3D19jxZLelfS8HT+bZIuT1+3l/RQWmahpGdyjiX3/dxQ0vWS3kt/rpe0YbqsMrbzJL0vaX7WM/xITAAWAjuk21vb4zpY0kuSlqTzL80SQz5JzdLP/q30imWapM7Kq15Ky1ZVMVXz2V+Wxtgrp3wHSZ9K2iydrvHv2WrnRNC4nQGMAgZHxNRqlv8G6K5q6u8l7QSMBb4DbArcDIyv/ELKcwnQFdgaGAIcV02ZI4EDgK1IvoSG5yz7CtAe6AicCIzJOWP9FdAm3fZewAlA7hffAGA2sDlJUrsMmAS0BTql69dKiYHAdsBLOTG1A7YETgPOAg5NY9gCWERylYWkLYGH0311AHoD06vZ1XnA3LTM5sCPger6cLkI2CXdzo5Af+AnOcu/QvKedAROBm6S1DbDca4naSjJe/1mOnttj+sTks9iE+Bg4AxJhxaKoRrnklyNHURyxTICWJZx3dzPfhTw13RblY4EnoqI9+v492z5IsI/jfAHmAMsAR4A1stb1pXkC2h94LvA8+n8PwKXpq9/C1yWt95rwF452983fT2b5IqjstwpwNy8WI7Lmb4SGJ2+HgSsBFrmLB8H/BRoBnwG9MxZ9h1gcvp6OPBOXox3AGOATgXen8r3YDHJl9+rwNk5MX0GVOSUf5UkoVZOfxX4PH0PfwTcV8N+bgMuT1+PSj+Pr9XweVW+n28BB+Us2x+YkxPbp8D6OcvfB3apYf+DgC/S41wBrAL+37oeVzX7uR64Lv/vK52eDJxSw3qvAcNq+Xxyj7NqOzV89vsCb+VMPwuckOXv2T+1//iKoHE7A+gO3CJJNZS5Bdhc0jfy5m8JnJdeRi+WtBjoTHLWmG8L4N2c6XerKfPfnNfLgFY504si4pOc6bfTbbYHmqfTucs61rKvCwABLyq5E2dENbHkah8RbSPi6xFxY878DyJiec70lsB9Oe/FqyRfqpuTvC9vFdgPwFUkZ+KTJM1WzY3TW7DmMee+7wsiYmXO9DKglaQu+l/j99Kc5e9FxCYkZ9w3Avus63FJGiDpSSVVdh8Bp5N8XnWV9b2rTv5n/ySwURpbV5IrmPvSZXX5e7Y8TgSN2/8Bg4E9SKqB1hARnwEjSapUcpPFu8DPImKTnJ+NIuKuajYzn6QaplLnOsbZVlLLnOkuwHvAhyRnp1vmLZuXewh5x/PfiDg1IrYguXr4jaSv1TGeNbZL8n4cmPd+VETEvHTZNgU3GPFxRJwXEVsDQ4FzJQ2upuh7rHnM72XY/jsR0aryp5rlK4AfAtvnVOOs7XH9CRgPdI6INsBoVv/7yaqmfVSeGGyUM+8reWXyP/tVJFeTR6c/D0XExzn7yfr3bHmcCBq5iHiPJBkcIOm6Gor9AaggqcOv9Dvg9PTsSpJapg2EG1ez/jjgR5LaSuoInLkWoY6UtIGkPYBDgHty/rF/JmnjtM76XJIqrGpJ+pakyqS0iOTL4ou1iCff6DSOLdP9dJA0LF12J7CvpCMlrS9pU0m9q4ntECWN2QI+Ijnzri62u4CfpPtoD1xMLcdcF2nivybd5roc18bAwohYLqk/cMxahnQLSUNvt/TvbAdJm0bEByQJ/7i0QXkEGZItSYI6Cjg2fV2pLn/PlseJoAmIiHdIqgO+KekX1SxfRfLF0C5n3lTgVODXJF+ob7J6A2+uUSSNoP8BHgPuJamPzuq/6T7eI/nyOT0i/p0uO4vk7HA28HeSf+6xtWyrH/BCWjUyHjgnImbXIZaa3JBub5Kkj4HnSRorK9/fg0gagxeSNKjuWM02upG8P0uB54DfRMST1ZS7HJgKvAz8C/hnOq++jAW6pNWBa3tc3wVGpetcTJKw18a16bqTSNq0bgVapMtOBc4HFpA05Off9baGiHiB5O9lC5KG7sr5dfl7tjyK8MA0VjeSzgC+HRF7NXQsZrbufEVgBUn6qqSB6e2JPUjOIO8rtJ6ZNQ5+Ws+y2IDkvuytSG5T/DM1NE6bWePjqiEzszLnqiEzszLX6KqG2rdvH127dm3oMMzMGpVp06Z9GBEdqlvW6BJB165dmTq1um51zMysJpLermmZq4bMzMqcE4GZWZlzIjAzK3NOBGZmZc6JwMyszBUtEUgaq2SovVdqWC5JN0p6U8kQh32KFYuZmdWsmFcEt7F6t8f5DiTprbEbyVCBvy1iLGZmVoOiJYKIeJqka9uaDAPuiMTzwCaSvlqseEY+OJORD84s1ubNzBqthnygrCOrD0U3N503P7+gpNNIrhro0qXLWu1s1ntL1mo9M7OmrlE0FkfEmIjoGxF9O3So9glpMzNbSw2ZCOax+ti3nVh9rFozMyuBhkwE44ET0ruHdgE+iog1qoXMzKy4itZGIOkuYBDQXtJc4BKgOUBEjAYmkIyX+iawDDipWLGYmVnNipYIIuLoAssD+F6x9m9mZtk0isZiMzMrHicCM7My50RgZlbmnAjMzMqcE4GZWZlzIjAzK3NOBGZmZc6JwMyszDkRmJmVOScCM7My50RgZlbmnAjMzMqcE4GZWZlzIjAzK3NOBGZmZc6JwMyszDkRmJmVOScCM7My50RgZlbmnAjMzMpc0Qav/zKaNX8JR938XEOH0WgN692RYwZ0aegwzKyelU0iGNa7Y0OH0KjNmr8EwInArAkqm0RwzIAu/hJbB76SMmu63EZgZlbmnAjMzMqcE4GZWZlzIjAzK3NOBGZmZS7TXUOSNgMGAlsAnwKvAFMj4osixmZmZiVQayKQtDdwIdAOeAl4H6gADgW2kXQvcE1ELClynGZmViSFrggOAk6NiHfyF0haHzgEGAL8pQixmZlZCdSaCCLi/FqWrQTur++AzMystNa6sVjSSRnKHCDpNUlvSrqwmuVdJD0p6SVJL0s6aG3jMTOztbMudw2NrG2hpGbATcCBQE/gaEk984r9BBgXETsB3wZ+sw7xmJnZWijUWPxyTYuAzQtsuz/wZkTMTrf1Z2AYMCunTACt09dtgPcKBWxmZvWrUGPx5sD+wKK8+QL+UWDdjsC7OdNzgQF5ZS4FJkk6C2gJ7FvdhiSdBpwG0KWLO44zM6tPhaqGHgJaRcTbeT9zgMn1sP+jgdsiohPJHUp/kLRGTBExJiL6RkTfDh061MNuzcysUqG7hk6uZdkxBbY9D+icM90pnZfrZOCAdHvPSaoA2pM8r2BfMh7YZ914YB/7sipmFxNTgG6StpK0AUlj8Pi8Mu8AgwEkfZ3kYbUPihiTraVhvTvS86utCxe0as2av4QHpuefB5l9ORRtYJqIWCnpTGAi0AwYGxEzJY0i6Z5iPHAe8DtJ3ydpOB4eEVGsmGzteWCfdeMrKfsyK+oIZRExAZiQN+/inNezSPowMjOzBuLeR83MylzmRCBpTG3TZmbWONXliuDmAtNmZtYIZU4EETGttmkzM2ucCnUx8SDJ3TzVioih9R6RmZmVVKG7hq4uSRRmZtZgCj1Z/FTla0ktgC4R8VrRozIzs5LJ1EYg6RvAdOCRdLq3pPynhM3MrBHK2lh8KUm30osBImI6sFVRIjIzs5LKmgg+j4iP8ua5KwgzsyYgaxcTMyUdAzST1A04m8LjEZiZWSOQ9YrgLGA7YAVwF7AE+H9FisnMzEoo0xVBRCwDLpJ0RTIZHxc3LDMzK5Wsdw31k/Qv4GXgX5JmSNq5uKGZmVkpZG0juBX4bkQ8AyBpd+D3wA7FCszMzEojaxvBqsokABARfwdWFickMzMrpUJ9DfVJXz4l6WaShuIAjqJ+Bq83M7MGVqhq6Jq86UtyXvs5AjOzJqBQX0N7lyoQMzNrGJnHLJZ0MMmzBBWV8yJiVDGCMjOz0sl6++hoknaBswAB3wK2LGJcZmZWIlnvGtotIk4AFkXESGBXoHvxwjIzs1LJmgg+TX8vk7QF8Dnw1eKEZGZmpZS1jeAhSZsAVwH/JLlj6JZiBWVmZqWTta+hy9KXf5H0EFBRTbfUZmbWCBV6oOzwWpYREX+t/5DMzKyUCl0RfKOWZQE4EZiZNXKFHig7qVSBmJlZw8h615CZmTVRTgRmZmXOicDMrMxl7WJiI0k/lfS7dLqbpEMyrHeApNckvSnpwhrKHClplqSZkv5Ut/DNzGxdZX2g7PfANJKuJQDmAfcAD9W0gqRmwE3AEGAuMEXS+IiYlVOmG/AjYGBELJK0Wd0PwaxxmDV/CUfd/FxDh9HkDOvdkWMGdGnoMBq1rFVD20TElSRdS1QOZq8C6/QH3oyI2RHxGfBnYFhemVOBmyJiUbrd9zNHbtaIDOvdkZ5fbd3QYTQ5s+Yv4YHp8xo6jEYv6xXBZ5JakA5GI2kbYEWBdToC7+ZMzwUG5JXpnm7vWaAZcGlEPJK/IUmnAacBdOnizG+NzzEDuvistQh8hVU/sl4RXAo8AnSWdCfwOHBBPex/faAbMAg4Gvhd2qfRaiJiTET0jYi+HTp0qIfdmplZpax9DU2SNA3YhaRK6JyI+LDAavOAzjnTndJ5ueYCL0TE58B/JL1OkhimZInLzMzWXda7hh4E9gMmR8RDGZIAJF/m3SRtJWkD4NvA+Lwy95NcDSCpPUlV0exsoZuZWX3IWjV0NbAHMEvSvZK+KamithUiYiVwJjAReBUYFxEzJY2SNDQtNhFYIGkW8CRwfkQsWKsjMTOztZK1augp4Kn0ltB9SO72GQvUehtEREwAJuTNuzjndQDnpj9mZtYA6jJ4fQuS3kiPAvoAtxcrKDMzK51MiUDSOJLnAh4Bfg08FRFfFDMwMzMrjaxXBLcCR0fEqmIGY2ZmpVdohLJ9IuIJoCUwTFr9YWKPUGZm1vgVuiLYC3iC6kcq8whlZmZNQKERyi5JX46KiP/kLpO0VdGiMjOzksn6HMFfqpl3b30GYmZmDaNQG8G2wHZAG0mH5yxqDdT6QJmZmTUOhdoIegCHAJuwejvBxyQPlZmZWSNXqI3gAeABSbtGhPt7NTNrggpVDV2QDkhzjKSj85dHxNlFi8zMzEqiUNXQq+nvqcUOxMzMGkahqqEH099V/QpJWg9oFRFLihybmZmVQNbxCP4kqbWklsArJN1Rn1/c0MzMrBSyPkfQM70COBR4GNgKOL5YQZmZWelkTQTNJTUnSQTj06Elo2hRmZlZyWRNBDcDc0g6n3ta0paA2wjMzJqArCOU3QjcmDPrbUl7FyckMzMrpayNxW0kXStpavpzDcnVgZmZNXJZq4bGknQrcWT6swT4fbGCMjOz0sk6Qtk2EXFEzvRISdOLEI+ZmZVY1iuCTyXtXjkhaSDwaXFCMjOzUsp6RXA6cIekNun0IuDE4oRkZmalVDARSOoNfA34NjAPwN1LmJk1HbVWDUm6GBgHHAH8DTjKScDMrGkpdEVwFNA7IpZJ2hR4BPhd8cMyM7NSKdRYvCIilgFExIIM5c3MrJEpdEWwtaTx6WsB2+RMExFDixaZmZmVRKFEMCxv+upiBWJmZg2j0MA0T5UqEDMzaxiF7hp6UNI30i6o85dtLWmUpBHFC8/MzIqtUNXQqcC5wPWSFgIfABVAV+At4NcR8UBRIzQzs6IqVDX0X+AC4AJJXYGvknQt8Xrl3US1kXQAcAPQDLglIn5ZQ7kjgHuBfhExtU5HYGZm6yRrFxNExBySwWkykdQMuAkYAswFpkgaHxGz8sptDJwDvJB122ZmVn+K+VxAf+DNiJgdEZ8Bf2bNu5AALgOuAJYXMRYzM6tBMRNBR+DdnOm56bwqkvoAnSPib7VtSNJplYPifPDBB/UfqZlZGWuwJ4UlrQdcC5xXqGxEjImIvhHRt0OHDsUPzsysjGRqI0jHH7gU2DJdR0BExNa1rDYP6Jwz3SmdV2ljoBcwWRLAV4Dxkoa6wdjMrHSyNhbfCnwfmAasyrjOFKCbpK1IEsC3gWMqF0bER0D7ymlJk4EfOAmYmZVW1kTwUUQ8XJcNR8RKSWcCE0luHx0bETMljQKmRsT42rdgZmalkDURPCnpKuCvwIrKmRHxz9pWiogJwIS8eRfXUHZQxljMzKweZU0EA9LffXPmBbBP/YZjZmallikRRMTexQ7EzMwaRqbbRyW1kXRt5b38kq7JGcjezMwasazPEYwFPgaOTH+WAL8vVlBmZlY6WdsItomII3KmR0qaXoR4zMysxLJeEXwqaffKifQBs0+LE5KZmZVS1iuCM4Db03YBAQuB4cUKyszMSifrXUPTgR0ltU6nlxQzKDMzK51aE4Gk4yLij5LOzZsPQERcW8TYzMysBApdEbRMf29c7EDMzKxhFBqq8ub098jShGNmZqWW9YGyKyW1ltRc0uOSPpB0XLGDMzOz4st6++h+aQPxISTjFn8NOL9YQZmZWelkTQSVVUgHA/ekYwmYmVkTkPU5gock/ZvkIbIzJHXAg82bmTUJma4IIuJCYDegb0R8DnwCDCtmYGZmVhqFniPYJyKekHR4zrzcIn8tVmBmZlYahaqG9gKeAL5RzbLAicDMrNEr9BzBJenvk0oTjpmZlVrW5wh+LmmTnOm2ki4vWlRmZlYyWW8fPTAiFldORMQi4KCiRGRmZiWVNRE0k7Rh5YSkFsCGtZQ3M7NGIutzBHcCj0uqHJ7yJOD24oRkZmallHU8giskzQD2TWddFhETixeWmZmVStYrAoBXgZUR8ZikjSRtHBEfFyswMzMrjax3DZ0K3AvcnM7qCNxfpJjMzKyEsjYWfw8YCCwBiIg3gM2KFZSZmZVO1kSwIiI+q5yQtD7Jk8VmZtbIZU0ET0n6MdBC0hDgHuDB4oVlZmalkjUR/BD4APgX8B1gAvCTYgVlZmalU/CuIUnNgJkRsS3wu+KHZGZmpVTwiiAiVgGvSepS141LOkDSa5LelHRhNcvPlTRL0svpWMhb1nUfZma2brI+R9AWmCnpRZJBaQCIiKE1rZBeSdwEDAHmAlMkjY+IWTnFXiIZ7GaZpDOAK4Gj6ngMZma2DrImgp+uxbb7A29GxGwASX8mGdWsKhFExJM55Z8HjluL/ZiZ2TooNEJZBXA68DWShuJbI2Jlxm13BN7NmZ4LDKil/MnAwzXEcRpwGkCXLnWuoTIzs1oUaiO4HehLkgQOBK4pRhCSjkv3c1V1yyNiTET0jYi+HTp0KEYIZmZlq1DVUM+I2B5A0q3Ai3XY9jygc850p3TeaiTtC1wE7BURK+qwfTMzqweFrgg+r3xRhyqhSlOAbpK2krQB8G1gfG4BSTuR9F80NCLer+P2zcysHhS6IthR0pL0tUieLF6Svo6IaF3TihGxUtKZwESgGTA2ImZKGgVMjYjxJFVBrYB7JAG8U9udSGZmVv8KDV7fbF02HhETSJ5Czp13cc7rfddYyczMSiprFxNmZtZEORGYmZU5JwIzszLnRGBmVuacCMzMypwTgZlZmXMiMDMrc04EZmZlzonAzKzMORGYmZU5JwIzszLnRGBmVuacCMzMypwTgZlZmXMiMDMrc04EZmZlzonAzKzMFRqqslH4/PPPmTt3LsuXL2/oUMy+VCoqKujUqRPNmzdv6FDsS6xJJIK5c+ey8cYb07VrV9Kxj83KXkSwYMEC5s6dy1ZbbdXQ4diXWJOoGlq+fDmbbrqpk4BZDklsuummvlK2gppEIgCcBMyq4f8Ly6LJJAIzM1s7TgT1pFWrVlWvJ0yYQPfu3Xn77be59NJL2WijjXj//ferLVuTgw46iMWLF9daZtCgQUydOnWN+bfddhtnnnlm9uDr4Oqrr2bbbbeld+/e9OvXjzvuuKPWWNbG1KlTOfvsswFYsWIF++67L7179+buu+/mlFNOYdasWeu0/euvv74qboCVK1fSoUMHLrzwwtXKDRo0iB49erDjjjsycOBAXnvttXXaL8Dtt99Ot27d6NatG7fffnuN5X71q1+x7bbbst1223HBBRcAsGDBAvbee29atWq1xue77777smjRonWOz8pTk2gs/jJ5/PHHOfvss5k4cSJbbrklAO3bt+eaa67hiiuuyLydCRMmFCvEWkUEEcF66615jjB69GgeffRRXnzxRVq3bs2SJUu477776j2Gvn370rdvXwBeeuklAKZPnw7AUUcdVadtrVq1imbNmlVNr1y5krFjx/LPf/6zat6jjz5K9+7dueeee/jFL36xWnXKnXfeSd++fRkzZgznn38+48ePX9vDYuHChYwcOZKpU6ciiZ133pmhQ4fStm3b1co9+eSTPPDAA8yYMYMNN9yw6iSioqKCyy67jFdeeYVXXnlltXWOP/54fvOb33DRRRetdXxWvprcFcHIB2dy1M3P1evPyAdnZtr3008/zamnnspDDz3ENttsUzV/xIgR3H333SxcuHCNdf74xz/Sv39/evfuzXe+8x1WrVoFQNeuXfnwww8BuOyyy+jRowe77747Rx99NFdffXXV+vfccw/9+/ene/fuPPPMM1Xz3333XQYNGkS3bt0YOXJk1fxrr72WXr160atXL66//noA5syZQ48ePTjhhBPo1asX7777LsOHD6dXr15sv/32XHfddQD8/Oc/57e//S2tW7cGoHXr1px44olrHNMZZ5xB37592W677bjkkkuq5l944YX07NmTHXbYgR/84AdV8ffq1Ysdd9yRPffcE4DJkydzyCGH8P7773PccccxZcoUevfuzVtvvbXalcekSZPYdddd6dOnD9/61rdYunRp1Xv3wx/+kD59+nDPPfesFtsTTzxBnz59WH/9/50D3XXXXZxzzjl06dKF5557rppPFvbcc0/efPPNapdlNXHiRIYMGUK7du1o27YtQ4YM4ZFHHlmj3G9/+1suvPBCNtxwQwA222wzAFq2bMnuu+9ORUXFGusMHTqUu+66a53is/LlK4J6smLFCg499FAmT57Mtttuu9qyVq1aMWLECG644YbVvpRfffVV7r77bp599lmaN2/Od7/7Xe68805OOOGEqjJTpkzhL3/5CzNmzODzzz+nT58+7LzzzlXLV65cyYsvvsiECRMYOXIkjz32GAAvvvgir7zyChtttBH9+vXj4IMPRhK///3veeGFF4gIBgwYwF577UXbtm154403uP3229lll12YNm0a8+bNqzrrXLx4MUuWLOHjjz9m6623Lvhe/OxnP6Ndu3asWrWKwYMH8/LLL9OxY0fuu+8+/v3vfyOpqtpr1KhRTJw4kY4dO65RFbbZZptxyy23cPXVV/PQQw+ttuzDDz/k8ssv57HHHqNly5ZcccUVXHvttVx88cUAbLrppqud9Vd69tlnV3v/li9fzmOPPcbNN9/M4sWLueuuu9htt93WWO/BBx9k++23X2P+VVddxZ133rnG/D333JMbb7xxtXnz5s2jc+fOVdOdOnVi3rx5a6z7+uuv88wzz3DRRRdRUVHB1VdfTb9+/dYol6tt27asWLGCBQsWsOmmm9Za1ixfk0sEl3xjuwbZb/Pmzdltt9249dZbueGGG9ZYfvbZZ9O7d++qM2FIqpGmTZtW9U/+6aefVp39VXr22WcZNmwYFRUVVFRU8I1vfGO15YcffjgAO++8M3PmzKmaP2TIkKovhMMPP5y///3vSOKwww6jZcuWVfOfeeYZhg4dypZbbskuu+wCwNZbb83s2bM566yzOPjgg9lvv/2qzrazGDduHGPGjGHlypXMnz+fWbNm0bNnTyoqKjj55JM55JBDOOSQQwAYOHAgw4cP58gjj6w6liyef/55Zs2axcCBAwH47LPP2HXXXauW11SFNH/+fL7+9a9XTT/00EPsvffetGjRgiOOOILLLruM66+/vqo66dhjj6VFixZ07dqVX/3qV2ts7/zzz+f888/PHHcWK1euZOHChTz//PNMmTKFI488ktmzZxe8A2izzTbjvffecyKwOmtyiaChrLfeeowbN47Bgwfz85//nB//+MerLd9kk0045phjuOmmm6rmRQQnnngiv/jFL9Z6v5XVB82aNWPlypVV8/O/NAp9iVQmB0jOLmfMmMHEiRMZPXo048aNY+zYsbRq1YrZs2fXelXwn//8h6uvvpopU6bQtm1bhg8fzvLly1l//fV58cUXefzxx7n33nv59a9/zRNPPMHo0aN54YUX+Nvf/sbOO+/MtGnTMh13RDBkyJAaq0NyjydXixYtVruv/q677uLvf/87Xbt2BZIG2SeeeIIhQ4YA/2sjqEldrgg6duzI5MmTq6bnzp3LoEGD1li3U6dOHH744Uiif//+rLfeenz44Yd06NChxjggubpp0aJFrWXMqtPk2gga0kYbbcTf/vY37rzzTm699dY1lp977rncfPPNVV/YgwcP5t57761qDFy4cCFvv/32ausMHDiQBx98kOXLl7N06dI1qkhq8uijj7Jw4UI+/fRT7r//fgYOHMgee+zB/fffz7Jly/jkk0+477772GOPPdZY98MPP+SLL77giCOO4PLLL6+qYvnRj37E9773PZYsWQLA0qVLV7v7BmDJkiW0bNmSNm3a8H//9388/PDDVWU/+ugjDjroIK677jpmzJgBwFtvvcWAAQMYNWoUHTp04N133810fLvssgvPPvtsVb39J598wuuvv15wva9//etV6yxZsoRnnnmGd955hzlz5jBnzhxuuummOtW1n3/++UyfPn2Nn/wkALD//vszadIkFi1axKJFi5g0aRL777//GuUOPfRQnnzySSCpJvrss89o3759rXFEBP/973+rEppZXfiKoJ61a9eORx55hD333HONM7j27dtz2GGHVTW+9uzZk8svv5z99tuPL774gubNm3PTTTdV3W0E0K9fP4YOHcoOO+zA5ptvzvbbb0+bNm0KxtG/f3+OOOII5s6dy3HHHVd1Vjt8+HD69+8PwCmnnMJOO+20WpUSJHXZJ510El988QVA1RXLGWecwdKlS+nXrx/NmzenefPmnHfeeautu+OOO7LTTjux7bbb0rlz56qqm48//phhw4axfPlyIoJrr70WSL5I33jjDSKCwYMHs+OOO/LUU08VPL4OHTpw2223cfTRR7NixQoALr/8crp3717regceeCDHH388APfddx/77LNP1VUVwLBhw7jggguqtlmf2rVrx09/+tOqqsCLL76Ydu3aAclncfrpp9O3b19GjBjBiBEj6NWrFxtssAG333571RVd165dWbJkCZ999hn3338/kyZNomfPnkybNo1ddtlltUbwcjFr/hKOurn6Rv6mpucWrYtS/a2IqPeNFlPfvn0j/371V199dbV636Zm6dKltGrVimXLlrHnnnsyZswY+vTp09BhNVqHHXYYV155Jd26dWvoUOrNOeecw9ChQxk8ePAay5ry/8efXniHB6av2eDeVK1LIpA0LSKqrecsv9OHRui0005j1qxZLF++nBNPPNFJYB398pe/ZP78+U0qEfTq1avaJNDUHTOgC8cM6NLQYTR6RU0Ekg4AbgCaAbdExC/zlm8I3AHsDCwAjoqIOcWMqTH605/+1NAhNCk9evSgR48eDR1GvTr11FMbOgRrxIrWWCypGXATcCDQEzhaUs+8YicDiyLia8B1QPZHb/M0tious1Lw/4VlUcy7hvoDb0bE7Ij4DPgzMCyvzDCgssOVe4HBWovuEisqKliwYIH/6M1yVI5HUN2TyGa5ilk11BHIvRdwLjCgpjIRsVLSR8CmwIe5hSSdBpwG0KXLmvWBnTp1Yu7cuXzwwQf1FrxZU1A5QplZbRpFY3FEjAHGQHLXUP7y5s2bewQmM7O1VMyqoXlA55zpTum8astIWh9oQ9JobGZmJVLMRDAF6CZpK0kbAN8G8vvwHQ9Udl/5TeCJcEW/mVlJFa1qKK3zPxOYSHL76NiImClpFDA1IsYDtwJ/kPQmsJAkWZiZWQk1uieLJX0AvF2wYPXak9cQXQZ8zOXBx1we1uWYt4yIansubHSJYF1ImlrTI9ZNlY+5PPiYy0Oxjtm9j5qZlTknAjOzMlduiWBMQwfQAHzM5cHHXB6Kcsxl1UZgZmZrKrcrAjMzy+NEYGZW5ppkIpB0gKTXJL0p6cJqlm8o6e50+QuSujZAmPUqwzGfK2mWpJclPS5py+q205gUOuacckdICkmN/lbDLMcs6cj0s54pqdEPZpHhb7uLpCclvZT+fR/UEHHWF0ljJb0v6ZUalkvSjen78bKkdR+pKiKa1A/JU8xvAVsDGwAzgJ55Zb4LjE5ffxu4u6HjLsEx7w1slL4+oxyOOS23MfA08DzQt6HjLsHn3A14CWibTm/W0HGX4JjHAGekr3sCcxo67nU85j2BPsArNSw/CHgYELAL8MK67rMpXhGUbByEL5GCxxwRT0bEsnTyeZJOABuzLJ8zwGUkAx4tL2VwRZLlmE8FboqIRQAR8X6JY6xvWY45gNbp6zbAeyWMr95FxNMkXe7UZBhwRySeBzaR9NV12WdTTATVjYPQsaYyEbESqBwHobHKcsy5TiY5o2jMCh5zesncOSL+VsrAiijL59wd6C7pWUnPp8PFNmZZjvlS4DhJc4EJwFmlCa3B1PX/vaBGMR6B1R9JxwF9gb0aOpZikrQecC0wvIFDKbX1SaqHBpFc9T0tafuIWNyQQRXZ0cBtEXGNpF1JOrLsFRFfNHRgjUVTvCIox3EQshwzkvYFLgKGRsSKEsVWLIWOeWOgFzBZ0hySutTxjbzBOMvnPBcYHxGfR8R/gNdJEkNjleWYTwbGAUTEc0AFSedsTVWm//e6aIqJoBzHQSh4zJJ2Am4mSQKNvd4YChxzRHwUEe0jomtEdCVpFxkaEVMbJtx6keVv+36SqwEktSepKppdwhjrW5ZjfgcYDCDp6ySJoCmPWzseOCG9e2gX4KOImL8uG2xyVUNRhuMgZDzmq4BWwD1pu/g7ETG0wYJeRxmPuUnJeMwTgf0kzQJWAedHRKO92s14zOcBv5P0fZKG4+GN+cRO0l0kybx92u5xCdAcICJGk7SDHAS8CSwDTlrnfTbi98vMzOpBU6waMjOzOnAiMDMrc04EZmZlzonAzKzMORGYmZU5JwIrOkmrJE2X9IqkByVtUs/bn5PeM4+kpTWUaSHpKUnNJHWV9Gka0yxJo9Mnkeuyz76SbkxfD5K0W86y0yWdsC7HlG7nUkk/KFDmNknfrMM2u9bUq2VeuZ9Jejf//ZR0pqQRWfdnjYMTgZXCpxHROyJ6kTy38b0GiGEE8NeIWJVOvxURvYEdSHqsPLQuG4uIqRFxdjo5CNgtZ9noiLhjXQNuYA+SdPiWbyxNvy+fsuNEYKX2HGkHWZK2kfSIpGmSnpG0bTp/c0n3SZqR/uyWzr8/LTtT0ml13O+xwAP5M9NOB/8BfC09W35C/xuzoUu632+lVzMzJD2dzhsk6SElY1mcDnw/vcLYo/JMXtK2kl6s3Fe6/X+lr3dOr1CmSZqoAr1HSjpV0pQ0hr9I2ihn8b6Spkp6XdIhaflmkq5K13lZ0nfq8mZFxPPVPa2a9mA7R1J1ScIaKScCKxlJzUi6Aqh86ncMcFZE7Az8APhNOv9G4KmI2JGkX/aZ6fwRadm+wNmSMvUYm3ZNsHVEzKlm2UZpTP8CfgXcHhE7AHemcQBcDOyfxrPa09jpNkcD16VXPc/kLPs3sIGkrdJZRwF3S2qe7uub6fGMBX5W4DD+GhH90hheJelfp1JXkrP3g4HRkirS5R9FRD+gH3BqThyVx76FpAkF9ludqcAea7GefUk1uS4m7EuphaTpJFcCrwKPSmpFUp1S2eUFwIbp732AEwDSqpyP0vlnSzosfd2ZpDO1LN0ntAcW583bJo0pgAci4mFJfwAOT5f/Abgyff0scJukccBfM+wv1ziSBPDL9PdRQA+SDvEeTY+9GVCor5heki4HNiHpKmRi7j7SnjbfkDQb2BbYD9ghp/2gDcn79XrlShHxHklXBXX1froPayKcCKwUPo2I3unZ90SSNoLbgMVpPX1BkgYB+wK7RsQySZNJOhfLtP9qyr6Vdd8RcbqkASRn3NMk7ZxxvwB3kyS7vyabijckbQ/MjIhd67Cd24BDI2KGpOGkHctVhpgfMsnoVWdFRG7CQPUzLGsFyXtqTYSrhqxk0vrls0k6CVsG/EfSt6BqHNYd06KPkwynWVnX3YbkjHZRmgS2JelWOut+FwHN0iqT2vyD/3VAeCzwTBrDNhHxQkRcTNKrZee89T4m6fa6un2/RdL5209JkgLAa0AHJX3nI6m5pO0KxLYxMD+tVjo2b9m3JK0naRuSIR1fI0m4Z6TlkdRdUssC+8iqO1DwziNrPJwIrKQi4iXgZZLBRI4FTpY0g6QdoHIIwnOAvdOG1Wkkd/U8Aqwv6VWSapbn67jrScDuBcqcBZwk6WXg+DQOgKsk/Su97fIfJOPm5noQOKyysbia7d4NHMf/+sz/jKT78yvSY59Ozl1HNfgp8AJJNdW/85a9A7xIMurc6RGxHLgFmAX8M437ZvJqAGprI5B0pZKeLzeSNFfSpTmLBwKPFojXGhH3PmplQcmwld+PiOMbOpbGTMm4Fuf6fWxafEVgZSEi/gk8md65ZGuvPcnViTUhviIwMytzviIwMytzTgRmZmXOicDMrMw5EZiZlTknAjOzMvf/AbtKpIsgCvNCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "disp = plot_precision_recall_curve(clf3, x_test, y_test)\n",
    "disp.ax_.set_title('KNeighbors Precision-Recall curve')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# F1 Score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "SGDCLassifier'ın f1 score :  0.9872873730346972\n"
     ]
    }
   ],
   "source": [
    "f1_score_clf = f1_score(y_test, y_predict, average='weighted')\n",
    "print(\"SGDCLassifier'ın f1 score : \", f1_score_clf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Complement NaiveBayes'ın f1 score :  0.953325771246443\n"
     ]
    }
   ],
   "source": [
    "f1_score_clf2 = f1_score(y_test, y2_predict, average='weighted')\n",
    "print(\"Complement NaiveBayes'ın f1 score : \", f1_score_clf2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KNeighbors'ın f1 score :  0.9840925520633168\n"
     ]
    }
   ],
   "source": [
    "f1_score_clf3 = f1_score(y_test, y3_predict, average='weighted')\n",
    "print(\"KNeighbors'ın f1 score : \", f1_score_clf3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  &&&&&&&&&&&&&&& Kümeleme Algoritmaları &&&&&&&&&&&&&&&&&"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  --------------------------------------------KMeans----------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "K-ortalama kümeleme ya da K-means kümeleme (K-means clustering) yöntemi N adet veri nesnesinden oluşan bir veri kümesini giriş parametresi olarak verilen K adet kümeye bölümlemektir. Amaç, gerçekleştirilen bölümleme işlemi sonunda elde edilen kümelerin, küme içi benzerliklerinin maksimum ve kümeler arası benzerliklerinin ise minimum olmasını sağlamaktır. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model oluşturma ve eğitim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Veri setimizde 'pos' ve 'neg' olmak üzere 2 farklı grup var. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "kmeans = KMeans(n_clusters=2, random_state=0).fit(x_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test Verisinin Tahmini"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 0, 0, 0])"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_cls_predict = kmeans.predict(x_test)\n",
    "y_cls_predict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[15346,   279],\n",
       "       [  105,   270]], dtype=int64)"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD4CAYAAAAn3bdmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAgyUlEQVR4nO3dd5wV1f3/8ddnF4gCwtJFlggoatCoAUWMftVIAgRFUFFBpVn2p4IYsQQVA0FjjwX7ojQbIhZQUCSIGg1diQ0LisAivcuqsPD5/XEP5AK77N0GzPB+Ph7z2JkzZ2bOsJfPPfuZMzPm7oiISDSk7ekGiIhI6hS0RUQiREFbRCRCFLRFRCJEQVtEJELKlfUBBhxRXsNTZCcDZi3a002QvVGl2lbSXRQl5gz4clOJj7e7qactIhIhZd7TFhHZnSLXdS4iBW0RiZX0mEdtBW0RiRVT0BYRiY6Yx2wFbRGJF/W0RUQiJO5D4hS0RSRW1NMWEYmQNAVtEZHoiHnMVtAWkXhRT1tEJEJiHrMVtEUkXnQhUkQkQtIt3g8WVdAWkViJeUc79uPQRWQfY0WYCt2X2RAzW2Zmn+Wz7jozczOrGZbNzAaZ2Vwz+8TMmibV7WZm34SpW1J5MzP7NGwzyKzw5I6CtojEilnqUwqGAW12PobVB1oBC5KK/ww0DlMW8HioWx3oD5wANAf6m1m1sM3jwOVJ2+10rB0paItIrJRmT9vd3wdW5bPqAeBGIDmB3h4Y4QlTgQwzqwu0Bia6+yp3Xw1MBNqEdVXcfaq7OzAC6FBYm5TTFpFYKetx2mbWHljk7v/dIZtRD1iYtJwTynZVnpNP+S4paItIrBQlaJtZFolUxlbZ7p69i/oVgZtJpEb2CAVtEYmVonS0Q4AuMEjn4xCgIbC1l50JfGRmzYFFQP2kupmhbBFw2g7l74byzHzq75Jy2iISK2mW+lRU7v6pu9d29wbu3oBESqOpuy8BxgJdwyiSFsBad18MTABamVm1cAGyFTAhrFtnZi3CqJGuwJhCz6/ozRYR2XuV8pC/F4ApwOFmlmNml+6i+njgO2AuMBi4CsDdVwG3ATPCNDCUEeo8Fbb5FnizsDYpPSIisVKat7G7e+dC1jdImnegZwH1hgBD8imfCRxVlDYpaItIrOgpfyIiERL3nK+CtojEip7yJyISITGP2QraIhIvymmLiERIzGO2graIxEu5mEdtBW0RiRVdiBQRiRAN+RMRiRD1tEVEIiRNL/YVEYkOpUdERCIkXekREZHoUE5bRCRClB4REYkQ3cYuIhIhSo+IiESILkSKiESIctoiIhES95x23L+URGQfU8pvYx9iZsvM7LOksnvN7Esz+8TMXjWzjKR1N5nZXDP7ysxaJ5W3CWVzzaxvUnlDM5sWyl80swqFtUlBW0RiJc1Sn1IwDGizQ9lE4Ch3Pxr4GrgJwMyaAJ2AI8M2j5lZupmlA48CfwaaAJ1DXYC7gQfc/VBgNXBpoeeXUrNFRCKiNHva7v4+sGqHsrfdPS8sTgUyw3x7YKS7/+Lu84C5QPMwzXX379x9IzASaG9mBpwOjA7bDwc6FNYmBW0RiZVyaZ7yZGZZZjYzacoq4uEuAd4M8/WAhUnrckJZQeU1gDVJXwBby3d9fkVsoIjIXq0o1yHdPRvILtZxzG4B8oDnirN9cSloi0is7I7RI2bWHTgTaOnuW58Fuwion1QtM5RRQPlKIMPMyoXednL9Aik9sgvt/zGYGz5cxFVjP853fYPmp9B3xgqueHUmV7w6k1OvuqXEx0wvX4GO9z9H7wlzuOzFD8mod/B266vWrc/Ns1bz+0uuLfGxpOgWL1lKl6zetD33Ys7o2IXhz7+0U53163/kimv+ylkXdOeMjl14ecy4Eh93zdp19LjyWlq170yPK69l7br1AIwd/zbtzu9Gu/O70an7lXz59dwSHyvqSjOnne/+zdoANwJnuXtu0qqxQCcz+5WZNQQaA9OBGUDjMFKkAomLlWNDsJ8MdAzbdwPGFHZ8Be1dmP3qcJ69/Mxd1lkw6wOeOPs4njj7ON577B8p7zuj3sF0H/GvncqbdryEn9etYVDr3zB1+EP88bo7tlvfuu+9fPPvt1I+jpSu9PR0+l7bk/EvP8uLw5/k+VGvMPe7edvVeW7UKxzSqAFjXxzGM4MHcfcDj7Jx06aU9j9t5sf07b/z5yh76LOc2LwZb495gRObNyN76LMAZNary7NPPcLro4Zz5eXduPX2e0p+khFXmqNHzOwFYApwuJnlmNmlwCPAAcBEM5ttZk8AuPvnwCjgC+AtoKe7bw696F7ABGAOMCrUBfgr0MfM5pLIcT9dWJuUHtmF+TM/2Kmnm6qj213ICV16kV6+AjmfTGfc33vhW7YUut3hLdvx7iO3AfDFhJdpe+tD29Yd0fIsVud8z6afNhSrTVJytWvVpHatmgBUrlSRRg0bsHTZCg5t1HBbHTNjQ24u7s6G3J+oWqUK5dLTAXhq+PO8OXEyGzdu5E9/OIXeVxY6wguASe99wDPZgwDocGYbumT15oZrrqTpMb/dVufY3x7JkqXLS+tUI6s0e6Lu3jmf4gIDq7v/A9jpW9fdxwPj8yn/jsTokpSldH5mdo+ZVTGz8mY2ycyWm9nFRTlQXGUe24IrXpvFRdmvU+vQxNDLmo2O4Mi25/H0hafwxNnH4Zs3c3S7C1PaX5XaB7FuceJC85bNm/l5/VoqZtSgQsVKnHT5Dbz36G1ldi5SNDk/LGbOV19zzFFNtiu/6IJz+XbefP6vdQfOOr87t9zQm7S0ND6YMp35C3IY/Uw2Y0YO5fM5XzFj1uyUjrVy5eptXxa1atZg5crVO9UZ/dobnHLSCSU+r6hLT0t9iqJUe9qt3P1GMzsb+B44B3gfeDa/ymHYTBbAmXXSaJYR0X+dQiz+/GMePP0QNuZuoPEpbej0yGgebtOERieezkFHNiXrpakAlNtvPzasWgbABQ+/RLXMhqSXL0/Vur/mildnAjD1mYeZ/crwAo91Wq+/MXXYQ2zMVS97b7AhN5fe1/fj5ut6U7lype3WfTBlGr857FBGPPkQCxYuosdVfTjud8fw4dQZfDh1Bh06XwJAbu5PfL8wh+ObHct5XbPYuHETubk/sXbdOtp36gHA9b2v4P9+v30gNrOdnmQ3dcZHjH5tHM8PebTsTjoiYn4Xe8pBe2u9M4CX3H2t7eL5h8nDaAYcUT62b9n8ZcP6bfPfvP8WZ/R/mIoZNcCM2a89w6T7++20zYtXnwckctod7nyaYV3/uN36dct+oErd+qxbuoi09HT2O6AquWtWUu/o5jRpfQ5/uuFO9jsgA9+yhbxffmH6c4+V7UnKTjZtyqP39f1o1/ZPtGp56k7rXxk7nqzuF2NmHPzrTDIPqst338/H3cnqcTGdOrbfaZuXRiRGnU2b+TGvvj6eu/6+/UXtGjWqsWz5CmrXqsmy5SuoXr3atnVffj2XfrfdzeCH76VaRtVSPtvo2VVsioNUu8BvmNmXQDNgkpnVAn4uu2ZFQ+WadbbN1/vt8ZilkbtmJfOmvEOTVudQqXotAPavWo2qB/06pX1+9c4bHNuhCwBNWp/LvKmTARh68R94sGVjHmzZmKkjBvHv7LsUsPcAd+eWgXfRqGEDelzcKd86dQ+sw5TpswBYsXIV8+YvILPeQZx8YnNeHjuODbmJAQdLly1n5aqd0xz5Of2Uk3jtjcQF6NfeeIuWp54MwA+Ll3L19f2457Z+NDw4tc9Y3JmlPkVRSj1td+9rZvcAa919s5ltIHHLZqyd+89naHD8qVSsVpM+785j8sMDSS9XHoCZL2bTpPW5HNcpiy2bN5P380+Mvi6R5l/+7Rzeeag/XZ5+E0tLY3PeJsYP7M3aHxYUesyPRw/h7HuG0XvCHH5au5rRfS4q03OUopk1+1PGjJvAYYc22pbC6NMrix+WLAWgc8cOXHV5d27qfwftzu+Gu3N97yuoXi2Dk09szrfz5tOp+5UAVNx/f+69/VZqJPWaC5LV42L+8te/Mfq1cRxUtw4P3j0QgEcHD2XN2rX8/c77gcTolleee6osTj06ohqNU2T/Gxe+i0pm5YErgVNC0XvAE+5e6DimOKdHpPgGzCr0HgLZF1WqXeKI+91ZqcecRmM3RS7Cp5rTfhwoD2z9e7xLKLusLBolIlJccc9ppxq0j3f3Y5KW3zGz/5ZFg0RESiSeg9W2SfX0NpvZIVsXzKwRsLlsmiQiUnyJIZGpTVGUak/7BmCymX0XlhsAPcqkRSIiJRDRWJyyVHvaHwJPAltIPBD8SRL344uI7FXU004YAawDtt5DfSHwDHBeWTRKRKS4LOZv9k01aB/l7skPWJhsZl+URYNEREoioh3olKWaHvnIzFpsXTCzE4CZZdMkEZHiU3okoRnwHzPbekvfr4GvzOxTwMNbiUVE9ryIBuNUpRq0d3yFvIjIXinmMTvlZ4/ML+uGiIiUhqimPVKlN9eISKxo9IiISITEvKMd97v0RWRfU5qjR8xsiJktM7PPksqqm9lEM/sm/KwWys3MBpnZXDP7xMyaJm3TLdT/xsy6JZU3M7NPwzaDLIVGKWiLSKyU8pC/Yew8EKMvMMndGwOTwjLAn4HGYcoi8SRUzKw60B84gcRLfPtvDfShzuVJ2xU66ENBW0RipTTfXOPu75N4dEey9sDWF7oOBzoklY/whKlAhpnVBVoDE919lbuvBiYCbcK6Ku4+1RMvNhiRtK8CKactIrGyGy5E1nH3xWF+CbD1vYP1gIVJ9XJC2a7Kc/Ip3yX1tEUkVoqSHjGzLDObmTRlFeVYoYe8W9/OpZ62iMRKUUaPuHs2kF3EQyw1s7ruvjikOJaF8kVA/aR6maFsEXDaDuXvhvLMfOrvknraIhIvZf869rHA1hEg3YAxSeVdwyiSFiRehL4YmAC0MrNq4QJkK2BCWLfOzFqEUSNdk/ZVIPW0RSRWSvOOSDN7gUQvuaaZ5ZAYBXIXMMrMLgXmA+eH6uOBtsBcIJfwohh3X2VmtwEzQr2B7r714uZVJEao7A+8GaZdUtAWkVgpzZtr3L1zAata5lPXgZ4F7GcIMCSf8pnAUUVpk4K2iMSKbmMXEYkQS4v3pToFbRGJl5g/fERBW0TiRUFbRCQ6zJQeERGJDuW0RUSiQ2+uERGJEqVHRESiQ+O0RUSiROkREZEIUXpERCQ6dEekiEiUKD0iIhIdurlGRCRK1NMWEYkODfkTEYkSpUdERKJDo0dERKJEOW0RkQiJeXok3mcnIvscM0t5SmFf15rZ52b2mZm9YGb7mVlDM5tmZnPN7EUzqxDq/ioszw3rGyTt56ZQ/pWZtS7J+Sloi0i8pFnq0y6YWT2gN3Ccux8FpAOdgLuBB9z9UGA1cGnY5FJgdSh/INTDzJqE7Y4E2gCPmVl6sU+vuBuKiOyNLC095SkF5YD9zawcUBFYDJwOjA7rhwMdwnz7sExY39IS3fn2wEh3/8Xd5wFzgebFPT8FbRGJF7OUJzPLMrOZSVPW1t24+yLgPmABiWC9FpgFrHH3vFAtB6gX5usBC8O2eaF+jeTyfLYpMl2IFJFYKcqba9w9G8guYD/VSPSSGwJrgJdIpDf2KPW0RSReLC31adf+CMxz9+Xuvgl4BTgJyAjpEoBMYFGYXwTUBwjrqwIrk8vz2abIFLRFJF6KkB4pxAKghZlVDLnplsAXwGSgY6jTDRgT5seGZcL6d9zdQ3mnMLqkIdAYmF7c0yvz9MiAjxaX9SEkinzLnm6BxFRpvdjX3aeZ2WjgIyAP+JhEKmUcMNLMbg9lT4dNngaeMbO5wCoSI0Zw98/NbBSJgJ8H9HT3zcVtlyW+CMpQ7ooyPoBEkoK25KdS7RJH3C0PnppyzEn7y3uRu31SFyJFJF5ifkekgraIxIuePSIiEiHqaYuIRIh62iIiEaKetohIhKT2TJHIUtAWkXhRekREJEIUtEVEIkQ5bRGRCFFPW0QkQnQhUkQkQpQeERGJEKVHREQiRD1tEZEIUU9bRCRC1NMWEYkQjR4REYkQpUdERCJE6RERkQiJeU873l9JIrLvsbTUp8J2ZZZhZqPN7Eszm2NmJ5pZdTObaGbfhJ/VQl0zs0FmNtfMPjGzpkn76Rbqf2Nm3UpyegraIhIvZqlPhXsIeMvdjwCOAeYAfYFJ7t4YmBSWAf4MNA5TFvB4ojlWHegPnAA0B/pvDfTFoaAtIvGSnp76tAtmVhU4BXgawN03uvsaoD0wPFQbDnQI8+2BEZ4wFcgws7pAa2Ciu69y99XARKBNcU9PQVtE4qUIPW0zyzKzmUlTVtKeGgLLgaFm9rGZPWVmlYA67r441FkC1Anz9YCFSdvnhLKCyotFFyJFJF6KMHrE3bOB7AJWlwOaAle7+zQze4j/pUK2bu9m5sVtanGopy0i8VJ6Oe0cIMfdp4Xl0SSC+NKQ9iD8XBbWLwLqJ22fGcoKKi8WBW0RiZdSGj3i7kuAhWZ2eChqCXwBjAW2jgDpBowJ82OBrmEUSQtgbUijTABamVm1cAGyVSgrFqVHRCRe0ko1rF0NPGdmFYDvgB4kOrujzOxSYD5wfqg7HmgLzAVyQ13cfZWZ3QbMCPUGuvuq4jbI3Ms4HZO7YrfmeyQifMueboHsjSrVLvGdMVte75NyzElrd3/k7sRRT1tE4kW3sYuIRIiCtohIhMT82SMK2iISL+ppi4hESOmOHtnrxPvsRGTfo/SIiEiEKD0iIhIhCtoiIhGSpqAtIhIdehu7iEiEKD0iIhIhGj0iIhIh6mmLiESIgraISIQoaIuIRIhGj4iIRIh62iIiEaLRIyIiEaKetiS7acAdvPv+h9SoXo03Rj9b4v29OnY8jz81HIArL+vG2We15aeffuaaG/uxIGcR6Wlp/OGUk7n+mitLfCwpucVLlnLj3/7BypWrMDPOP+csul143nZ1nhr+PK+/ORGAzZs38+28+UyZ9DoZVasU+7gbN27kxlv/wedzviIjowoP3PV3Mg+qy4dTZ/DPQU+wKS+P8uXKccNfruLE5s1KdI6RV8pB28zSgZnAInc/08waAiOBGsAsoIu7bzSzXwEjgGbASuACd/8+7OMm4FJgM9Db3Yv9NvZ4fyWVgXPateWpR+8v8nZdLutFzg+Ltytbs3Ydj2QPZdQzg3np2cE8kj2UtevWAXBJ18689eoLvDpyGB/99xPe+2BKqbRfSiY9PZ2+1/Zk/MvP8uLwJ3l+1CvM/W7ednUu63YhY0YOZczIofTp9f84vumxKQfsnB8W0+Xyq3cqf+m1cVSpcgATx46k+0Xnc99DTwBQLaMqjz90N6+PGs5dA2/hxltvL/lJRp2lpT6l5hpgTtLy3cAD7n4osJpEMCb8XB3KHwj1MLMmQCfgSKAN8Fj4IigWBe0iOr7ZsVTd4T/ggoU5XNqzD+dceAkXXnIl386bn9K+PvjPNE5qcTwZVatQtUoVTmpxPP/+cBr7778fLY5P9JYqlC9PkyMOZ+my5aV+LlJ0tWvV5MjfHA5A5UoVadSwAUuXrSiw/rgJ/+LMNi23LY8ZN4GOXbJo36kHf7v9XjZv3pzScd9599+cfWYbAFq3PI0pM2bh7jQ54jDq1KoJQONDGvLLL7+wcePG4p5ePKSnpz4VwswygTOAp8KyAacDo0OV4UCHMN8+LBPWtwz12wMj3f0Xd58HzAWaF/f0FLRLwa2338OtN17LK88P4a/X9uLvd96X0nZLly/nwDq1ty3XqV2Lpcu3D87r1q9n8vsf6k/evVDOD4uZ89XXHHNUk3zX//TTz/z7P9No1fI0AL797nvefPsdXhjyGGNGDiUtPW1bGqUwS5evoO6Bic9KuXLlOKByJVavWbtdnQmT3qXJEYdRoUKF4p9UHBShp21mWWY2M2nK2mFvDwI3AlvCcg1gjbvnheUcoF6YrwcsBAjr14b628rz2abIUsppm9l6wHcoXksiz3Odu3+3Q/0sIAvgyYf/SdYlXYvbvr3ehtxcPv7vp1xzY79tZRs3bQLg5THjGPH8KAAWLFxEVq/rKV++HJn1DuLR++8sdN95eXn06TuALp07Uj+z2L9jKQMbcnPpfX0/br6uN5UrV8q3zuT3P6TpMb/dlhqZMn0Wn835io5dLgfg519+oUa1agD0vO5mchYtZtOmTSxesoz2nXoA0LVzR85tf0ah7fnm23ncN+gJhhQjdRc7Rchpu3s2kJ3vbszOBJa5+ywzO61U2lYKUr0Q+SCJb4fnASORnzkE+AgYApyWXHm7f4jcFTsG+1jxLU6VAw5gzIvDd1p3bvsztv2H63JZL+4ceAuZB9Xdtr5OrVpMn/XxtuWly5bTvNnvti3fevs9NPh1Jt0vuqAMz0CKatOmPHpf3492bf9Eq5anFlhv3NuTOKPNH7ctO87Z7dpw3dVX7FT30X/eASR67zf1v4NnBj+83fo6tWqyeMkyDqxTm7y8PNb/uIFqGVUBWLJ0Gb2uu5m7B97Cr+vry70Uh/ydBJxlZm2B/YAqwENAhpmVC73pTGBRqL8IqA/kmFk5oCqJC5Jby7dK3qbIUv1KOsvdn3T39e6+LgTl1u7+IlCtuAePg8qVK5F5UF3enPgOAO7Ol199k9K2J//+BD6YMp2169axdt06PpgynZN/fwIADzyazY/rf+TmG64ps7ZL0bk7twy8i0YNG9Dj4k4F1lu//kdmzJpNy9NO3lZ2YvNmTPjXe6xctRpIXIhe9MOSlI57+qkn8+obbwGJNEiL45tiZqxbv56s3jdy3dVX0OzYo0twZnFiRZgK5u43uXumuzcg0VF9x90vAiYDHUO1bsCYMD82LBPWv+PuHso7mdmvwsiTxsD04p5dqj3tXDM7n/8l3zsCP4f5WPekd9Snb3+mz/qY1WvWcErrDlx9xaXce0d/BtxxH48PHk5eXh5tW7fkiMMbF7qvjKpVuOry7nS8+DIAemb1IKNqFZYsXcYTTw2nUcODObtz4s/kiy84l/POOatMz00KN2v2p4wZN4HDDm20LYXRp1cWPyxZCkDnjh0AmDj5fU5qcTwV999/27aHNmrIX666jEuu6sOWLVsoX64cf+vbh3oHHVjocTt2OIMbbr2dP53ViapVq/DAnQMAePbFV1iwcBGPDh7Go4OHATDksfupUX0f7kuV/W3sfwVGmtntwMfA06H8aeAZM5sLrCIR6HH3z81sFPAFkAf0dPfUrkDnwxJfBIVUMmtE4s+CE0kE6anAtSS6+M3c/YMCN455ekSKybcUXkf2PZVqlzi3seWzUSnHnLSjzo/c7ZMp9bTDhcZ2BawuOGCLiOxuMb+NPaWctpkdZmaTzOyzsHy0mfUrbDsRkd3OLPUpglK9EDkYuAnYBODunxDyNSIie5e0IkzRk+qFyIruPt22/2bKK6iyiMgeE9EedKpSDdorzOwQwkgRM+sILN71JiIie0DxH+sRCakG7Z4kbpY5wswWAfOAi8qsVSIixaWeNpAY2jeUxKDy6sA6EoPIB5ZRu0REikfP0wYSd/ysIXHb+g9l1hoRkRJTTxsg093blGlLRERKQ8zTI6n+HfEfM/ttmbZERKQ0lP5LEPYqqfa0Twa6m9k84BcSf3+4u+sJNSKyVynBS2EiIdWg/ecybYWISGmJeXok1WePpPb+LBGRPU1BW0QkSqKZq06VgraIxIt62iIiEaILkSIiEaKetohIhER0/HWqFLRFJGbU0xYRiY6Yp0fi/XeEiOx7Suk2djOrb2aTzewLM/vczK4J5dXNbKKZfRN+VgvlZmaDzGyumX1iZk2T9tUt1P/GzLqV5PQUtEUkXkrv2SN5wHXu3gRoAfQ0syZAX2CSuzcGJoVlSNw53jhMWcDjkAjyQH/gBKA50H9roC8OBW0RiZdSCtruvtjdPwrz64E5QD2gPTA8VBsOdAjz7YERnjAVyDCzukBrYKK7r3L31cBEoNhPTVXQFpGYsZQnM8sys5lJU1a+ezRrAPwOmAbUcfetr1tcAtQJ8/WAhUmb5YSygsqLRRciRSReinAh0t2zSbxKcRe7s8rAy8Bf3H1d8gvO3d3NzIvZ0mJRT1tEYib1nnahezIrTyJgP+fur4TipSHtQfi5LJQvAuonbZ4ZygoqLxYFbRGJF7PUp13uxgx4Gpjj7vcnrRpL4h25hJ9jksq7hlEkLYC1IY0yAWhlZtXCBchWoaxYlB4RkXgpvXHaJwFdgE/NbHYouxm4CxhlZpcC84Hzw7rxQFtgLpAL9ABw91VmdhswI9Qb6O6ritsocy/jdEzuit2a75GI8C17ugWyN6pUu8QR15d9nnLMsdpHRu5OHPW0RSReYn5HpIK2iMSMgraISHSopy0iEiF6NKuISJSopy0iEh1Kj4iIREm8g3a8kz8iIjGjnraIxIopPSIiEiEaPSIiEiXqaYuIRIfSIyIiUaKgLSISHeppi4hEiYK2iEh0aPSIiEiExLujraAtInET76itoC0i8aILkSIiUaKgLSISHTG/EFn2b2OXbcwsy92z93Q7ZO+iz4UURby/kvY+WXu6AbJX0udCUqagLSISIQraIiIRoqC9eylvKfnR50JSpguRIiIRop62iEiEKGiLiESIbq4pBWZWA5gUFg8ENgPLw3Jzd9+4Rxome4SZbQY+TSrq4O7fF1D3R3evvFsaJrGgnHYpM7MBwI/ufl9SWTl3z9tzrZLdqSiBWEFbikrpkTJiZsPM7AkzmwbcY2YDzOz6pPWfmVmDMH+xmU03s9lm9qSZpe+pdkvpM7PKZjbJzD4ys0/NrH0+deqa2fvhM/CZmf1fKG9lZlPCti+ZmQL8Pk5Bu2xlAr939z4FVTCz3wAXACe5+7EkUisX7Z7mSRnZPwTf2Wb2KvAzcLa7NwX+APzTbKdH0V0ITAifgWOA2WZWE+gH/DFsOxMo8LMk+wbltMvWS+6+uZA6LYFmwIzw/3h/YFlZN0zK1E8h+AJgZuWBO8zsFGALUA+oAyxJ2mYGMCTUfc3dZ5vZqUAT4MPw2agATNk9pyB7KwXtsrUhaT6P7f+y2S/8NGC4u9+021olu9tFQC2gmbtvMrPv+d/vHwB3fz8E9TOAYWZ2P7AamOjunXd3g2XvpfTI7vM90BTAzJoCDUP5JKCjmdUO66qb2cF7pIVSVqoCy0LA/gOw0+83/M6Xuvtg4CkSn5WpwElmdmioU8nMDtuN7Za9kHrau8/LQFcz+xyYBnwN4O5fmFk/4G0zSwM2AT2B+XuspVLangNeN7NPSeSlv8ynzmnADWa2CfgR6Oruy82sO/CCmf0q1OtH+OzIvklD/kREIkTpERGRCFHQFhGJEAVtEZEIUdAWEYkQBW0RkQhR0BYRiRAFbRGRCPn/BBKB8v2Eg6UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm_cls = confusion_matrix(y_test, y_cls_predict)\n",
    "df_cm_cls = pd.DataFrame(cm_cls, index=['pos', 'neg'], columns=['True', 'False'])\n",
    "ax = sn.heatmap(df_cm_cls, cmap='Oranges', annot=True)\n",
    "cm_cls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.976"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_cls = accuracy_score(y_test, y_cls_predict)\n",
    "accuracy_cls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dağılım"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD4CAYAAADy46FuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZd0lEQVR4nO3dfYxU153m8e+TxhjsxG6MO4jQzIATnBXeWGBXHKJJLG9ebECjYK+sDGi1EI83JH6RJsp6J7CR1tlMpCSzyWTHWoeEjFnwKjH2+A0U4SGEWIm0WmyKwPDiN9rElptg6MGAV2vjF/zbP+5pc2l3H7qriu4q+vlIpb71u/ece8qU++l776lbigjMzMwG8r6RHoCZmTU3B4WZmWU5KMzMLMtBYWZmWQ4KMzPLGjPSA2i0iy++OKZNmzbSwzAzaynbtm37l4jo6G/dWRcU06ZNo1qtjvQwzMyGx/FDsG81HN0Jbx6DsRdC++VwyU0wrt/f+/2S9OJA6866oDAzGxUOb4U934U/PgYCThw/ua7tYdh5J3xoHly2HCZ+vK5d+RqFmVmreW4F/Poa6H4U3jl+akgAnHi9qHc/Wmz33Iq6ducjCjOzVvLcCth+B5x4bRAbR7Hd9juKp5feUtMufURhZtYqDm8dQkiU9IbF4dqu3zoozMxaxZ7vFqeVanHi9aJ9DRwUZmat4Pih4sI1td7INeCPG+B4z5BbOijMzFrBvtXF7KZ6SEU/Q+SgMDNrBUd3vnd201CdeB2O7hpyMweFmVkrePNYY/p568iQmzgozMxawdgLG9PPOROG3MRBYWbWCtovh7Zx9fXRNh7aPzbkZg4KM7NWcMmXap/w1Cui6GeIHBRmZq1g3AeLezfVPPVJ8KH5Q7pRYC8HhZlZq7hseXH6qBZt44v2NXBQmJm1iokfh9k/gLbzhtau7byi3cRKTbv1TQHNzFpJ7439tt+RbueRu3Ch4khi9g9qviEg+IjCzKz1XHoLfO630HkDvG/ce09HtY0v6p03FNvVERLgIwozs9Y0sQJXP1Tcu2nf6uIT128dKT4n0f6xYnZTDReu++OgMDNrZeM6YOZ/OqO78KknMzPLOm1QSFol6ZCk3aXa/ZJ2pMcLknak+jRJr5fW/aTU5kpJuyR1SbpLklL9IkmbJO1NPyekutJ2XZJ2Srqi4a/ezMxOazBHFKuBueVCRPxFRMyKiFnAQ8DDpdXP966LiK+W6iuALwMz0qO3z2XA5oiYAWxOzwHmlbZdmtqbmdkwO21QRMTvgFf6W5eOCr4I3JfrQ9Jk4IKI2BIRAdwLXJ9WLwDWpOU1fer3RmEL0J76MTOzYVTvNYpPAwcjYm+pNl3Sdkm/lfTpVJsCdJe26U41gEkRcSAtvwxMKrV5aYA2p5C0VFJVUrWnZ+jf3mRmZgOrNygWcerRxAHgTyJiNvB14BeSLhhsZ+loY8i3vYqIlRFRiYhKR0djpoOZmVmh5umxksYA/xa4srcWEW8Ab6TlbZKeBy4F9gOdpeadqQZwUNLkiDiQTi0dSvX9wNQB2piZ2TCp54jic8AzEfHuKSVJHZLa0vIlFBei96VTS69KmpOuaywG1qVm64ElaXlJn/riNPtpDnCsdIrKzMyGyWCmx94H/B/go5K6Jd2cVi3kvRexrwZ2pumyDwJfjYjeC+G3Av8AdAHPA4+l+veAz0vaSxE+30v1DcC+tP3PUnszMxtmKi4LnD0qlUpUq9WRHoaZWUuRtC0i+r29rD+ZbWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZg/kq1FWSDknaXap9S9J+STvSY35p3XJJXZKelXRdqT431bokLSvVp0t6ItXvlzQ21c9Nz7vS+mkNe9VmZjZogzmiWA3M7af+o4iYlR4bACTNpPgu7ctSmx9LapPUBtwNzANmAovStgDfT319BDgC9H4n983AkVT/UdrOzMyG2WmDIiJ+B7wyyP4WAGsj4o2I+APQBVyVHl0RsS8i3gTWAgskCfgM8GBqvwa4vtTXmrT8IPDZtL2ZmQ2jeq5R3C5pZzo1NSHVpgAvlbbpTrWB6hOBoxHxdp/6KX2l9cfS9u8haamkqqRqT09PHS/JzMz6qjUoVgAfBmYBB4AfNmpAtYiIlRFRiYhKR0fHSA7FzOysU1NQRMTBiDgREe8AP6M4tQSwH5ha2rQz1QaqHwbaJY3pUz+lr7T+wrS9mZkNo5qCQtLk0tMbgN4ZUeuBhWnG0nRgBvAksBWYkWY4jaW44L0+IgJ4HLgxtV8CrCv1tSQt3wj8Jm1vZmbDaMzpNpB0H3ANcLGkbuBO4BpJs4AAXgC+AhAReyQ9ADwFvA3cFhEnUj+3AxuBNmBVROxJu/gGsFbSd4DtwD2pfg/wvyR1UVxMX1jvizUzs6HT2fZHeqVSiWq1OtLDMDNrKZK2RUSlv3X+ZLaZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMws67RBIWmVpEOSdpdq/03SM5J2SnpEUnuqT5P0uqQd6fGTUpsrJe2S1CXpLklK9YskbZK0N/2ckOpK23Wl/VzR8FdvZmanNZgjitXA3D61TcC/jojLgeeA5aV1z0fErPT4aqm+AvgyMCM9evtcBmyOiBnA5vQcYF5p26WpvZmZDbPTBkVE/A54pU/tVxHxdnq6BejM9SFpMnBBRGyJiADuBa5PqxcAa9Lymj71e6OwBWhP/ZiZ2TBqxDWKvwQeKz2fLmm7pN9K+nSqTQG6S9t0pxrApIg4kJZfBiaV2rw0QJtTSFoqqSqp2tPTU8dLMTOzvuoKCknfBN4Gfp5KB4A/iYjZwNeBX0i6YLD9paONGOo4ImJlRFQiotLR0THU5mZmljGm1oaSvgT8OfDZ9AueiHgDeCMtb5P0PHApsJ9TT091phrAQUmTI+JAOrV0KNX3A1MHaGNmZsOkpiMKSXOBvwa+EBGvleodktrS8iUUF6L3pVNLr0qak2Y7LQbWpWbrgSVpeUmf+uI0+2kOcKx0isrMzIbJaY8oJN0HXANcLKkbuJNiltO5wKY0y3VLmuF0NfBtSW8B7wBfjYjeC+G3UsygGk9xTaP3usb3gAck3Qy8CHwx1TcA84Eu4DXgpnpeqJmZ1UbprNFZo1KpRLVaHelhmJm1FEnbIqLS3zp/MtvMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMsgYVFJJWSTokaXepdpGkTZL2pp8TUl2S7pLUJWmnpCtKbZak7fdKWlKqXylpV2pzV/pe7QH3YWZmw2ewRxSrgbl9asuAzRExA9icngPMA2akx1JgBRS/9Cm+b/sTwFXAnaVf/CuAL5fazT3NPszMbJgMKigi4nfAK33KC4A1aXkNcH2pfm8UtgDtkiYD1wGbIuKViDgCbALmpnUXRMSWKL7A+94+ffW3DzMzGyb1XKOYFBEH0vLLwKS0PAV4qbRdd6rl6t391HP7OIWkpZKqkqo9PT01vhwzM+tPQy5mpyOBaERftewjIlZGRCUiKh0dHWdyGGZmo049QXEwnTYi/TyU6vuBqaXtOlMtV+/sp57bh5mZDZN6gmI90DtzaQmwrlRfnGY/zQGOpdNHG4FrJU1IF7GvBTamda9KmpNmOy3u01d/+zAzs2EyZjAbSboPuAa4WFI3xeyl7wEPSLoZeBH4Ytp8AzAf6AJeA24CiIhXJP0NsDVt9+2I6L1AfivFzKrxwGPpQWYfZmY2TFSc+j97VCqVqFarIz0MM7OWImlbRFT6W+dPZpuZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZVs1BIemjknaUHq9K+pqkb0naX6rPL7VZLqlL0rOSrivV56Zal6Rlpfp0SU+k+v2Sxtb+Us3MrBY1B0VEPBsRsyJiFnAlxfdjP5JW/6h3XURsAJA0E1gIXAbMBX4sqU1SG3A3MA+YCSxK2wJ8P/X1EeAIcHOt4zUzs9o06tTTZ4HnI+LFzDYLgLUR8UZE/AHoAq5Kj66I2BcRbwJrgQWSBHwGeDC1XwNc36DxmpnZIDUqKBYC95We3y5pp6RVkiak2hTgpdI23ak2UH0icDQi3u5TNzOzYVR3UKTrBl8A/jGVVgAfBmYBB4Af1ruPQYxhqaSqpGpPT8+Z3p2Z2ajSiCOKecDvI+IgQEQcjIgTEfEO8DOKU0sA+4GppXadqTZQ/TDQLmlMn/p7RMTKiKhERKWjo6MBL8nMzHo1IigWUTrtJGlyad0NwO60vB5YKOlcSdOBGcCTwFZgRprhNJbiNNb6iAjgceDG1H4JsK4B4zUzsyEYc/pNBibpfODzwFdK5b+VNAsI4IXedRGxR9IDwFPA28BtEXEi9XM7sBFoA1ZFxJ7U1zeAtZK+A2wH7qlnvGZmNnQq/nA/e1QqlahWqyM9DDOzliJpW0RU+lvnT2abmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMsuoOCkkvSNolaYekaqpdJGmTpL3p54RUl6S7JHVJ2inpilI/S9L2eyUtKdWvTP13pbaqd8xmZjZ4jTqi+DcRMav0favLgM0RMQPYnJ4DzANmpMdSYAUUwQLcCXwCuAq4szdc0jZfLrWb26Axm5nZIJypU08LgDVpeQ1wfal+bxS2AO2SJgPXAZsi4pWIOAJsAuamdRdExJaICODeUl9mZjYMGhEUAfxK0jZJS1NtUkQcSMsvA5PS8hTgpVLb7lTL1bv7qZ9C0lJJVUnVnp6eel+PmZmVjGlAH5+KiP2SPghskvRMeWVEhKRowH4GFBErgZUAlUrljO7LzGy0qfuIIiL2p5+HgEcorjEcTKeNSD8Ppc33A1NLzTtTLVfv7KduZmbDpK6gkHS+pA/0LgPXAruB9UDvzKUlwLq0vB5YnGY/zQGOpVNUG4FrJU1IF7GvBTamda9KmpNmOy0u9WVmZsOg3lNPk4BH0ozVMcAvIuKfJG0FHpB0M/Ai8MW0/QZgPtAFvAbcBBARr0j6G2Br2u7bEfFKWr4VWA2MBx5LDzMzGyYqJhOdPSqVSlSr1ZEehplZS5G0rfQRh1P4k9lmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmllVzUEiaKulxSU9J2iPpr1L9W5L2S9qRHvNLbZZL6pL0rKTrSvW5qdYlaVmpPl3SE6l+v6SxtY7XzMxqU88RxdvAf4yImcAc4DZJM9O6H0XErPTYAJDWLQQuA+YCP5bUJqkNuBuYB8wEFpX6+X7q6yPAEeDmOsZrZmY1qDkoIuJARPw+Lf9f4GlgSqbJAmBtRLwREX8AuoCr0qMrIvZFxJvAWmCBJAGfAR5M7dcA19c6XjMzq01DrlFImgbMBp5Ipdsl7ZS0StKEVJsCvFRq1p1qA9UnAkcj4u0+9f72v1RSVVK1p6enES/JzMySuoNC0vuBh4CvRcSrwArgw8As4ADww3r3cToRsTIiKhFR6ejoONO7MzMbVcbU01jSORQh8fOIeBggIg6W1v8M+GV6uh+YWmremWoMUD8MtEsak44qytubmdkwqWfWk4B7gKcj4u9K9cmlzW4Adqfl9cBCSedKmg7MAJ4EtgIz0gynsRQXvNdHRACPAzem9kuAdbWO18zMalPPEcWfAf8e2CVpR6r9Z4pZS7OAAF4AvgIQEXskPQA8RTFj6raIOAEg6XZgI9AGrIqIPam/bwBrJX0H2E4RTGZmNoxU/OF+9qhUKlGtVkd6GGZmLUXStoio9LfOn8w2M7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZll1XX32LPC8UOwbzUc3QlvHoOxF0L75XDJTTDOtyw3Mxu9QXF4K+z5LvzxMRBw4vjJdW0Pw8474UPz4LLlMPHjIzZMM7ORNjpPPT23An59DXQ/Cu8cPzUkAE68XtS7Hy22e27F8I/RzKxJjL4jiudWwPY74MRrg9g4iu2231E8vfSWMzo0M7NmNLqOKA5vHUJIlPSGxWHfvtzMRp/RFRR7vlucVqrFideL9mZmo8zoCYrjh4oL19T6RU0Bf9wAx3saOSozs6Y3eoJi3+pidlM9pKIfM7NRpOmDQtJcSc9K6pK0rOaOju587+ymoTrxOhzdVV8fZmYtpqmDQlIbcDcwD5gJLJI0s6bO3jzWmEG9daQx/ZiZtYimDgrgKqArIvZFxJvAWmBBTT2NvbAxIzpnQmP6MTNrEc0eFFOAl0rPu1PtFJKWSqpKqvb0DHCxuf1yaBtX32jaxkP7x+rrw8ysxTR7UAxKRKyMiEpEVDo6Brg/0yVfqn3C08kdFf2YmY0izR4U+4GppeedqTZ04z5Y3Lup5qlPgg/N940CzWzUafag2ArMkDRd0lhgIbC+5t4uW16cPqpF2/iivZnZKNPUQRERbwO3AxuBp4EHImJPzR1O/DjM/gG0nTe0dm3nFe0mVmretZlZq2r6mwJGxAZgQ8M67L2x3/Y70u08chcuVBxJzP6BbwhoZqNWUx9RnDGX3gKf+y103gDvG/fe01Ft44t65w3Fdg4JMxvFmv6I4oyZWIGrHyru3bRvdfGJ67eOFJ+TaP9YMbvJF67NzFBEvXNGm4ukHuDFkR7HIF0M/MtID2KIWnHM0Jrj9piHTyuOu9Fj/tOI6Pev47MuKFqJpGpEtNQV8lYcM7TmuD3m4dOK4x7OMY/OaxRmZjZoDgozM8tyUIyslSM9gBq04pihNcftMQ+fVhz3sI3Z1yjMzCzLRxRmZpbloDAzs7yI8GOID2AVcAjYXapdBGwC9qafE1JdwF1AF7ATuKLUZknafi+wpFS/EtiV2tzFyVOE/e5jkGOeCjwOPAXsAf6qRcY9DngS+Oc07v+a6tOBJ9K+7gfGpvq56XlXWj+t1NfyVH8WuK5Un5tqXcCyUr3ffQxh7G3AduCXrTBm4IX077cDqLbI+6MdeBB4huJ+cJ9sgTF/NP037n28Cnytmcc94r90W/EBXA1cwalB8be9/8MCy4Dvp+X5wGPpH3sO8ETpH2xf+jkhLfe+MZ5M2yq1nZfbxyDHPLn3DQZ8AHiO4utlm33cAt6fls+h+CU4B3gAWJjqPwFuScu3Aj9JywuB+9PyTIqwOZfil+nzFL/I29LyJcDYtM3M1KbffQxh7F8HfsHJoGjqMVMExcV9as3+/lgD/Ie0PJYiOJp6zH3G3wa8DPxpM497xH/ptuoDmMapQfEsMDktTwaeTcs/BRb13Q5YBPy0VP9pqk0GninV391uoH3UOP51wOdbadzAecDvgU9QfCJ1TKp/EtiYljcCn0zLY9J2ovjLfHmpr42p3bttU315emigfQxyrJ3AZuAzwC9z/TXRmF/gvUHRtO8P4ELgD6S/llthzP28hmuB/93s4/Y1isaZFBEH0vLLwKS0PNDXuebq3f3Uc/sYEknTgNkUf503/bgltUnaQXG6bxPFX9NHo7gNfd99vTu+tP4YMLGG1zMxs4/B+O/AXwPvpOe5/pplzAH8StI2SUtTrZnfH9OBHuB/Stou6R8knd/kY+5rIXDfafoc8XE7KM6AKOI6mnEfkt4PPAR8LSJebUSfQ1HLPiLiRETMovgr/SrgX52BoTWMpD8HDkXEtpEeyxB9KiKuAOYBt0m6uryyCd8fYyhOAa+IiNnA/6M4nVJrfzWp4//FscAXgH9sVJ9DMZR9OCga56CkyQDp56FUH+jrXHP1zn7quX0MiqRzKELi5xHxcKuMu1dEHKW4IP9JoF1S792Py/t6d3xp/YXA4Rpez+HMPk7nz4AvSHoBWEtx+unvm3zMRMT+9PMQ8AhFKDfz+6Mb6I6IJ9LzBymCo5nHXDYP+H1EHDxNnyM+bgdF46ynmIFA+rmuVF+swhzgWDr02whcK2mCpAkU5yo3pnWvSpojScDiPn31t4/TSn3dAzwdEX/XQuPukNSelsdTXFd5miIwbhxg3L37uhH4TfrLaT2wUNK5kqYDMygu+PX7dbupzUD7yIqI5RHRGRHTUn+/iYh/18xjlnS+pA/0LlP8u+6mid8fEfEy8JKkj6bSZylm9TXtmPtYxMnTTrk+R37ctVyAGe2P9I97AHiL4q+amynOD2+mmHb2a+CitK2AuynOq+8CKqV+/pJi+loXcFOpXqH4n/R54H9wcmpbv/sY5Jg/RXGYuZOT0/Lmt8C4L6eYYroz9f1fUv0Sil+aXRSH7uem+rj0vCutv6TU1zfT2J4lzQJJ9fkUs8CeB75Zqve7jyG+V67h5Kynph1zavfPnJyG/M3cv10TvT9mAdX0/niUYvZPU485tT+f4gjwwlKtacftW3iYmVmWTz2ZmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZln/H7aYL/NGdDoBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(kmeans.cluster_centers_[:, 0], kmeans.cluster_centers_[:, 1], s=300, c='orange', label = 'Centroids')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  ----------------------------------Agglomerative Clustering------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Aglomeratif kümeleme, kümelerdeki nesneleri benzerliklerine göre gruplamak için kullanılan en yaygın hiyerarşik kümeleme türüdür. AGNES (Aglomerative Nesting) olarak da bilinir. Algoritma, her nesneyi tek bir küme olarak ele alarak başlar. Daha sonra, küme çiftleri, tüm kümeler tüm nesneleri içeren büyük bir kümede birleştirilene kadar art arda birleştirilir. Sonuç, dendrogram adlı nesnelerin ağaç temelli bir temsilidir."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model oluşturma ve test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, ..., 0, 0, 0], dtype=int64)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cls2 = AgglomerativeClustering(n_clusters=2, affinity='euclidean', linkage='ward')\n",
    "y2_cls_predict = cls2.fit_predict(x_test)\n",
    "y2_cls_predict"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dendrogram görünümü"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 0, 'Number of points in node (or index of point if no parenthesis).')"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEZCAYAAABy91VnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAvZElEQVR4nO3debgcVZnH8e+PBAiEJSxXkM0gKKuKElFZJCPIgCgooLITRKPOIOPCjDuLMorijA7ICFEgKggCwuCwiYwgIGtASFgElR0SuAgECBBIeOeP9zS30ul7bye3O7kpf5/nuc/t7qo6derUqbdOna4+pYjAzMzqaanFnQEzM+seB3kzsxpzkDczqzEHeTOzGnOQNzOrMQd5M7Mac5BfRCTdIWn8MMjHBEnXDDD9EkkHdXMdbSx/paSPDyUPnSBpO0l3L+58dIKk8ZIeXtz5sEXPQb4DJN0vacemz+YJdBGxWURcucgzt4AiYpeI+Gk31yFpGUlHSfqzpFml/E6VNLaD6xjSiQYgIq6OiI06laeqciJ7UdKzkp6RdLOkL0lathvrs79fDvLDnKSRC7HMiG7kpYPOBXYD9gVWBt4C3AzssDgzVbUw5b4QDo2IFYHXAl8A9gYulqRFsO5XdXpbF1HZWZsc5BeRamtf0lKl1fZXSX+TdLakVcu0sZJC0iGSHgR+Vz4/R9IMSTMlXSVps0rakyX9SNLFkmYB/yBpXUnnSeot6/hhU36+J+kpSfdJ2qXy+TxdJZI+Iemu0uK8U9LbyueN/Dc+/1Cb5bAj8F5g94i4KSLmRMTMiDgxIk5pMf9Rkk6vvG+Uz8jyfoKke0s+7pO0n6RNgJOAd0l6TtLTZd5ly3Y/KOkxSSdJWq5MGy/pYUlflDQDOK25i6Psw8MlTS374ZeSRlWm/5uk6ZIelfTxks8NByuTiJhVrvJ2A94F7FrSa6eeHFS25wlJX63kZblSL56SdCfw9qZyvb9s61RglqSRknZTdis+XerBJpX53ybpj6WczynbfswAZbeKpAtL/XuqvF6nkt6Vko6RdG3ZR/8raTVJZyivbG5SB6/s/p45yC8enwE+CGwPrAU8BZzYNM/2wCbAP5b3lwBvAF4D3AKc0TT/vsC/AysC1wEXAg8AY4G1gbMq874DuBtYHfgucIo0f+tR0oeBo4ADgZXIIPS3MvmvwHZkS/xo4HRJr21j23cEboyIh9qYd0CSRgPHA7uUFvHWwK0RcRfwKeC6iFghIsaURY4F3ghsAWxIlssRlSTXBFYFXgdM7Ge1HwF2BtYH3gxMKHnZGfh82b4NgfELuj0R8SAwhSxXaK+ebAtsRF4FHVEJzEcCG5S/fwRafc+yD3lCGQO8HjgT+CzQA1wM/K+ya20Z4HxgMlk+ZwLNJ/XmslsKOK28Xw94Afhh0zJ7AweQ+2EDst6eVtK5q2yDDVVEdOUPOBV4HLi9jXnXA64A/ghMBd7XrXx1aVvvB54Dnq78PQ9c0zTPjuX1XcAOlWmvBV4GRpJBOYDXD7C+MWWelcv7ycDPKtPfBfQCI1ssOwH4S+X98iWtNcv7K4GPl9e/Af6lzTK4lWydN9ZxTT/z/Rg4a5C0qnk4Cji9Mq1RPiOB0aWs9wSWa7Gd1fIXMAvYoKmc7iuvxwMvAaMq08cDDzftw/0r778LnFSp79+uTNuw5HPDwbax6fOzgB8vQD1ZpzL9RmDv8vpeYOfKtIkttuVjlfdfB86uvF8KeKSUwbvLa1WmXwMc01/ZtdiuLYCnmrb/q5X3/wFcUnn/AfKEvdiP7yX9r5st+clki6cdXyMr2FvJs/t/dytTXfTBiBjT+AP+aYB5XwecXy6LnyYP5rnAGpV5Xm3pShoh6dhy2f4MeYBCtsTnmx9YF3ggIub0s/4ZjRcR8Xx5uUKL+dYlW+zzkXSgpFsr27B5U3768zcyWA1ZRMwCPkq22qdLukjSxv3M3kOe0G6u5PnS8nlDb0S8OMhqZ1ReP09fua3FvPtgYa9U1gaeLK/bqSft5ueBFuuqTl+rOk9EvFKmr12mPRIl+rZYFprKTtLykk6W9ECps1cBYzTv90WPVV6/0OJ9qzppC6hrQT4irqKvsgIgaQNJlyrvJLi6ckAG2R0Aefn/aLfyNUw8RHYxjKn8jYqIRyrzVA+ofYHdya6AlclWHGTrtNX8DwHraehfgD1EXkbPQ9LryBb5ocBq5aR2e1N++nM5sFW1f3YQs8jg3LBmdWJE/CYi3kueOP5U8gXzlgfAE2Tg2KxS5itHRDWQDGVI1ulAdZvWXdAEJK0LbAlcXT5qp54MlJ9qHtZrMU91ex8lTyqNvKgs/0hJa+2mLr3m7Wsuuy+Q3UjviIiVyKsBaK+OWAct6j75ScBnImJL4HD6WuxHAfuXL7kuJvsi6+wk4N9LsERSj6TdB5h/RWA22QpeHvjWIOnfSB6Yx0oaLWmUpG0WIp8/AQ6XtKXShiXPo8mDurfk/2CyJT+oiLgc+C3ZQt2yfOG3oqRPSfpYi0VuBd4taT1JKwNfbkyQtIak3Uvf/Gyyy+yVMvkxYJ3Sn9xomf4Y+L6k15Tl15b0j3TG2cDBkjaRtDzZ/dGW0urdHriA3HcXl0kLWk+a8/Pl8gXoOgx+TJ0N7CppB0lLk0F6NnAt2Vc+Fzi07K/dga0GSW9F8qT6tPLLYvevLyaLLMhLWoH8YuwcSbcCJ9N32b4PMDki1gHeB/xcUp2/FP4v4NfAZZKeBa4nvwztz8/IS+lHgDvL/P2KiLlkn+aGwIPAw2S3xgKJiHPIL3N/ATwL/A+wakTcSfahXkcG0zcBf1iApPciA9kvgZnkVcA4spXfnIfflvmmkrdZXliZvBT5Zeej5FXj9sCny7TfAXcAMyQ9UT77IvAX4PrShXA52docsoi4hPwS+IrGOsqk2QMs9sOy/x8DfgD8iuxHb5yoFrSeVB1N1pn7gMuAnw+S/7uB/YETyKueDwAfiIiXIuIlYA/gEPI7kP3J/TDQtv0AWK6kdT3ZNWaLgebtZutw4nkL1IURsbmklYC7I2K+/lhJd5CV+6Hy/l7gnRHxeNcyZ9ZF5S6X24FlB/huZIkl6QbyS+fTFndebGCLrLUcEc8A95Xb8iiX/28pkx+k/BCmHByjKF0BZksKSR9S3ou/CvAd4H/rEuAlbS9pzdJdcxB5+6hb50uArgV5SWeSl/MbKX8ocQiwH3CIpNvIS+lG/+IXgE+Uz88EJkQ3LzHMuuOT5G3DfyX7sD898OxLlI2A28jumi8Ae0XE9MWaI2tLV7trzMxs8arzl5tmZn/3ujKQ0Oqrrx5jx47tRtJmZrV08803PxERPYPPuWC6EuTHjh3LlClTupG0mVktSWr1q+Qhc3eNmVmNOcibmdWYg7yZWY05yJuZ1ZiDvJlZjTnIm5nVmIO8mVmNOcibmdVYV34MNZBf3PAgF9zazoNtFp3dt1ibfd/R6sE5ZmZLtkXekr/g1ke4c/ozi3q1/bpz+jPD7qRjZtYpi7wlD7Dpa1fil5981+JY9Xw+evJ1izsLZmZd4z55M7Mac5A3M6uxtoK8pM9JukPS7ZLOlDSq2xkzM7OhGzTIS1obOAwYFxGbAyOAvbudMTMzG7p2u2tGAstJGgksDzzavSyZmVmnDBrkI+IR4HvAg8B0YGZEXNY8n6SJkqZImtLb29v5nJqZ2QJrp7tmFWB3YH1gLWC0pP2b54uISRExLiLG9fR0/AlWZma2ENrprtkRuC8ieiPiZeA8YOvuZsvMzDqhnSD/IPBOSctLErADcFd3s2VmZp3QTp/8DcC5wC3AtLLMpC7ny8zMOqCtYQ0i4kjgyC7nxczMOsy/eDUzqzEHeTOzGnOQNzOrMQd5M7Mac5A3M6sxB3kzsxpzkDczqzEHeTOzGnOQNzOrMQd5M7Mac5A3M6sxB3kzsxpzkDczqzEHeTOzGnOQNzOrMQd5M7Maa+dB3htJurXy94ykzy6CvJmZ2RAN+mSoiLgb2AJA0gjgEeD87mbLzMw6YUG7a3YA/hoRD3QjM2Zm1lkLGuT3Bs5sNUHSRElTJE3p7e0des7MzGzI2g7ykpYBdgPOaTU9IiZFxLiIGNfT09Op/JmZ2RAsSEt+F+CWiHisW5kxM7POWpAgvw/9dNWYmdnw1FaQlzQaeC9wXnezY2ZmnTToLZQAETELWK3LeTEzsw7zL17NzGrMQd7MrMYc5M3MasxB3sysxhzkzcxqzEHezKzGHOTNzGrMQd7MrMYc5M3MasxB3sysxhzkzcxqzEHezKzGHOTNzGrMQd7MrMYc5M3MasxB3sysxtp9MtQYSedK+pOkuyS9q9sZMzOzoWvryVDAfwGXRsRekpYBlu9inszMrEMGDfKSVgbeDUwAiIiXgJe6my0zM+uEdrpr1gd6gdMk/VHST8qDvechaaKkKZKm9Pb2djyjZma24NoJ8iOBtwE/ioi3ArOALzXPFBGTImJcRIzr6enpcDbNzGxhtBPkHwYejogbyvtzyaBvZmbD3KBBPiJmAA9J2qh8tANwZ1dzZWZmHdHu3TWfAc4od9bcCxzcvSyZmVmntBXkI+JWYFx3s2JmZp3mX7yamdWYg7yZWY05yJuZ1ZiDvJlZjTnIm5nVmIO8mVmNOcibmdWYg7yZWY05yJuZ1ZiDvJlZjTnIm5nVmIO8mVmNOcibmdWYg7yZWY05yJuZ1ZiDvJlZjbX10BBJ9wPPAnOBORHhB4iYmS0B2n38H8A/RMQTXcuJmZl1nLtrzMxqrN0gH8Blkm6WNLHVDJImSpoiaUpvb2/ncmhmZgut3SC/bUS8DdgF+GdJ726eISImRcS4iBjX09PT0UyamdnCaSvIR8Qj5f/jwPnAVt3MlJmZdcagQV7SaEkrNl4DOwG3dztjZmY2dO3cXbMGcL6kxvy/iIhLu5orMzPriEGDfETcC7xlEeTFzMw6zLdQmpnVmIO8mVmNOcibmdWYg7yZWY05yJuZ1ZiDvJlZjTnIm5nVmIO8mVmNOcibmdWYg7yZWY05yJuZ1ZiDvJlZjTnIm5nVmIO8mVmNOcibmdWYg7yZWY21HeQljZD0R0kXdjNDZmbWOQvSkv8X4K5uZcTMzDqvrSAvaR1gV+An3c2OmZl1Urst+R8A/wa80t8MkiZKmiJpSm9vbyfyZmZmQzRokJf0fuDxiLh5oPkiYlJEjIuIcT09PR3LoJmZLbx2WvLbALtJuh84C3iPpNO7miszM+uIQYN8RHw5ItaJiLHA3sDvImL/rufMzMyGzPfJm5nV2MgFmTkirgSu7EpOzMys49ySNzOrMQd5M7Mac5A3M6sxB3kzsxpzkDczqzEHeTOzGnOQNzOrMQd5M7Mac5A3M6sxB3kzsxpzkDczqzEHeTOzGnOQNzOrMQd5M7Mac5A3M6sxB3kzsxpr50HeoyTdKOk2SXdIOnpRZMzMzIaunSdDzQbeExHPSVoauEbSJRFxfZfzZmZmQzRokI+IAJ4rb5cuf9HNTJmZWWe01ScvaYSkW4HHgd9GxA0t5pkoaYqkKb29vR3OppmZLYy2gnxEzI2ILYB1gK0kbd5inkkRMS4ixvX09HQ4m2ZmtjAW6O6aiHgauALYuSu5MTOzjmrn7poeSWPK6+WA9wJ/6nK+zMysA9q5u+a1wE8ljSBPCmdHxIXdzZaZmXVCO3fXTAXeugjyYmZmHeZfvJqZ1ZiDvJlZjbXTJz88TTkNpp079HRm7J7/TztmaOm8aS8Yd/DQ82Nm1kFLbpCfdi7MmAZrvmlIyfxyvQuGnpcZ0/K/g7yZDTNLbpCHDPAHX7S4cwGn7bq4c2Bm1pL75M3MasxB3sysxhzkzcxqzEHezKzGHOTNzGrMQd7MrMYc5M3MasxB3sysxhzkzcxqzEHezKzGHOTNzGps0LFrJK0L/AxYAwhgUkT8V7cztsh0YjTLGVPzfyfGsPFolmbWQe205OcAX4iITYF3Av8sadPuZmsRaoxmORRrvjn/hmrGtM4Mn2xmVrTz+L/pwPTy+llJdwFrA3d2OW+LjkezNLOaWqA+eUljyee93tBi2kRJUyRN6e3t7VD2zMxsKNoO8pJWAH4FfDYinmmeHhGTImJcRIzr6enpZB7NzGwhtRXkJS1NBvgzIuK87mbJzMw6ZdAgL0nAKcBdEfGf3c+SmZl1SjuP/9sGOACYJunW8tlXIuLiruVqSdOxh4p36FZM34ZpZkU7d9dcA2gR5GXJ1aGHinfsNkxwkDczYEl/kPdw4tswzWwY8rAGZmY15iBvZlZjDvJmZjXmIG9mVmMO8mZmNeYgb2ZWYw7yZmY15vvkh5NO/HL2kSkw92X49rpDz08nfpwF/gWu2WLklvxw0okHmIxYpjN56RQ/CMVssXJLfrgZ6i9nG794HQ6/vgX/AtdsMXNL3sysxtySt9Y6NbJmp74j8PcDZgvFQd5a69TImiOWySA/FHNf6huGeShmPwMPXge/PWLoaXXipOMTji0CDvLWv06MrNmJ7whO27UzJ5wZU+GlWUNLAzpz0pn9TKbRiaslnyxsAA7ytmQYLiecRjqdOOl0gp8fYIMYNMhLOhV4P/B4RGze/SyZLQH8/ABbQrRzd81kYOcu58PMzLpg0CAfEVcBTy6CvJiZWYd17D55SRMlTZE0pbe3t1PJmpnZEHTsi9eImARMAhg3blx0Kl2zWhpuv0No8O8Rasd315gtDsPtdwiNNDr1e4RO3R7a4JPGQnOQN1tchsttoZ26JfTZ6TCrF5ZdKd934oQBrU8ajXUN1d/BlUs7t1CeCYwHVpf0MHBkRJzS7YyZ2SLUqRPOrCcWze8HZvXmD9uWGd3d9bRzMmn3ymUxnQgGDfIRsc+iyIiZ1UAnThbtfl+xzOiBW+KdCKqdOnEtxh+tubvGzIaXdr6vGKybpZNBtZPdaouBg7yZDT+deq6CeTx5M7M6c5A3M6sxB3kzsxpzkDczqzEHeTOzGnOQNzOrMQd5M7Mac5A3M6sxB3kzsxpzkDczqzEHeTOzGnOQNzOrMQd5M7Mac5A3M6uxtoK8pJ0l3S3pL5K+1O1MmZlZZwwa5CWNAE4EdgE2BfaRtGm3M2ZmZkPXTkt+K+AvEXFvRLwEnAXs3t1smZlZJygiBp5B2gvYOSI+Xt4fALwjIg5tmm8iMLG83Qi4u/PZNTOrrddFRE+nE+3Y4/8iYhIwqVPpmZnZ0LXTXfMIsG7l/TrlMzMzG+baCfI3AW+QtL6kZYC9gV93N1tmZtYJg3bXRMQcSYcCvwFGAKdGxB1dz5mZmQ3ZoF+8mpnZksu/eDUzqzEHeTOzGnOQNzOrkPRtSZ9d3PkYiKQ1JN0ladlBZ46IIf0BY4GLgaeAGcAPyS90twOea/oLYM+y3ARgbtP08U3pXgE8D/wJ2BF4A/AicHqZZ1fgGuDpsu6Xm9KbC5xQSXN54L+BJ4CZwFWVaUdVlp9b8jqrvL+78v5F4JUy7/8Aq5blDwWmlM+fLvP+tZTDDmUbZpftmQVcCqwFLAucCjxTtuFXJZ3ZwLllvY3teQHoLWlcAdzTYnsb636qrH82MBl4D3BLWc+jwPRKOuPIO6YeLet7qaQ3u6TZSGNspRwaZfQCcDk55EV1W54HngWeBKYBt1XS2QT4XdkHjwAPV/JyWNM2vVhZz8PARyr7LJr+nq7Ui/tKHl8p+fhai7q7akn/pZLne4CPk/XkzLIvo/y/HNi0suyhpQyjpPEdQMA7gd+Wdc4pfzdVly3LLwPcBTxc+Wx14A9l30XZ19uUafuWNKOk2dg/z5btOL/slweAfStpCvgq8CB9deSXLcri1JL2hpXt+1ulPtwDfLxM+zjwl5LepcBalXQuYf46+VLJ562lHvSW8n4aeKysY3xTfu6gr469XNKYVsrtXOD+Mu1fyv9jWhzHjb/XV9KdBNxblrm28vlnyDozu0x7howVywN30lfHXqimWdY3p5LXucBOlXQ/ANxelrmWeeuQgGPIY2AmcCWwWWX694A/l7L7E3BgUxn9N/CZQWN0B4L8xeSBOwpYs+yIw1rMN75kdnQlyF8zQLrXAf8JLAfsWSrEFcDV9AX5fYGdy45YpVSwk8q0FUrBvruS5unksAw95J1CWzYF+Ua6V1IqdFNA2blsw7tL+r8AzirT9wCOLNMvIq+S1gY2LzvwSOBx8mC6AfgR8Hvg22WbViGD31PA0WV6I8iPJAPATODDpayPA66v5G/TUsG+V+Y/EPhUSeenZdlPlnQaJ4y3l3RuBv4JeFdZ3wmVbfpgSWMyfUF+tfJapRwPA6ZWtuVIslI+TlbyK8v+bOTlHuDzwGvoC+Sb97NNjZPSyLLeDZr2yZdb1J19gZ8AW5ftvarsl72b5vsxeUL9Q3m/MXmivZg82b6tbN/bG9tYWfYk8uD8GXA2GQg+RY7x9JmS7x2A08r2/gUYWVn+qyVf1SA/ivy1+GWlHK8kT5JvpkW9K/vkVPKE9MsybduyrzcraR5U9sW6wP+RgfuvTeWwLVkXq0F+j7IdJ5f1NMpmYtmvm5EB90fA7/s5hkeTwfg/yePh/eSJaIOy7H+SJ/fHmD/IN9e9K4EjynKfLXmeXsr1euYN8qcPEFf+uezzZylBHnhHyddJ5AngerIheD3wTfJE9puyvuZ8Hk+egLYl6+iXG/uabJQ+02paWfYjZN1+PVnPvg3cUkn76FLuS5U8PgVsXZm+DXD7ogjydwHvq7w/Dji5xXynAadV3k+gnyAPvJE8o67YtJ4pA+3EUjGmVSr3vfTdQbRxKfCV+ln21XTpP8j/CPhF5bMNyg5esby/FvhfYHJlnonl8++RA72NJgPsdiXNx5j3zP9N8gA+hnmD/ETmbXk00tm4vL+8lNnopnwfU9IL8mTYyM9NwD7VdMp6Xg3yTWlMpi/IV4PVSPLAeb5U2J1KOX23si27kldDx5BXP8+RJ4hGXi4r8zZv0y/IIHFkP/usZZDvp148wbxXdVuTJ56DKfWQDLCPl21ZqdU2Vj67tuS/UTaHUE5QZCv4ovL6bWRAeQHYoXy2Plmfd6ES5Mu0vcmTxtFkoyaAH9C63j0L/GN5/cbK9J8Dx5bX5wL/Wkn3FLIxsHxl2/5InkheDfIt9v1GZJD7NXBiZfpaZbkNWpT72LKusZXPptJ3Nb8VeYKfL8g3rf/c5nTKtJnkCW4y7Qf5Rjk8SF+Q/yhwIxlfppV9GWTD56FGmuSVZHOQvxh4pPJ+qca+rtaD5mnl/ReBsyvTNwNeHCDvvwa+0FQvnyeHQ+i3/neiT/4HwN6Slpe0NllxL63OIGk0sBfZiqt6q6QnJN0j6euSGvftbwbcGxHPluVXIq8Spg6Sl3eTl3mQQf5nUUqDrFAPAEeXdU6TtGfT8h+Q9CTZcvt+me8PksaX6RPKPOdJGhsRf6UcYGW0znFkoNpD0sOSfkgePLc1iiIiGt04G5fPXlOZTnm9WVO+HgD+A1hV0uoAlXQa825FttiulzRd0umSVi3TXiQPhoPJFvMM4HVkcGtOB+AASU9KurlFGQE8ULZvdkn7BLJV9tqS/1PIVsaDZX37kVdZzTYr8wvYvEVetgXWI0c+bd6mhmMkvSLpb5L2brEOyHoxglI3yr76IXkQBrCBpEa34PPkZXujnjS6q04AvtUi7w3N+02Vdd/R2Mby2QnAV8gDvm+BrOffAN4EfI28+v0JeVJ4dV2l3r1CBrnHgTkRcc8AeRlV0v18eb8U2coE+BzZbdnfsbUzsD9ZNtPJslFleuP15szvQODqiLi/bN8aZAPuUEkvkle0V5LdKwPZoJpOSet1ZKPlZy3m/0Cpv3dI+nRlmUb5fr5p/kvI+jGaPC42JruWZpK/8F+WvCJdCzitmmbxmsb6yKu56r5uLqvqtLPIuvdGSUuTMWue2FnJ+3JkXHr1N0oRMYe8MnhLq2UaOhHkryIr1DPkmW4K2VqrarSkft+03OZkkNuTbFX+a5m2AlnADd8kWxv95lfSe8lCOqJUgO2Z96SyDn1dJ2uRB/hPJW1Spp9Ndpf0kP18s8gDYBLZOt+HbPkdQbZYLywnpZnAisAawNJkBbkE2AJ4K32Xz5cCH5HUuPTelwwwNG1rIz3IAPp2suI1fmV8RvO8kpav5GFP8gBejgwmDWeWvB9KXgZ/NSIearFOyODyGuDrZCtpnfL5E5X8bFnK5cqS5p8raf2ZbAFNIuvGJuTB1Zj+OLmvVyS7qbYnD9jmvKxFnkR362ebPkXurzXJ+nGGpDdWpjfqxSfJE9tp5ePDgBsi4uby/q9lnduRXSub0ldP3kMG42PKOhqa6+hMYAVJIq+qtpf0MbLMbye7GZaX9CFgREScz/y+CZwSEZuQl+7XlL/mdTVcXqY90/R5tQwvJYPaeWS927Z8vrykdUvZHNEi7YZLyTq3XUnjYko9LoHnCPquEpsdSNYfShA7A/hpRPxDyd/7yKu4wWzQSKfieHK7X2j6vHocf4KMB/uUaY3yfbhpmWfJ7rk3lnx9nOxaOaxMv6Sk+Sh5RV5N80TyJLUX8Gnyu5llyPJo1IPxZbSAr1SmQZ40ryGvcl8gu2I/108ZnESevH/TIu9j+lkGGGKQl7QUWQnOI8+Cq5MH7XeaZm1uVRM5dPF9EfFKREwjg8BeZfJzwEplHVuQX7reVTaoVT7eSV7a71VaNAeQrdT7KrO9QO6MYyLipYj4PXk5vFPJz50R8WhEzI2In5BXKB+MiJ+SX4b1lPUvTZ4E1id3/Er0XY5DnuReiIgnyNbtWuSl/+VkX/WvyBb/I5XtWamSz5Uqn8+JiCnljP0EeZLZSdKKTfPuQelOioh7IuI5stX5vjLfymSr4UAySJ4F/JukXVusE/ILzDkRcTF5YG5cyui5Rn4i4jEyuO9AXsp+v5LWiWTrZwLZ8jiPvpZ8kCeZXckT59bkgflwZflGXgRc0s82EREnR8STEfE42W0RZZ25cNaL88jvc94bEbMlrUUevF+tbC9lv19D1uO5zF9PngR+Juk1ZZFX62gl389F+hN5KX4yWV9mkX32vWQ31mE0qdTzRjnOIa/gvkQepytV5l2vlO95LfLRXIY3k/V+b3JfNI6Jh8k6/o2IaHUCaSqeuIY82W9EXz2+v/w9S9/+a+RxW/Lke26JEz8nT9iHlgRfjohLyONv1ADrXpc8uZ9bSfsDZDBuDvDNx/G1wH8Be7Uo36pDyKvcLUqaAi4kv8N4mYwlj5Z572ikWdZ3EXlcHU9++T21UR6lHhxEXjVOJ+PjnZWyOoJsNK1byuBo4Hel0fYqSceRDdSPVGNosSJZv/s11Jb8quTl9A8jYnZE/I1sLb16IJbWwnhaX1ZVBX2XNncAry/BbDzZt/cJsjAPB/aUdEtJ/61kK/djEfF/ZfkDmb9rqNXlaHOB9Zefxus76Ls0Cvou5e6JiKfInRdNabzYWCYiTiQr0stkYBxJtjCrl1tvoXJJVnEHfZfgS5UusA3K5weRLefmdTeMKXn8TZn/deSXw7s0pdNfOfSnMW0keSD2lvxvQba8NiBbsSeQ3UnLAUTE1IjYnjxZPkR+8XRjNS+Stqlsdzt5aVgKXq0Xl5FdLdtUWm9bkd1Kd0qaQR6wW0maUbpxZjLvJXZjvSJbYGtX8tVyv5UrycPJOx9WJIPiWPJEPRa4uqz7POC15fUeZdqD5f3h5FXZ+uSXpdV1HVbydAV55TFS0hsq06t1aHvyxDWK3E/bl88vIE/Qx5Vtn1E+v07Svs2FWowk+95PjIg3RMQaZLAfSe7nqoPK9s0iu+/WIPvim7tmRjLw8CpvBh4oJ/mGHciG0mvLOj4KfFbSBS2Wb+y78cxbvmsD40oc2QK4sDQ2ryW/f7mdjG83R8Qr/aSZbyLOjYjNI2I1ss6NJr/zap7WqAc3lUW3IO90erg0nCaTjeRXH8ok6WiyC3yniJjniq30JGzIvN2GLUpgkC+tBvsjv9z8ErmjxpBns+qXRF+hcqti5fNdgDXK641LoR5ZmX49eWm0CnmWnUkGue+RZ/Ue8uz2GPDRynJbkxVrxab1LU32X3295HUb8ozb+JJv97KuMeRB9EhZ735kP+1uZF/pM2TL8x6yldu4u2Yk8O/kJd3ZZCvmarIvfSbZknobeZVzC9nN8S3gWLIba5VSDtPJ/r9vk5fGbyp5fyPZErqdPGC/U8poHbLV90WylfZ6MhidU/L37bJPniNbMo0rkunkF0uNdEaRlTPIbqbl6Lub6AyyJbYdWQF3Ig+as8lut+PLdh9XtuUM8rJyBtli/1opz2+XdBrfXaxHtpweI1uf36Hvy8tJ5BVUdZvOBn5epr+TvDxenrxSuYTsp96Uvm65p4FNmurBsmXfrEnWp8nkQbcWeTUwq2zL5JLP7UoZ/Lx8Pqqk809kP/UJZH28s3y2Ntn9832yn7en5PsXZB1Zs/K3R0lzTbLb5X1kPVuXbGVPK/ttPFnvtivlNpPsbmps01lkd9xosl5X765Zm7wzY03y+4EnyNZ9D9klV81PlHJdjgygB5DH2+n03RmzZylflf13JfCtpjJeruThPWQ3w/Vl+zYmj/vlyDo9gazTj9HXolfleBpDNpIuK9Mad6WsRjZUHiUD/DmlvFel7zgWeUJ/hDzhLF+2cd2y7COlHNYp+biHvLvsKLIB9CJ5jOxUtnlNshH3mUaalbgxvuzrnchYcV2lLLakqR5Uph1JdtesQTZODihlPKZM/zLZ9blmP7F3a+DOQWN0B4L8FmVHP1Uq0NmU4F2m/wk4pMVy3ys7dxZ5ovgGsHRl+tiS7gtkn9WO0fTtOXnV8Arz3hP7JCUQtFjnZmSXR+Py+UOVaWeSLabG7YUvkkHierL/+O6y3MyyI58nW0OrVvIVTX83kJVzx1KJXiG7AnrJgDeCee8tf4wMjs3pvFQq3GVlp79QymZsqQhXlzwcXdLuJc/uzek8Tgas3rKt1XSa5w0ySDd/9lQpmznkFckT5EHx5sq2NLqvni1l+GCLdBr3G99IBsVqXkaV5XZo2qafA6uUbf1QWSZKuT4JTKjUi+b1zaXcXlvZ5z1k/ZxTyn8aecW4Gbm/X6HvC86LgDdXlj2qxTqOIg/cxvoan79M011PJY3xzHsL5fZlvzXK7zHKLcDkdzgPls/nAOtVlluV/B5sVpmnep/8G8u2PE92//yG/u9OC/puofxui+37NRl4p5Z1zaDU46Z09inreh19V7PP0ff7ihfK/p3dYh1jByrfMu3+FtNOaHEc/4mm27nJOta87Hgy/jxW9neQx9t+ZZnnBshn9fcUc8mGyejK+q6h7/ciJzdNG0V2bU4n698t5AOaqvtjNvPGt69Upp/YvH2t/jxAmZlZhaRvAY9HxA8Wd176U74b+j3w1oh4ccB5HeTNzOrLY9eYmdWYg7yZWY05yJuZ1ZiDvJlZjTnIm5nV2LAJ8pJC0n9U3h8u6agOpT1Z0l6Dzznk9XxYOZD/FR1I62JJYwaZZ0L5mX67ae4m6UtDzdsCrG+spOZfQg62zHKSfl9+fbqw6x207JrmX+B8LihJZ0qaKqm/sUnaTWecpOMHmWeMpH8aYPq1ldfHlYG8jhtKvha3sg/3rbyfoBwgcKjpfkrSgQNMf7+kb/Q3fTgYNkGevOl/D5VRFocL9Y2M2Y5DgE9EDsA0JBHxvoh4epDZJpC/1Gw3zV9HxLFDydci8DHgvIiY287MrfZPm2W3yEhaE3h7RLw5Ir4/lLQixw6ab+ybJmPIX9/2l8bWlbcTyR95/Wt/8y9qC3jMNYwlfzDWURFxUkQMNCTLReSol60GaBsWhlOQn0P+lH2+lk5zS1zSc+X/+NLqu0DSvZKOlbSfpBuVQwlvUElmR0lTlMMav78sP6K0ZG4qraxPVtK9WtKvyV/GNudnn5L+7ZK+Uz47ghzh75TmVlFJ7ypJF0m6W9JJZdCmlmmVz++XtHppodwl6celxXVZae3uRf7s/gxJt5bPjpV0Z9mW77XI96utm1Kmx0u6tpTdfFc6/a27TNtC0vVlXedLWqV8vqWk2yTdRo4B0kirZVm3sB/5S2KUjitlM03SR9vcPwOW3cLkU9LnJJ1aXr+p5Kl5IKlRkk4ref2jpMbJ/jJg7bKftmtaZnKpD811s2VaZdsvLK+PknSqpCvLPmwE/2PJIWxvba6LZbnG8fNrcriBmxtlW5mnv7SR9Pmy/bern8fkSXpO0vdLuf+fpJ7y+SdK2d4m6VeNMqyUww3AdyVtIOlS5XDXV0vauDJfq3p7LLBd2eZGDFmrpPFnSd+t5G0nSddJukXSOZJWKJ/Pd/yUcji8vD6sMv0syJHbyF/Rvr9VOQwLQx3WoFN/9I2mdz85Fsnh9P2MeTI5wuSr81Z+Fv40Oc7GsuSYEkeXaf8C/KCy/KX0jaP9MPmT4omUx8KV5aeQA0KNJ3+CvX6LfK5F/nS8hxxf43fkaJWQO3tcPz9ff5G+J8D8lhzFbqC07idHrRtLngC3KJ+fDezfvD5yPI+76fuB25gW+ZhADibXKJNzSplsCvylxfwDrXsqsH15/Y1KWU+l76f4x1GeXNNfWTetbxlgRuX9nqWsRpDjezxY9nW/+2cBym6B8lnK6SpyOIUplMfyNa33C8Cp5fXGJb+jSj5aPsGH/utmf2mNJwfTgvzp/7Uln6uTP+dfeqD1VY+f5tdN8/SX9pbk8A+jyRPEHeSvLlsNkdAYFuAI+urdapV5jqE8vq6Uw4WUIRLIESDfUF6/A/jdQPW2Wi6Vun4vGUtGkcMsrEvfk8IaT6j7Yslfy+OnlMPh5fWjwLLNxxfZMDmhVTkOh7/h1JIncpS1n9FiKNYB3BQR0yNiNjkGSmN86mlkZW84O3JY4z+TO39jckChAyXdSo4zsxp9D1O4MeYdqrjh7cCVEdEbOQTwGeTAT4O5MXJ45bnkeBfbLkBa90XEreX1zU3b1TCTPJGcImkPcqySwfxPKZM7ySDaynzrlrQyWckbzwf4KfBuZT/4mIi4qnz+80o6A5V1w+rMO2zqtsCZkcPGPkb+jPvtZVp/+6ed/C9wPiNHIpxQ5v19RPyhxbq2JQfzInKY2QfIsWMG06putpvWRZEjwD5Bjk3U335cGK3S3hY4PyJmRY4MeR45cFqzV8hHElK2ozGO/ealZT6NDI7Vh5ucExFzS8t6a+Ccsh9OJk/uDe3UW4D/i4iZkT/7v5McS+ed5MnhDyXtg8rn7Rw/U8kr5/3JxkPD4yxAt+mitjB9X932A3KgntMqn82hbwjZpcgWX8PsyutXKu9fYd7tax6/IciR6j4TOQTvqySNJ1uKndRq/e2qbuNcypC98yQWMUfSVuSgXnuR43a/ZwHSbR5et+11t6llWTd5gYHHFq9qd/8saP4HyucbyCvOTh/QnawbnTymO5l2Y5smk1ert0maQLbAGxr7dCnymQZbtJGv/upt83yN/Av4bUTs0zxzG8fPrmQj7APAVyW9qTTORtFibPvhYli15AEi4knysvqQysf3k5eJkEOxLr0QSX9Y0lLKfvrXk5dmvwE+rXxqDcrHcI0eJJ0byae9rK68A2Qf5n3iVX+2krR+OUl9lBydbmHTaniW8gSg0vpZOfJBH59jkEeCDUXkQyaeqvQvH0C2bp8GnlY+MAKypdYwaFlHjsk/QlIj0F8NfFTZT95DHmA3diD/C5zPcvVyfMnDamp9t9bVjbSUT6haj6xng2lVNxc2LajUiy64Gvig8nGfo8nuq6tbzLcUfQ8B2pes75R8TS/lu1+L5RpX9PdJ+jC8+t3MYPW53W2+HthG0oYl7dFlHw94/JTjdt2IuILs4lmZ7K6CvMLq6t1ZQzEcW/KQY7AfWnn/Y+AC5Zdkl7JwrewHyQCxEvCpiHhR0k/Iro9bJIkczvaDAyUSEdOVtyFeQbYKLoqIVg8raHYT+YSYDcuy50fEKwuZVsNk4CRJL5DjdF9QAqSY/zmWnXZQWffyZBfDweXzg4FTJQXzPtqt3bK+jLy0v5wcB/9d9A2b/G8RMaPxJdwQLWg+v08+wPoeSYcAV0i6KvKpVA3/DfyodEXMIYc+np3JDKhV3VzYtIiIvymfTXw7+WStjt05ExG3SJpM38n2JxHxxxazziIbNl8juzMaX+x+newG6y3/+wvM+5Hb/zWyUXcWAz8cYyowt8SIyeSQ2K3y31uuIM6UtGz5+GvkSWKg42cEcHo52Qs4Pvru4PoHcshvJO1Gfk820CMVFymPQrkIlO6fwyNi+H4DP0xIehvwuYg4YHHnZVEoAfPCiDh3sHmXJJKei4gVBp9zyaZ8OPkvImKHxZ2X/gy77hr7+xYRt5Ct5IX+MZTZIrQeeSfUsOWWvJlZjbklb2ZWYw7yZmY15iBvZlZjDvJmZjXmIG9mVmP/D9I64mpTf5k3AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_dendrogram(model, **kwargs):\n",
    "    counts = np.zeros(model.children_.shape[0])\n",
    "    n_samples = len(model.labels_)\n",
    "    for i, merge in enumerate(model.children_):\n",
    "        current_count = 0\n",
    "        for child_idx in merge:\n",
    "            if child_idx < n_samples:\n",
    "                current_count += 1\n",
    "            else:\n",
    "                current_count += counts[child_idx - n_samples]\n",
    "        counts[i] = current_count\n",
    "\n",
    "    linkage_matrix = linkage(x_test)\n",
    "    dendrogram(linkage_matrix, **kwargs)\n",
    "\n",
    "plt.title('Hierarchical Clustering Dendrogram')\n",
    "plot_dendrogram(cls2, truncate_mode='level', p=16)                          \n",
    "plt.xlabel(\"Number of points in node (or index of point if no parenthesis).\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9789375"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy2_cls = accuracy_score(y_test, y2_cls_predict)\n",
    "accuracy2_cls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[15502,   123],\n",
       "       [  214,   161]], dtype=int64)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD4CAYAAAAn3bdmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAf70lEQVR4nO3deZgV1ZnH8e+vuzGKSjeIIJsBFTXgGAOR6DhjjCSgRgQS3BWMKKODGoPLiCFiwBg1KkoCGKIoLnEBo6DBIYgyZhERl7hiJK7sRNaIAo3v/HELclmavr0BVfw+z1NP3zp1quqUHN8+/dapW4oIzMwsHYq2dwPMzKxwDtpmZinioG1mliIO2mZmKeKgbWaWIiV1fYJrD67n6Sm2mWtfnr+9m2A7ovqNVdNDVCXmXDtrbY3Pt615pG1mliJ1PtI2M9uWUjd0riIHbTPLlOKMR20HbTPLFDlom5mlR8ZjtoO2mWWLR9pmZimS9SlxDtpmlikeaZuZpUiRg7aZWXpkPGY7aJtZtnikbWaWIhmP2Q7aZpYtvhFpZpYixcr2F4s6aJtZpmR8oO2gbWbZ4qBtZpYiWc9pZ/2JTzPbyagKS6XHksZIWiTpjS1su0xSSGqcrEvScEmzJb0mqUNe3T6S3k2WPnnlHSW9nuwzXKr8V46DtpllSpEKXwpwD3DcpoWSWgFdgI/yio8H2iZLP2BUUrcRMBj4BtAJGCypYbLPKOD8vP02O9dm11dQs83MUqI2g3ZEPAcs2cKmYcCVQP5Ule7AvZEzHSiT1AzoCkyJiCURsRSYAhyXbGsQEdMjIoB7gR6Vtck5bTPLlLpOaUvqDsyNiL9uks1oAXyctz4nKdta+ZwtlG+Vg7aZZUpVHmOX1I9cKmO90RExeiv16wNXk0uNbBcO2maWKVUZaScBusIgvQX7A22A9aPslsDLkjoBc4FWeXVbJmVzgWM2KZ+WlLfcQv2tck7bzDJFKnypqoh4PSKaRETriGhNLqXRISIWABOB3skskiOA5RExH5gMdJHUMLkB2QWYnGxbIemIZNZIb2BCZW3wSNvMMqU2v+VP0oPkRsmNJc0BBkfEXRVUnwScAMwGVgE/AIiIJZKGAi8m9YZExPqbm/9NbobKbsBTybJVDtpmlim1mT6IiNMr2d4673MA/SuoNwYYs4XymcAhVWmTg7aZZUrWn4h00DazTMl4zHbQNrNs8ZtrzMxSJOMx20HbzLKlJONR20HbzDLFNyLNzFIk608MOmibWaZ4pG1mliJFfrGvmVl6OD1iZpYixU6PmJmlh3PaZmYp4vSImVmK+DF2M7MUcXrEzCxFfCPSzCxFnNM2M0sR57TNzFIk4zHbQdvMsiXrI+2sp3/MbCejKiyVHksaI2mRpDfyyn4haZak1yQ9Jqksb9tASbMlvSOpa175cUnZbElX5ZW3kfRCUv6wpF0qa5ODtpllSklRFLwU4B7guE3KpgCHRMShwN+AgQCS2gGnAe2TfUZKKpZUDIwAjgfaAacndQFuBIZFxAHAUqBvZQ1y0DazTKnNkXZEPAcs2aTsDxFRnqxOB1omn7sDD0XE6oh4H5gNdEqW2RHxXkSsAR4CuksScCwwPtl/LNCjsjY5aJtZphSp8EVSP0kz85Z+VTzducBTyecWwMd52+YkZRWV7wUsy/sFsL58q3wjciu6/+w3HHjMCXz6ySJGnvS1LdZp3elojht4K0UlJaxa9gn3nN25RucsrrcLPW+8m+btO7Bq2RLGDziDZXM/3LC9tFkr+j/5GtNGDOEvY4bV6FxWPQOvvZ5pz/2ZvRo15Mnx92+2feKkyfzmngcggt3r1+faqy/n4IPa1uica9as4cqfDOXNt9+hrLSUYTcOoWXzZvx5+gxuGX4Ha9eupV69elxxaX+O7NSxRudKu6rch4yI0cDoap1H+jFQDjxQnf2ryyPtrXj1sbHcf/6JFW7fdc9SvnvNL3nwv3systthjPvhaQUfu6zFlznn3qc3K+/Q61w+X7GM4V2/wvSxt/Pty67faHvXq37Bu3/838Ivwmrd97qdwJ0jbq1we8vmzbn/zl/xxLj7uPD8c/jJdTcVfOw58+Zz9nkXbVY+7vEnabDnnkyZ+AjnnHkqN98+EoCGZWWMuu1Gnhh3HzcMGcSVg4ZU/YIypioj7eqSdA5wInBmRKxPjs8FWuVVa5mUVVT+CVAmqWST8q1fX/WbnX0fzvwTny1fUuH2fzvxdN6e8jjL5+f+8vl0yeIN2w7tdgbnP/IXLnhsJif+dCQqKuw/9UGdu/Hq4/cB8NbkR9nvyGM3bDu480ksnfMBi2e/VZ3LsVpyeMfDKC1tUOH2Dof9G6UNctsPO7Q9CxYu2rBtwu8n0+us8+h+ah+uue4m1q1bV9A5n5n2R3p2OwGArt8+hudnvERE0O7gA2naZG8A2u7fhtWrV7NmzZrqXlomFFVhqQ5JxwFXAidFxKq8TROB0yR9SVIboC0wA3gRaJvMFNmF3M3KiUmwfxbolezfB5hQyPUV0sibJDWQVE/SVEmLJZ1V6EVm1V6t27Jrg4acc+/T9Hv0Bb7aPfefpPF+B9P+hJO564yjuaPn14l16zi02xkFHbNBk+asSH4JfLFuHZ+vXE79sr3Ypf7uHHX+FfzfiKF1dj1W+8Y//iRHH3UEAH9/7wOe+sNUHrz7DiY8PJaioiKemPSHgo6zcNFimu3TBICSkhL23GN3li5bvlGdyU9Po93BB7HLLpXOGsu04qLCl8pIehB4HjhI0hxJfYFfAXsCUyS9KukOgIh4E3gEeAv4X6B/RKxLctYXAZOBt4FHkroA/wMMkDSbXI77rsraVGhOu0tEXCmpJ/AB8D3gOWDzhF7uQvsB/QBObFpEx7JsDuiLSkpo3r4DY3/QhXpf2o2+D/2ROX99gf2OPJbm7TvQb9x0AEp23ZVPl+RGW6f+chwNW7ahuF49SpvtywWPzQRg+n2/5NXfja3wXMdcdA3T77mdNas+rfsLs1ox/cWXGP/4k/x2zCgAnp8xkzfemkWvs3Kzuj5fvZq9GjUEoP+AgcyZO4+1a8uZv2Ah3U/tA0DvM07h+92/W+m53v37e9w8fCRjRvo+R20+WxMRp2+huMLAGhE/A362hfJJwKQtlL9HbnZJwQoN2uvrfRcYFxHLtZXvP8xP7l97cL3MvmVzxYI5rFr2CWs/W8Xaz1bx4cw/0fSgQ0Hi1cfvY+qtgzbb5+GLTwZyOe0eP7+Le3p/e+NjLppHg2atWLFwLkXFxey6Zymrln1Ci0M70a7r9/jOFT9n1z3LiC++oHz1amY8MHKbXKtVzay/zWbQkBv4za9uoWFZKQARQc9ux3PZJRduVn/ErT8Hcjntgdf8jPvu/NVG25s22Zv5CxaxT9MmlJeXs/Kfn2447oKFi7howNXcOPQn7Nuq5WbH3tlsLTZlQaFD4CclzQI6AlMl7Q18XnfNSodZU59g3w5HUVRcTL1dd6PloYfzj/dm8f7zz9Cuy/fYvVEu17hbaUNKm+9b0DHfeeZJDutxNgDtun6f96c/C8DdZ32L2zq35bbObZl+73D+OPoGB+wd1Lz5C7j48qu5aeg1tPnyv/7dj+z0dSY/PY1PliwFYNnyFcydt6CgYx77zf/gsSdyA7XJT0/jiMM7IokVK1fS7+IruOySC+h42KG1fzEpJBW+pFFBI+2IuErSTcDyiFgn6VNyE8kz7fu33Efrw79J/YaNGTDtfZ795RCKS+oBMPPh0fzjvVnM/uNkLpzwMvHFF7w8/m4WvZtLVT1z+2DOvuspVFTEuvK1TBpyCcvnfVTpOV8ZP4aeN93DJZPf5rPlSxk/4Mw6vUarugFXDWbGS6+wdNkyju7ag4sv6Et5eW6q7ekn92TE6LtZtmwFP/35zQAUFxfzu9+O4YD923Bp//M598JL+SKCeiUlXHPVAFo036fSc/bqcSJXDBrKd046hdIGDRh2w08BuP+hR/no4zmMGH03I0bfDcCYUbdtSLvslNIajQukf81W2UolqR5wIXB0UvR/wB0RsbayfbOcHrHqu/bl+du7CbYjqt+4xhH3vZMKjzn7TVybughfaE57FFAPWP/3+NlJ2Xl10Sgzs+rKek670KB9eER8NW/9GUl/rYsGmZnVSDYnq21Q6OWtk7T/+hVJ+wGFPRVgZrYNSSp4SaNCR9pXAM9Kei9Zbw38oE5aZGZWAymNxQUrdKT9Z+DXwBfkvqbw1+SeEjIz26F4pJ1zL7ACWP8M9RnAfcDJddEoM7PqUsbfN1Zo0D4kItrlrT8ryd9aZGY7nJQOoAtWaHrkZUlHrF+R9A1gZt00ycys+pweyekI/EXS+kf69gXekfQ6EMm70szMtr+UBuNCFRq0N32xpZnZDinjMbvg7x75sPJaZmbbX1rTHoXyOyLNLFM8e8TMLEUyPtB20DazbHF6xMwsRbIetDP+fVhmtrOpzTfXSBojaZGkN/LKGkmaIund5GfDpFyShkuaLek1SR3y9umT1H9XUp+88o6SXk/2Ga4CfuM4aJtZpqhIBS8FuIfNpzxfBUyNiLbA1GQd4HigbbL0I/fOASQ1AgYD3yD3Et/B6wN9Uuf8vP0qnV7toG1mmVKbT0RGxHPkviQvX3dgbPJ5LNAjr/zeyJkOlElqBnQFpkTEkohYCkwBjku2NYiI6ZF7hdi9eceqkIO2mWVKVdIjkvpJmpm39CvgFE0jYv378hYATZPPLYCP8+rNScq2Vj5nC+Vb5RuRZpYtVbgRGRGjgdHVPVVEhKRt+h5cj7TNLFO2wRdGLUxSGyQ/FyXlc4FWefVaJmVbK2+5hfKtctA2s0ypzdkjFZgIrJ8B0geYkFfeO5lFcgSwPEmjTAa6SGqY3IDsAkxOtq2QdEQya6R33rEq5PSImWVKbT7GLulB4BigsaQ55GaB3AA8Iqkv8CFwSlJ9EnACMBtYRfJKxohYImko8GJSb0hErL+5+d/kZqjsBjyVLFvloG1mmaKi2ksgRMTpFWzqvIW6AfSv4DhjgDFbKJ8JHFKVNjlom1m2ZPyJSAdtM8sWB20zs/SQsj2/wkHbzLKlFnPaOyIHbTPLlKx/y5+Dtplli9MjZmbp4deNmZmlidMjZmYp4vSImVl61OYTkTsiB20zyxanR8zM0sMP15iZpYlH2mZm6eEpf2ZmaeL0iJlZenj2iJlZmjinbWaWIk6PmJmlR9a/5S/bv5LMbOdTpMKXSkj6kaQ3Jb0h6UFJu0pqI+kFSbMlPSxpl6Tul5L12cn21nnHGZiUvyOpa40uryY7m5ntaFRUXPCy1eNILYBLgK9HxCFAMXAacCMwLCIOAJYCfZNd+gJLk/JhST0ktUv2aw8cB4yUtPWTb4WDtplli1T4UrkSYDdJJUB9YD5wLDA+2T4W6JF87p6sk2zvrFyupjvwUESsjoj3gdlAp+penoO2mWWKpKos/STNzFv6rT9ORMwFbgY+IheslwMvAcsiojypNgdokXxuAXyc7Fue1N8rv3wL+1SZb0SaWbZUYfZIRIwGRm/xMFJDcqPkNsAyYBy59MZ25ZG2mWVL7aVHvg28HxGLI2It8DvgKKAsSZcAtATmJp/nAq1yTVAJUAp8kl++hX2qrM5H2te+PK+uT2FmtkEtTvn7CDhCUn3gM6AzMBN4FugFPAT0ASYk9Scm688n25+JiJA0EfitpFuB5kBbYEZ1G+X0iJllS3G1J2ZsJCJekDQeeBkoB14hl0r5PfCQpOuSsruSXe4C7pM0G1hCbsYIEfGmpEeAt5Lj9I+IddVtlyKiuvsWZtXiOj6BpVO2H4CwaqrfuMYd44vbv1VwzCn64bOp64geaZtZtmT8iUgHbTPLFn/3iJlZinikbWaWIh5pm5mlSCXfKZJ2Dtpmli1Oj5iZpYiDtplZijinbWaWIh5pm5mliG9EmpmliNMjZmYp4vSImVmKeKRtZpYiHmmbmaWIR9pmZini2SNmZini9IiZWYo4PWJmliIeaZuZpUjGR9rZvjoz2/lIhS+VHkplksZLmiXpbUlHSmokaYqkd5OfDZO6kjRc0mxJr0nqkHecPkn9dyX1qcnlOWibWbYUFxe+VO524H8j4mDgq8DbwFXA1IhoC0xN1gGOB9omSz9gFICkRsBg4BtAJ2Dw+kBfHQ7aZpYttTTSllQKHA3cBRARayJiGdAdGJtUGwv0SD53B+6NnOlAmaRmQFdgSkQsiYilwBTguOpenoO2mWWLigpeJPWTNDNv6Zd3pDbAYuBuSa9IulPS7kDTiJif1FkANE0+twA+ztt/TlJWUXm1+EakmWVLFWaPRMRoYHQFm0uADsDFEfGCpNv5Vypk/f4hKarb1OrwSNvMsqUKI+1KzAHmRMQLyfp4ckF8YZL2IPm5KNk+F2iVt3/LpKyi8mpx0DazbCkqKXzZiohYAHws6aCkqDPwFjARWD8DpA8wIfk8EeidzCI5AliepFEmA10kNUxuQHZJyqrF6REzy5bafbjmYuABSbsA7wE/IDfYfURSX+BD4JSk7iTgBGA2sCqpS0QskTQUeDGpNyQillS3QYqo43TMqsXbNN9jaZHtp9asmuo3rnHH+OLJywuOOUUn3py6juiRtpllS8afiHTQNrNs8XePmJmliEfaZmYpUsmskLTL9tWZ2c7H6REzsxRxesTMLEUctM3MUqTIQdvMLD38NnYzsxRxesTMLEU8e8TMLEU80jYzSxEHbTOzFHHQNjNLEc8eMTNLEY+0zcxSxLNHzMxSxCPtndP8BQu58ifX8cknS5HglO+fRJ8zTtmozt/f/5CrB1/Pm7P+xo8uOp++vc+o8XnXrFnDlT+5jjfffoey0gYMu3EILZs348/TX+SW4aNYu7acevVKuOLS/hzZqWONz2dVN/Da65n23J/Zq1FDnhx//xbrvDDzZa7/xe2Ul5fTsKyM++8aUaNz5vrF0KRflOb1ixncMvwO1q5dS7169dwvIPNBO9tXVwPFxcVcNeAiJv3ufh6+dzS/ffh3zP77+xvVKSttwI//51L69j6tysefM28+Z5930Wbl4x5/kgZ77smUiQ9zzpmncvPtowBoWFbKqNtu4olx93LDkEFcOWho9S7Maux73U7gzhG3Vrh9xcqV/PT6Wxh12438/tEHuP0X1xV87Mr7xSNJvxgJQMOyMkbddiNPjLsv6RdDqn5BWaOiwpdCDicVS3pF0pPJehtJL0iaLenh5KW/SPpSsj472d467xgDk/J3JHWtyeU5aFegyd6Naf+VgwDYY/f67NemNQsX/2OjOns1asih7b9CScnmf7BM+P1kep11Pt1PPYdrrruJdevWFXTeZ6b9iZ7djgeg67eP4fkZLxERtDv4QJo2aQxA2/3bsHr1atasWVOTS7RqOrzjYZSWNqhw+xNPTeE7nb9J82b7ALl+sl6uX5xH91P7VLFf/JGe3U4AttQv9gbcLzYoLi58KcwPgbfz1m8EhkXEAcBSoG9S3hdYmpQPS+ohqR1wGtAeOA4YKanaU1wctAswZ9583n7nb3z1kHYF1f/7ex/w1B+m8uDdo5jw8D0UFRXxxKQ/FLTvwkWLabZPEwBKSkrYc4/dWbps+UZ1Jj89jXYHH8guu+xStQuxbeKDDz9ixYqVnH3eRXzvjHN5/ImngPx+cQcTHh5bR/3iIPeLWhxpS2oJfBe4M1kXcCwwPqkyFuiRfO6erJNs75zU7w48FBGrI+J9YDbQqbqXV1BOW9JKYNPX0i8HZgKXRcR7m9TvB/QD+PUvb6bfub2r277t7tNVq7jk8h9z9eU/ZI89di9on+dnvMQbb71Dr7POA+Dz1as3jLb6DxjInLnzWbu2nPkLFtL91HMA6H3GyXy/+3crPfa7f3+Pm4ePYszIYdW7IKtz69at4823Z3HPr4fz+eerOa3Pf/HVQ9vz/IyZvPHWLHqdlRuYbd4v5uX1iz4A9D7jlCr0i5HuF1ClnHZ+rEqMjojReeu3AVcCeybrewHLIqI8WZ8DtEg+twA+BoiIcknLk/otgOl5x8zfp8oKvRF5W3Ki3wIiN9TfH3gZGAMck185uejcha9avGmwT421a8u55PJBdDu+C106f7Pg/SKCnt2O57JLLths24hbfw7kRu8Dr/kZ9935q422N22yN/MXLGKfpk0oLy9n5T8/pWFZKQALFi7iogFXc+PQQezbqtr/5lbH9mnShLLSUurvthv1d9uNr3c4jFl/m53XLy7cbJ/a6Rc/Yd9WLev+And0VZjyt1Gs2uwwOhFYFBEvSTqmVtpWCwr9lXRSRPw6IlZGxIrkQrtGxMNAw8p2TqOI4Mc//Tn7tfkyPzi7ajcaj+zUkclPT+OTJUsBWLZ8BXPnLSho32O/eRSPJX9OT356Gkcc3gFJrFi5kn4XX8Fll1xIx8MOrdrF2DbV+Zj/5KVXX6O8vJzPPvuc1954k/3btObITl+vQb/4Dx57YhKwvl903KRfXOB+sYGqsGzVUcBJkj4AHiKXFrkdKJO0fsDbEpibfJ4LtAJItpcCn+SXb2GfKit0pL1K0in8K4/TC/g8+ZzakfTWvPTqa0z4/WQObLv/hhTGgIv+i3kLFgJw+sk9WPyPT/j+mefxz08/pUhFjH1gHJMevZ8D9m/Dpf3P59wLf8QXEdQrKeaaqwbQovk+lZ63V48TuWLQUL5z0qmUNmjAsBuuBeD+hx7lo4/nMmL03YwYfTcAY0YN2+gml20bA64azIyXXmHpsmUc3bUHF1/Ql/Ly3F/Lp5/ck/33a81//vs3OOmUPhQViV49u3HgAfsBJP3i0qRflFSjX5yS9IufAuv7xZxN+sVtO3e/qKXH2CNiIDAQIBlpXx4RZ0oaRy4GPgT0ASYku0xM1p9Ptj8TESFpIvBbSbcCzYG2wIzqtksRlcdcSfuR+w1zJLkgPR34EbnfFh0j4k8V7pzi9IjVpWw/tWbVVL9xjTvGF288UnDMKTrklILOlxe0T0zi4UNAI+AV4KyIWC1pV+A+4GvAEuC09ff7JP0YOBcoBy6NiKcKv6JN2lJI0K4RB23bIgdt24LaCNpvjis8aLc/OXUdsaCctqQDJU2V9EayfqikQXXbNDOzapAKX1Ko0BuRvyGX21kLEBGvkZtBYma2gymqwpI+hd6IrB8RM7Txb6byiiqbmW03KR1BF6rQoP0PSfuTzBSR1AuYX2etMjOrruo/IZ4KhQbt/uQmoB8saS7wPnBmnbXKzKy6PNIGclP77gaeJTfNZQW5+Yj+SjEz27Fk/KtZCw3aE4Bl5B5bn1dnrTEzqzGPtAFaRsRxddoSM7PakPH0SKF/R/xF0r/VaUvMzGpDLb8EYUdT6Ej7P4BzJL0PrCb390dEhL+hxsx2KDV4v0AqFBq0j6/TVpiZ1ZaMp0cKCtoR8WFdN8TMrFY4aJuZpUk6c9WFctA2s2zxSNvMLEV8I9LMLEU80jYzS5GUzr8ulIO2mWWMR9pmZumR8fRItv+OMLOdTy09xi6plaRnJb0l6U1JP0zKG0maIund5GfDpFyShkuaLek1SR3yjtUnqf+upD41uTwHbTPLltr77pFy4LKIaAccAfSX1A64CpgaEW2Bqck65J4cb5ss/YBRkAvywGDgG0AnYPD6QF8dDtpmli21FLQjYn5EvJx8Xgm8DbQAugNjk2pjgR7J5+7AvZEzHSiT1AzoCkyJiCURsRSYAlT7W1MdtM0sY1TwIqmfpJl5S78tHlFqDXwNeAFoGhHrX7e4AGiafG4BfJy325ykrKLyavGNSDPLlirciIyI0eRepbiVw2kP4FHg0ohYkf+C84gISVHNllaLR9pmljGFj7QrPZJUj1zAfiAifpcUL0zSHiQ/FyXlc4FWebu3TMoqKq8WB20zyxap8GWrh5GAu4C3I+LWvE0Tyb0jl+TnhLzy3skskiOA5UkaZTLQRVLD5AZkl6SsWpweMbNsqb152kcBZwOvS3o1KbsauAF4RFJf4EPglGTbJOAEYDawCvgBQEQskTQUeDGpNyQillS3UYqo43TMqsXbNN9jaZHtByCsmuo3rnHHiEVvFhxz1KR96jqiR9pmli0ZfyLSQdvMMsZB28wsPTzSNjNLEX81q5lZmnikbWaWHk6PmJmlSbaDdraTP2ZmGeORtpllipweMTNLEc8eMTNLE4+0zczSw+kRM7M0cdA2M0sPj7TNzNLEQdvMLD08e8TMLEWyPdB20DazrMl21HbQNrNs8Y1IM7M0cdA2M0uPjN+IrPu3sdsGkvpFxOjt3Q7bsbhfWFVk+1fSjqff9m6A7ZDcL6xgDtpmZinioG1mliIO2tuW85a2Je4XVjDfiDQzSxGPtM3MUsRB28wsRfxwTS2QtBcwNVndB1gHLE7WO0XEmu3SMNsuJK0DXs8r6hERH1RQ958Rscc2aZhlgnPatUzStcA/I+LmvLKSiCjffq2ybakqgdhB26rK6ZE6IukeSXdIegG4SdK1ki7P2/6GpNbJ57MkzZD0qqRfSyreXu222idpD0lTJb0s6XVJ3bdQp5mk55I+8Iak/0zKu0h6Ptl3nCQH+J2cg3bdagn8e0QMqKiCpK8ApwJHRcRh5FIrZ26b5lkd2S0Jvq9Kegz4HOgZER2AbwG3SJt9Fd0ZwOSkD3wVeFVSY2AQ8O1k35lAhX3Jdg7OadetcRGxrpI6nYGOwIvJ/8e7AYvqumFWpz5Lgi8AkuoB10s6GvgCaAE0BRbk7fMiMCap+3hEvCrpm0A74M9J39gFeH7bXILtqBy069aneZ/L2fgvm12TnwLGRsTAbdYq29bOBPYGOkbEWkkf8K9/fwAi4rkkqH8XuEfSrcBSYEpEnL6tG2w7LqdHtp0PgA4AkjoAbZLyqUAvSU2SbY0kfXm7tNDqSimwKAnY3wI2+/dN/s0XRsRvgDvJ9ZXpwFGSDkjq7C7pwG3YbtsBeaS97TwK9Jb0JvAC8DeAiHhL0iDgD5KKgLVAf+DD7dZSq20PAE9Iep1cXnrWFuocA1whaS3wT6B3RCyWdA7woKQvJfUGkfQd2zl5yp+ZWYo4PWJmliIO2mZmKeKgbWaWIg7aZmYp4qBtZpYiDtpmZinioG1mliL/Dy2uiR5M4CEgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm2_cls = confusion_matrix(y_test, y2_cls_predict)\n",
    "df_cm2_cls = pd.DataFrame(cm2_cls, index=['pos', 'neg'], columns=['True', 'False'])\n",
    "ax = sn.heatmap(df_cm2_cls, cmap='Oranges', annot=True)\n",
    "cm2_cls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  ------------------------------------------BIRCH-------------------------------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "MiniBatchKMeans'e alternatif olarak sağlanan, bellek açısından verimli, çevrimiçi öğrenme algoritmasıdır. Yapraktan okunan küme ağırlık merkezleriyle bir ağaç veri yapısı oluşturur. Bunlar, son küme merkezleri olabilir veya AglomerativeClustering gibi başka bir kümeleme algoritmasına girdi olarak sağlanabilir."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model oluşturma ve test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "cls3 = Birch(n_clusters=2)\n",
    "cls3.fit(x_test)\n",
    "y3_cls_predict = cls3.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[15502,   123],\n",
       "       [  214,   161]], dtype=int64)"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD4CAYAAAAn3bdmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAf70lEQVR4nO3deZgV1ZnH8e+vuzGKSjeIIJsBFTXgGAOR6DhjjCSgRgQS3BWMKKODGoPLiCFiwBg1KkoCGKIoLnEBo6DBIYgyZhERl7hiJK7sRNaIAo3v/HELclmavr0BVfw+z1NP3zp1quqUHN8+/dapW4oIzMwsHYq2dwPMzKxwDtpmZinioG1mliIO2mZmKeKgbWaWIiV1fYJrD67n6Sm2mWtfnr+9m2A7ovqNVdNDVCXmXDtrbY3Pt615pG1mliJ1PtI2M9uWUjd0riIHbTPLlOKMR20HbTPLFDlom5mlR8ZjtoO2mWWLR9pmZimS9SlxDtpmlikeaZuZpUiRg7aZWXpkPGY7aJtZtnikbWaWIhmP2Q7aZpYtvhFpZpYixcr2F4s6aJtZpmR8oO2gbWbZ4qBtZpYiWc9pZ/2JTzPbyagKS6XHksZIWiTpjS1su0xSSGqcrEvScEmzJb0mqUNe3T6S3k2WPnnlHSW9nuwzXKr8V46DtpllSpEKXwpwD3DcpoWSWgFdgI/yio8H2iZLP2BUUrcRMBj4BtAJGCypYbLPKOD8vP02O9dm11dQs83MUqI2g3ZEPAcs2cKmYcCVQP5Ule7AvZEzHSiT1AzoCkyJiCURsRSYAhyXbGsQEdMjIoB7gR6Vtck5bTPLlLpOaUvqDsyNiL9uks1oAXyctz4nKdta+ZwtlG+Vg7aZZUpVHmOX1I9cKmO90RExeiv16wNXk0uNbBcO2maWKVUZaScBusIgvQX7A22A9aPslsDLkjoBc4FWeXVbJmVzgWM2KZ+WlLfcQv2tck7bzDJFKnypqoh4PSKaRETriGhNLqXRISIWABOB3skskiOA5RExH5gMdJHUMLkB2QWYnGxbIemIZNZIb2BCZW3wSNvMMqU2v+VP0oPkRsmNJc0BBkfEXRVUnwScAMwGVgE/AIiIJZKGAi8m9YZExPqbm/9NbobKbsBTybJVDtpmlim1mT6IiNMr2d4673MA/SuoNwYYs4XymcAhVWmTg7aZZUrWn4h00DazTMl4zHbQNrNs8ZtrzMxSJOMx20HbzLKlJONR20HbzDLFNyLNzFIk608MOmibWaZ4pG1mliJFfrGvmVl6OD1iZpYixU6PmJmlh3PaZmYp4vSImVmK+DF2M7MUcXrEzCxFfCPSzCxFnNM2M0sR57TNzFIk4zHbQdvMsiXrI+2sp3/MbCejKiyVHksaI2mRpDfyyn4haZak1yQ9Jqksb9tASbMlvSOpa175cUnZbElX5ZW3kfRCUv6wpF0qa5ODtpllSklRFLwU4B7guE3KpgCHRMShwN+AgQCS2gGnAe2TfUZKKpZUDIwAjgfaAacndQFuBIZFxAHAUqBvZQ1y0DazTKnNkXZEPAcs2aTsDxFRnqxOB1omn7sDD0XE6oh4H5gNdEqW2RHxXkSsAR4CuksScCwwPtl/LNCjsjY5aJtZphSp8EVSP0kz85Z+VTzducBTyecWwMd52+YkZRWV7wUsy/sFsL58q3wjciu6/+w3HHjMCXz6ySJGnvS1LdZp3elojht4K0UlJaxa9gn3nN25RucsrrcLPW+8m+btO7Bq2RLGDziDZXM/3LC9tFkr+j/5GtNGDOEvY4bV6FxWPQOvvZ5pz/2ZvRo15Mnx92+2feKkyfzmngcggt3r1+faqy/n4IPa1uica9as4cqfDOXNt9+hrLSUYTcOoWXzZvx5+gxuGX4Ha9eupV69elxxaX+O7NSxRudKu6rch4yI0cDoap1H+jFQDjxQnf2ryyPtrXj1sbHcf/6JFW7fdc9SvnvNL3nwv3systthjPvhaQUfu6zFlznn3qc3K+/Q61w+X7GM4V2/wvSxt/Pty67faHvXq37Bu3/838Ivwmrd97qdwJ0jbq1we8vmzbn/zl/xxLj7uPD8c/jJdTcVfOw58+Zz9nkXbVY+7vEnabDnnkyZ+AjnnHkqN98+EoCGZWWMuu1Gnhh3HzcMGcSVg4ZU/YIypioj7eqSdA5wInBmRKxPjs8FWuVVa5mUVVT+CVAmqWST8q1fX/WbnX0fzvwTny1fUuH2fzvxdN6e8jjL5+f+8vl0yeIN2w7tdgbnP/IXLnhsJif+dCQqKuw/9UGdu/Hq4/cB8NbkR9nvyGM3bDu480ksnfMBi2e/VZ3LsVpyeMfDKC1tUOH2Dof9G6UNctsPO7Q9CxYu2rBtwu8n0+us8+h+ah+uue4m1q1bV9A5n5n2R3p2OwGArt8+hudnvERE0O7gA2naZG8A2u7fhtWrV7NmzZrqXlomFFVhqQ5JxwFXAidFxKq8TROB0yR9SVIboC0wA3gRaJvMFNmF3M3KiUmwfxbolezfB5hQyPUV0sibJDWQVE/SVEmLJZ1V6EVm1V6t27Jrg4acc+/T9Hv0Bb7aPfefpPF+B9P+hJO564yjuaPn14l16zi02xkFHbNBk+asSH4JfLFuHZ+vXE79sr3Ypf7uHHX+FfzfiKF1dj1W+8Y//iRHH3UEAH9/7wOe+sNUHrz7DiY8PJaioiKemPSHgo6zcNFimu3TBICSkhL23GN3li5bvlGdyU9Po93BB7HLLpXOGsu04qLCl8pIehB4HjhI0hxJfYFfAXsCUyS9KukOgIh4E3gEeAv4X6B/RKxLctYXAZOBt4FHkroA/wMMkDSbXI77rsraVGhOu0tEXCmpJ/AB8D3gOWDzhF7uQvsB/QBObFpEx7JsDuiLSkpo3r4DY3/QhXpf2o2+D/2ROX99gf2OPJbm7TvQb9x0AEp23ZVPl+RGW6f+chwNW7ahuF49SpvtywWPzQRg+n2/5NXfja3wXMdcdA3T77mdNas+rfsLs1ox/cWXGP/4k/x2zCgAnp8xkzfemkWvs3Kzuj5fvZq9GjUEoP+AgcyZO4+1a8uZv2Ah3U/tA0DvM07h+92/W+m53v37e9w8fCRjRvo+R20+WxMRp2+huMLAGhE/A362hfJJwKQtlL9HbnZJwQoN2uvrfRcYFxHLtZXvP8xP7l97cL3MvmVzxYI5rFr2CWs/W8Xaz1bx4cw/0fSgQ0Hi1cfvY+qtgzbb5+GLTwZyOe0eP7+Le3p/e+NjLppHg2atWLFwLkXFxey6Zymrln1Ci0M70a7r9/jOFT9n1z3LiC++oHz1amY8MHKbXKtVzay/zWbQkBv4za9uoWFZKQARQc9ux3PZJRduVn/ErT8Hcjntgdf8jPvu/NVG25s22Zv5CxaxT9MmlJeXs/Kfn2447oKFi7howNXcOPQn7Nuq5WbH3tlsLTZlQaFD4CclzQI6AlMl7Q18XnfNSodZU59g3w5HUVRcTL1dd6PloYfzj/dm8f7zz9Cuy/fYvVEu17hbaUNKm+9b0DHfeeZJDutxNgDtun6f96c/C8DdZ32L2zq35bbObZl+73D+OPoGB+wd1Lz5C7j48qu5aeg1tPnyv/7dj+z0dSY/PY1PliwFYNnyFcydt6CgYx77zf/gsSdyA7XJT0/jiMM7IokVK1fS7+IruOySC+h42KG1fzEpJBW+pFFBI+2IuErSTcDyiFgn6VNyE8kz7fu33Efrw79J/YaNGTDtfZ795RCKS+oBMPPh0fzjvVnM/uNkLpzwMvHFF7w8/m4WvZtLVT1z+2DOvuspVFTEuvK1TBpyCcvnfVTpOV8ZP4aeN93DJZPf5rPlSxk/4Mw6vUarugFXDWbGS6+wdNkyju7ag4sv6Et5eW6q7ekn92TE6LtZtmwFP/35zQAUFxfzu9+O4YD923Bp//M598JL+SKCeiUlXHPVAFo036fSc/bqcSJXDBrKd046hdIGDRh2w08BuP+hR/no4zmMGH03I0bfDcCYUbdtSLvslNIajQukf81W2UolqR5wIXB0UvR/wB0RsbayfbOcHrHqu/bl+du7CbYjqt+4xhH3vZMKjzn7TVybughfaE57FFAPWP/3+NlJ2Xl10Sgzs+rKek670KB9eER8NW/9GUl/rYsGmZnVSDYnq21Q6OWtk7T/+hVJ+wGFPRVgZrYNSSp4SaNCR9pXAM9Kei9Zbw38oE5aZGZWAymNxQUrdKT9Z+DXwBfkvqbw1+SeEjIz26F4pJ1zL7ACWP8M9RnAfcDJddEoM7PqUsbfN1Zo0D4kItrlrT8ryd9aZGY7nJQOoAtWaHrkZUlHrF+R9A1gZt00ycys+pweyekI/EXS+kf69gXekfQ6EMm70szMtr+UBuNCFRq0N32xpZnZDinjMbvg7x75sPJaZmbbX1rTHoXyOyLNLFM8e8TMLEUyPtB20DazbHF6xMwsRbIetDP+fVhmtrOpzTfXSBojaZGkN/LKGkmaIund5GfDpFyShkuaLek1SR3y9umT1H9XUp+88o6SXk/2Ga4CfuM4aJtZpqhIBS8FuIfNpzxfBUyNiLbA1GQd4HigbbL0I/fOASQ1AgYD3yD3Et/B6wN9Uuf8vP0qnV7toG1mmVKbT0RGxHPkviQvX3dgbPJ5LNAjr/zeyJkOlElqBnQFpkTEkohYCkwBjku2NYiI6ZF7hdi9eceqkIO2mWVKVdIjkvpJmpm39CvgFE0jYv378hYATZPPLYCP8+rNScq2Vj5nC+Vb5RuRZpYtVbgRGRGjgdHVPVVEhKRt+h5cj7TNLFO2wRdGLUxSGyQ/FyXlc4FWefVaJmVbK2+5hfKtctA2s0ypzdkjFZgIrJ8B0geYkFfeO5lFcgSwPEmjTAa6SGqY3IDsAkxOtq2QdEQya6R33rEq5PSImWVKbT7GLulB4BigsaQ55GaB3AA8Iqkv8CFwSlJ9EnACMBtYRfJKxohYImko8GJSb0hErL+5+d/kZqjsBjyVLFvloG1mmaKi2ksgRMTpFWzqvIW6AfSv4DhjgDFbKJ8JHFKVNjlom1m2ZPyJSAdtM8sWB20zs/SQsj2/wkHbzLKlFnPaOyIHbTPLlKx/y5+Dtplli9MjZmbp4deNmZmlidMjZmYp4vSImVl61OYTkTsiB20zyxanR8zM0sMP15iZpYlH2mZm6eEpf2ZmaeL0iJlZenj2iJlZmjinbWaWIk6PmJmlR9a/5S/bv5LMbOdTpMKXSkj6kaQ3Jb0h6UFJu0pqI+kFSbMlPSxpl6Tul5L12cn21nnHGZiUvyOpa40uryY7m5ntaFRUXPCy1eNILYBLgK9HxCFAMXAacCMwLCIOAJYCfZNd+gJLk/JhST0ktUv2aw8cB4yUtPWTb4WDtplli1T4UrkSYDdJJUB9YD5wLDA+2T4W6JF87p6sk2zvrFyupjvwUESsjoj3gdlAp+penoO2mWWKpKos/STNzFv6rT9ORMwFbgY+IheslwMvAcsiojypNgdokXxuAXyc7Fue1N8rv3wL+1SZb0SaWbZUYfZIRIwGRm/xMFJDcqPkNsAyYBy59MZ25ZG2mWVL7aVHvg28HxGLI2It8DvgKKAsSZcAtATmJp/nAq1yTVAJUAp8kl++hX2qrM5H2te+PK+uT2FmtkEtTvn7CDhCUn3gM6AzMBN4FugFPAT0ASYk9Scm688n25+JiJA0EfitpFuB5kBbYEZ1G+X0iJllS3G1J2ZsJCJekDQeeBkoB14hl0r5PfCQpOuSsruSXe4C7pM0G1hCbsYIEfGmpEeAt5Lj9I+IddVtlyKiuvsWZtXiOj6BpVO2H4CwaqrfuMYd44vbv1VwzCn64bOp64geaZtZtmT8iUgHbTPLFn/3iJlZinikbWaWIh5pm5mlSCXfKZJ2Dtpmli1Oj5iZpYiDtplZijinbWaWIh5pm5mliG9EmpmliNMjZmYp4vSImVmKeKRtZpYiHmmbmaWIR9pmZini2SNmZini9IiZWYo4PWJmliIeaZuZpUjGR9rZvjoz2/lIhS+VHkplksZLmiXpbUlHSmokaYqkd5OfDZO6kjRc0mxJr0nqkHecPkn9dyX1qcnlOWibWbYUFxe+VO524H8j4mDgq8DbwFXA1IhoC0xN1gGOB9omSz9gFICkRsBg4BtAJ2Dw+kBfHQ7aZpYttTTSllQKHA3cBRARayJiGdAdGJtUGwv0SD53B+6NnOlAmaRmQFdgSkQsiYilwBTguOpenoO2mWWLigpeJPWTNDNv6Zd3pDbAYuBuSa9IulPS7kDTiJif1FkANE0+twA+ztt/TlJWUXm1+EakmWVLFWaPRMRoYHQFm0uADsDFEfGCpNv5Vypk/f4hKarb1OrwSNvMsqUKI+1KzAHmRMQLyfp4ckF8YZL2IPm5KNk+F2iVt3/LpKyi8mpx0DazbCkqKXzZiohYAHws6aCkqDPwFjARWD8DpA8wIfk8EeidzCI5AliepFEmA10kNUxuQHZJyqrF6REzy5bafbjmYuABSbsA7wE/IDfYfURSX+BD4JSk7iTgBGA2sCqpS0QskTQUeDGpNyQillS3QYqo43TMqsXbNN9jaZHtp9asmuo3rnHH+OLJywuOOUUn3py6juiRtpllS8afiHTQNrNs8XePmJmliEfaZmYpUsmskLTL9tWZ2c7H6REzsxRxesTMLEUctM3MUqTIQdvMLD38NnYzsxRxesTMLEU8e8TMLEU80jYzSxEHbTOzFHHQNjNLEc8eMTNLEY+0zcxSxLNHzMxSxCPtndP8BQu58ifX8cknS5HglO+fRJ8zTtmozt/f/5CrB1/Pm7P+xo8uOp++vc+o8XnXrFnDlT+5jjfffoey0gYMu3EILZs348/TX+SW4aNYu7acevVKuOLS/hzZqWONz2dVN/Da65n23J/Zq1FDnhx//xbrvDDzZa7/xe2Ul5fTsKyM++8aUaNz5vrF0KRflOb1ixncMvwO1q5dS7169dwvIPNBO9tXVwPFxcVcNeAiJv3ufh6+dzS/ffh3zP77+xvVKSttwI//51L69j6tysefM28+Z5930Wbl4x5/kgZ77smUiQ9zzpmncvPtowBoWFbKqNtu4olx93LDkEFcOWho9S7Maux73U7gzhG3Vrh9xcqV/PT6Wxh12438/tEHuP0X1xV87Mr7xSNJvxgJQMOyMkbddiNPjLsv6RdDqn5BWaOiwpdCDicVS3pF0pPJehtJL0iaLenh5KW/SPpSsj472d467xgDk/J3JHWtyeU5aFegyd6Naf+VgwDYY/f67NemNQsX/2OjOns1asih7b9CScnmf7BM+P1kep11Pt1PPYdrrruJdevWFXTeZ6b9iZ7djgeg67eP4fkZLxERtDv4QJo2aQxA2/3bsHr1atasWVOTS7RqOrzjYZSWNqhw+xNPTeE7nb9J82b7ALl+sl6uX5xH91P7VLFf/JGe3U4AttQv9gbcLzYoLi58KcwPgbfz1m8EhkXEAcBSoG9S3hdYmpQPS+ohqR1wGtAeOA4YKanaU1wctAswZ9583n7nb3z1kHYF1f/7ex/w1B+m8uDdo5jw8D0UFRXxxKQ/FLTvwkWLabZPEwBKSkrYc4/dWbps+UZ1Jj89jXYHH8guu+xStQuxbeKDDz9ixYqVnH3eRXzvjHN5/ImngPx+cQcTHh5bR/3iIPeLWhxpS2oJfBe4M1kXcCwwPqkyFuiRfO6erJNs75zU7w48FBGrI+J9YDbQqbqXV1BOW9JKYNPX0i8HZgKXRcR7m9TvB/QD+PUvb6bfub2r277t7tNVq7jk8h9z9eU/ZI89di9on+dnvMQbb71Dr7POA+Dz1as3jLb6DxjInLnzWbu2nPkLFtL91HMA6H3GyXy/+3crPfa7f3+Pm4ePYszIYdW7IKtz69at4823Z3HPr4fz+eerOa3Pf/HVQ9vz/IyZvPHWLHqdlRuYbd4v5uX1iz4A9D7jlCr0i5HuF1ClnHZ+rEqMjojReeu3AVcCeybrewHLIqI8WZ8DtEg+twA+BoiIcknLk/otgOl5x8zfp8oKvRF5W3Ki3wIiN9TfH3gZGAMck185uejcha9avGmwT421a8u55PJBdDu+C106f7Pg/SKCnt2O57JLLths24hbfw7kRu8Dr/kZ9935q422N22yN/MXLGKfpk0oLy9n5T8/pWFZKQALFi7iogFXc+PQQezbqtr/5lbH9mnShLLSUurvthv1d9uNr3c4jFl/m53XLy7cbJ/a6Rc/Yd9WLev+And0VZjyt1Gs2uwwOhFYFBEvSTqmVtpWCwr9lXRSRPw6IlZGxIrkQrtGxMNAw8p2TqOI4Mc//Tn7tfkyPzi7ajcaj+zUkclPT+OTJUsBWLZ8BXPnLSho32O/eRSPJX9OT356Gkcc3gFJrFi5kn4XX8Fll1xIx8MOrdrF2DbV+Zj/5KVXX6O8vJzPPvuc1954k/3btObITl+vQb/4Dx57YhKwvl903KRfXOB+sYGqsGzVUcBJkj4AHiKXFrkdKJO0fsDbEpibfJ4LtAJItpcCn+SXb2GfKit0pL1K0in8K4/TC/g8+ZzakfTWvPTqa0z4/WQObLv/hhTGgIv+i3kLFgJw+sk9WPyPT/j+mefxz08/pUhFjH1gHJMevZ8D9m/Dpf3P59wLf8QXEdQrKeaaqwbQovk+lZ63V48TuWLQUL5z0qmUNmjAsBuuBeD+hx7lo4/nMmL03YwYfTcAY0YN2+gml20bA64azIyXXmHpsmUc3bUHF1/Ql/Ly3F/Lp5/ck/33a81//vs3OOmUPhQViV49u3HgAfsBJP3i0qRflFSjX5yS9IufAuv7xZxN+sVtO3e/qKXH2CNiIDAQIBlpXx4RZ0oaRy4GPgT0ASYku0xM1p9Ptj8TESFpIvBbSbcCzYG2wIzqtksRlcdcSfuR+w1zJLkgPR34EbnfFh0j4k8V7pzi9IjVpWw/tWbVVL9xjTvGF288UnDMKTrklILOlxe0T0zi4UNAI+AV4KyIWC1pV+A+4GvAEuC09ff7JP0YOBcoBy6NiKcKv6JN2lJI0K4RB23bIgdt24LaCNpvjis8aLc/OXUdsaCctqQDJU2V9EayfqikQXXbNDOzapAKX1Ko0BuRvyGX21kLEBGvkZtBYma2gymqwpI+hd6IrB8RM7Txb6byiiqbmW03KR1BF6rQoP0PSfuTzBSR1AuYX2etMjOrruo/IZ4KhQbt/uQmoB8saS7wPnBmnbXKzKy6PNIGclP77gaeJTfNZQW5+Yj+SjEz27Fk/KtZCw3aE4Bl5B5bn1dnrTEzqzGPtAFaRsRxddoSM7PakPH0SKF/R/xF0r/VaUvMzGpDLb8EYUdT6Ej7P4BzJL0PrCb390dEhL+hxsx2KDV4v0AqFBq0j6/TVpiZ1ZaMp0cKCtoR8WFdN8TMrFY4aJuZpUk6c9WFctA2s2zxSNvMLEV8I9LMLEU80jYzS5GUzr8ulIO2mWWMR9pmZumR8fRItv+OMLOdTy09xi6plaRnJb0l6U1JP0zKG0maIund5GfDpFyShkuaLek1SR3yjtUnqf+upD41uTwHbTPLltr77pFy4LKIaAccAfSX1A64CpgaEW2Bqck65J4cb5ss/YBRkAvywGDgG0AnYPD6QF8dDtpmli21FLQjYn5EvJx8Xgm8DbQAugNjk2pjgR7J5+7AvZEzHSiT1AzoCkyJiCURsRSYAlT7W1MdtM0sY1TwIqmfpJl5S78tHlFqDXwNeAFoGhHrX7e4AGiafG4BfJy325ykrKLyavGNSDPLlirciIyI0eRepbiVw2kP4FHg0ohYkf+C84gISVHNllaLR9pmljGFj7QrPZJUj1zAfiAifpcUL0zSHiQ/FyXlc4FWebu3TMoqKq8WB20zyxap8GWrh5GAu4C3I+LWvE0Tyb0jl+TnhLzy3skskiOA5UkaZTLQRVLD5AZkl6SsWpweMbNsqb152kcBZwOvS3o1KbsauAF4RFJf4EPglGTbJOAEYDawCvgBQEQskTQUeDGpNyQillS3UYqo43TMqsXbNN9jaZHtByCsmuo3rnHHiEVvFhxz1KR96jqiR9pmli0ZfyLSQdvMMsZB28wsPTzSNjNLEX81q5lZmnikbWaWHk6PmJmlSbaDdraTP2ZmGeORtpllipweMTNLEc8eMTNLE4+0zczSw+kRM7M0cdA2M0sPj7TNzNLEQdvMLD08e8TMLEWyPdB20DazrMl21HbQNrNs8Y1IM7M0cdA2M0uPjN+IrPu3sdsGkvpFxOjt3Q7bsbhfWFVk+1fSjqff9m6A7ZDcL6xgDtpmZinioG1mliIO2tuW85a2Je4XVjDfiDQzSxGPtM3MUsRB28wsRfxwTS2QtBcwNVndB1gHLE7WO0XEmu3SMNsuJK0DXs8r6hERH1RQ958Rscc2aZhlgnPatUzStcA/I+LmvLKSiCjffq2ybakqgdhB26rK6ZE6IukeSXdIegG4SdK1ki7P2/6GpNbJ57MkzZD0qqRfSyreXu222idpD0lTJb0s6XVJ3bdQp5mk55I+8Iak/0zKu0h6Ptl3nCQH+J2cg3bdagn8e0QMqKiCpK8ApwJHRcRh5FIrZ26b5lkd2S0Jvq9Kegz4HOgZER2AbwG3SJt9Fd0ZwOSkD3wVeFVSY2AQ8O1k35lAhX3Jdg7OadetcRGxrpI6nYGOwIvJ/8e7AYvqumFWpz5Lgi8AkuoB10s6GvgCaAE0BRbk7fMiMCap+3hEvCrpm0A74M9J39gFeH7bXILtqBy069aneZ/L2fgvm12TnwLGRsTAbdYq29bOBPYGOkbEWkkf8K9/fwAi4rkkqH8XuEfSrcBSYEpEnL6tG2w7LqdHtp0PgA4AkjoAbZLyqUAvSU2SbY0kfXm7tNDqSimwKAnY3wI2+/dN/s0XRsRvgDvJ9ZXpwFGSDkjq7C7pwG3YbtsBeaS97TwK9Jb0JvAC8DeAiHhL0iDgD5KKgLVAf+DD7dZSq20PAE9Iep1cXnrWFuocA1whaS3wT6B3RCyWdA7woKQvJfUGkfQd2zl5yp+ZWYo4PWJmliIO2mZmKeKgbWaWIg7aZmYp4qBtZpYiDtpmZinioG1mliL/Dy2uiR5M4CEgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "cm3_cls = confusion_matrix(y_test, y3_cls_predict)\n",
    "df_cm3_cls = pd.DataFrame(cm3_cls, index=['pos', 'neg'], columns=['True', 'False'])\n",
    "ax = sn.heatmap(df_cm3_cls, cmap='Oranges', annot=True)\n",
    "cm3_cls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9789375"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy3_cls = accuracy_score(y_test, y3_cls_predict)\n",
    "accuracy3_cls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# -----------------------Kümeleme Algoritmalarını karşılaştırma-----------------------"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Accuracy olarak bakarsak."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Çok az bir farkla BIRCH ve Agglomerative Clustering daha iyi sonuç verdi."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdUAAAE/CAYAAAAQZlkTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUTUlEQVR4nO3df5Bdd32f8eeNhUPKj4RGaqaxJMsQMUQxYIwqSB2CU5xUNlQmwYA1NGDGtUpbM0lJGRxCFGqmSQidMKExBKdlDLQgbKCpCipOwo/gYmwkg3/JxkQRpJZDgjDExCGJMXz6xzmLr5Zd7ZX1kbRrPa8Zz957ztlzvrvn6j73nHv3OFWFJEk6fA871gOQJOmhwqhKktTEqEqS1MSoSpLUxKhKktTEqEqS1GTZsdrw8uXLa82aNcdq85IkPSg33HDDV6pqxVzzjllU16xZw65du47V5iVJelCS/Nl88zz9K0lSE6MqSVKTBaOa5O1Jvpzk1nnmJ8mbk+xJcnOS0/uHKUnS4jfNkeoVwMaDzD8bWDv+twV46+EPS5KkpWfBqFbVJ4CvHmSRc4F31uA64PuT/OOuAUqStFR0vKd6EnDnxP194zRJko4rR/WDSkm2JNmVZNf+/fuP5qYlSTriOqJ6F7Bq4v7Kcdp3qarLq2p9Va1fsWLOv5uVJGnJ6ojqduAl46eAnwHcU1VfalivJElLyoJXVEryHuBMYHmSfcCvAg8HqKrfBXYA5wB7gG8ALztSg5UkaTFbMKpVtXmB+QX8u7YRSZK0RHlFJUmSmhyzC+p3W3PJh471EHQYvvgbzzmq2/PxsrT5eNGhOJqPF49UJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJajJVVJNsTHJHkj1JLplj/uokH0vy2SQ3Jzmnf6iSJC1uC0Y1yQnAZcDZwDpgc5J1sxZ7LXBlVT0VOB94S/dAJUla7KY5Ut0A7KmqvVV1H7ANOHfWMgU8Zrz9fcCf9w1RkqSlYdkUy5wE3Dlxfx/w9FnLvA74gySvAB4JnNUyOkmSlpCuDyptBq6oqpXAOcC7knzXupNsSbIrya79+/c3bVqSpMVhmqjeBayauL9ynDbpQuBKgKr6FPAIYPnsFVXV5VW1vqrWr1ix4sGNWJKkRWqaqO4E1iY5JcmJDB9E2j5rmf8HPBsgyY8wRNVDUUnScWXBqFbV/cDFwNXA7Qyf8t2d5NIkm8bFfhG4KMlNwHuAC6qqjtSgJUlajKb5oBJVtQPYMWva1onbtwFn9A5NkqSlxSsqSZLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktTEqEqS1MSoSpLUxKhKktRkqqgm2ZjkjiR7klwyzzIvTHJbkt1J3t07TEmSFr9lCy2Q5ATgMuCngH3AziTbq+q2iWXWAr8EnFFVX0vyj47UgCVJWqymOVLdAOypqr1VdR+wDTh31jIXAZdV1dcAqurLvcOUJGnxmyaqJwF3TtzfN06b9ATgCUk+meS6JBu7BihJ0lKx4OnfQ1jPWuBMYCXwiSRPqqq/mlwoyRZgC8Dq1aubNi1J0uIwzZHqXcCqifsrx2mT9gHbq+qbVfUF4PMMkT1AVV1eVeurav2KFSse7JglSVqUponqTmBtklOSnAicD2yftczvMxylkmQ5w+ngvX3DlCRp8VswqlV1P3AxcDVwO3BlVe1OcmmSTeNiVwN3J7kN+Bjwqqq6+0gNWpKkxWiq91SragewY9a0rRO3C3jl+J8kScclr6gkSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUhOjKklSE6MqSVIToypJUpOpoppkY5I7kuxJcslBlnt+kkqyvm+IkiQtDQtGNckJwGXA2cA6YHOSdXMs92jg54HruwcpSdJSMM2R6gZgT1Xtrar7gG3AuXMs93rgDcDfNY5PkqQlY5qongTcOXF/3zjtO5KcDqyqqg81jk2SpCXlsD+olORhwG8BvzjFsluS7Eqya//+/Ye7aUmSFpVponoXsGri/spx2oxHA6cCH0/yReAZwPa5PqxUVZdX1fqqWr9ixYoHP2pJkhahaaK6E1ib5JQkJwLnA9tnZlbVPVW1vKrWVNUa4DpgU1XtOiIjliRpkVowqlV1P3AxcDVwO3BlVe1OcmmSTUd6gJIkLRXLplmoqnYAO2ZN2zrPsmce/rAkSVp6vKKSJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2MqiRJTYyqJElNjKokSU2mimqSjUnuSLInySVzzH9lktuS3JzkI0lO7h+qJEmL24JRTXICcBlwNrAO2Jxk3azFPgusr6onA+8DfrN7oJIkLXbTHKluAPZU1d6qug/YBpw7uUBVfayqvjHevQ5Y2TtMSZIWv2miehJw58T9feO0+VwI/J/DGZQkSUvRss6VJfmXwHrgWfPM3wJsAVi9enXnpiVJOuamOVK9C1g1cX/lOO0ASc4CfhnYVFV/P9eKquryqlpfVetXrFjxYMYrSdKiNU1UdwJrk5yS5ETgfGD75AJJngq8jSGoX+4fpiRJi9+CUa2q+4GLgauB24Erq2p3kkuTbBoXeyPwKOCqJDcm2T7P6iRJesia6j3VqtoB7Jg1bevE7bOaxyVJ0pLjFZUkSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWpiVCVJajJVVJNsTHJHkj1JLplj/vckee84//oka9pHKknSIrdgVJOcAFwGnA2sAzYnWTdrsQuBr1XVDwNvAt7QPVBJkha7aY5UNwB7qmpvVd0HbAPOnbXMucA7xtvvA56dJH3DlCRp8ZsmqicBd07c3zdOm3OZqrofuAf4gY4BSpK0VCw7mhtLsgXYMt69N8kdR3P7S9xy4CvHehBHSnzDoJuPFx0KHy+H5uT5ZkwT1buAVRP3V47T5lpmX5JlwPcBd89eUVVdDlw+xTY1S5JdVbX+WI9DS4OPFx0KHy99pjn9uxNYm+SUJCcC5wPbZy2zHXjpePs84KNVVX3DlCRp8VvwSLWq7k9yMXA1cALw9qraneRSYFdVbQf+G/CuJHuArzKEV5Kk40o8oFwakmwZT59LC/LxokPh46WPUZUkqYmXKZQkqYlRPYKS3Dtx+5wkn09ycpLXJakkPzwx/xfGaX4Cr1GS542/1yce5nouSPI7XeM6HOPPtG7i/qVJzmpa939I8rkkNybZmeQl4/SPP5jHZpLTkpzzIL7vh5K871C/73iX5FvjvrspyWeS/NNx+pokt463z0xyz7jc55L854nvf1SStyX50yQ3jPv96eO8e2dta9H8m1hMjOpRkOTZwJuBs6vqz8bJt3DgB7peAOw+2mM7DmwG/u/4dckYLw86n+cxXDIUgKraWlV/1LDNlwM/BWyoqtOAZwOHe2W004BDimqSZVX151V13mFu+3j0t1V1WlU9Bfgl4NfnWe6acR8/FXhukjPG6f+V4cOma6vqacDLGP6GVVMyqkdYkp8Afg94blX96cSs32e83GOSxzNcheorE9/300k+Nb7avCrJo8bpW8cjiFuTXD5zOcjxFeUbknx6PCJ+5jj9R8dpNya5Ocnao/KDLwLj7+zHGa5Nff7E9Iclecv4Kv0Pk+xIct4475xx+g1J3pzkg3Osd02Sj46/z48kWT1OvyLJW5Ncl2TveETw9iS3J7li4vvn27dfHPfhZ4AXJLlo3Nc3JXl/kn8wHnlsAt447tPHj9s9L8P/+OKqie2cOTP++bY5y2uAf1NVXweoqq9X1TtmL5QDz8CcN/OzJXnB+Li8KcknMvwJ3qXAi8axvijJI8ffyaeTfDbJzL+BC5JsT/JR4COzjqwuSPKBJB9O8idJfnNi+xeOj/dPJ/k9j5wO8BjgawdboKr+FrgROGl8Hno68Nqq+vY4/wtV9aEjPdCHEqN6ZH0PQzyfV1WfmzXv68CdSU5leMJ/78yMJMuB1wJnVdXpwC7glePs36mqf1JVpwLfCzx3Yp3LqmoD8AvAr47TXg789viqdD3DZSaPF+cCH66qzwN3J3naOP1ngTUMR3s/B/wYQJJHAG9jOKPwNGDFPOv9L8A7qurJwP9gOAsx47Hj+v49w99vvwn4UeBJGU6FHmzfAtxdVadX1TbgA+O+fgpwO3BhVV07rvdV4xHJ5Au1PwKenuSR4/0XAdum2CZJHgM8uqr2zvvbXNhW4J+P4900Xit8K/DecazvBX6Z4e/YNwA/yfDiYGa8pwPnVdWz5lj3aePP8ySGSK9K8kPArwDPAM4ADusU/0PE944vYD7HcNT5+oMtnOSxwFrgEwyP0xur6lsLrPvGJDcyvGDSLEb1yPomcC3DkdJctjEE9XnA/5yY/gyGJ/xPjg/el/LAZbF+MsP/Xu8W4J8x/EOY8YHx6w0M0QD4FPCaJK8GTh5fmR4vNjP8jhm/zpwC/nHgqqr6dlX9BfCxcfoTgb1V9YXx/nvmWe+PAe8eb79rXN+M/z1e+OQW4C+r6pbxVf9uhn1ysH0LEy+ugFOTXDPu6xdz4L7+LuN1tz8M/IsMVzZ7DvC/pthml08CVyS5iOFv2ufy08Al4zg+DjwCWD3O+8Oq+uo83/eRqrqnqv4OuI1h/BuAP66qr1bVN4Gr5vne48nM6d8nAhuBd86czZrlmUluYrga3tXjv4Np133a+CJ9a9+wHzqO6rV/j0PfBl7IcDrrNVX1a7PmfxB4I8NFNL4+8dgPwxPMAe8DjkdSbwHWV9WdSV7H8KQ04+/Hr99i3LdV9e4k1zM8we5I8q+r6qNtP+EileQfMrzoeFKSYniSrySvOsKbntkH3564PXN/GcO++a59O+FvJm5fwXCW46YkFwBnTrH9bcDFDO+L7aqqvx6fVA+2TcbH371JHjfF0erk3+F95/FXVS/P8KGW5wA3TJwZmBTg+VV1wHW/x+/7mzmWnzH5u/zO41vzq6pPjWcp5jrjck1VPTfJKcB1Sa5keOH3lCQnHORoVQvwSPUIq6pvMDzJvDjJhXPMezXwn2Z923XAGRk/HTy+D/UEHngC+8r4ntiCH+RI8jiGo683Mxy1PPlwfp4l5DzgXVV1clWtqapVwBeAZzIcUT0/w3urP8gDsboDeFySNeP9F82z7mt54D3aFwPXHMK45tu3c3k08KUkDx+3M+Ovx3lz+WOG06gX8cBR+rTb/HXgsvFU8MwnQV8yx3J/meRHkjwM+JmZiUkeX1XXV9VWYD/D9cBnj/Vq4BUzR09JnjrPzzGNncCzkjx2PDJ//mGs6yEnwyfeT2CO67DPGM/K/Abw6vGthF3Af5zYP2uSPOdojPehwqgeBeMprY3Aa5NsmjVvW1V9Zta0/cAFwHuS3MxwCveJVfVXDB96upXhyWnnFJt/IXDreLrtVOCdh/XDLB2bOfCUOsD7x+nvZ3hv+TbgvwOfAe4ZT43/W+DDSW5gCMI9c6z7FcDLxn3zc8DPTzuo+fbtPIv/CnA9w4uAyffktwGvGj/o8/hZ6/8WwxmQs8evh7LNtzKcCt85fkjoGoYj7NkuGdd9LfClielvTHLL+L3XAjeN61s380Elhvf4Hg7cnGQ3C7zndzBVdRfwa8CnGX5HX2Tu/XU8+c77ngxvJbx0iqPO3wV+Ynwx+a+AHwT2jPvxCuDLR264Dz1eUUnHpSSPqqp7k/wAw5PyGVX1FxPTA1wG/ElVvenYjlbzmdhfyxheRL29qma/mJKOGt+X0PHqg0m+HzgReP3EBzUuSvLScfpnGT4NrMXrdRkufPEI4A8YPm0vHTMeqUqS1MT3VCVJamJUJUlqYlQlSWpiVCVJamJUJUlqYlQlSWry/wHCBRjuTSFLLwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "(0.976, 0.9789375, 0.9789375)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_axes([0,0,1,1])\n",
    "ax.bar(['KMeans', 'Agglomerative Clustering', 'BIRCH'],[accuracy_cls, accuracy2_cls, accuracy3_cls])\n",
    "plt.show()\n",
    "accuracy_cls, accuracy2_cls, accuracy3_cls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ama veri setimizde 59000 negatif kısma 1000 tane pozitif kısma ait veri olduğu için hepsini negatif seçsek bile accuracy yüksekmiş gibi durucak. Bu yüzden accuracy den çok veri setimizin açıklamasında da bahsedilen cost denklemini kullanalım."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Total cost = NTx500 + PNx10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confusion Matrix olarak bakarsak."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "55290"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost1 = cm_cls[0][1]*10 + cm_cls[1][0]*500\n",
    "cost1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "108230"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost2 = cm2_cls[0][1]*10 + cm2_cls[1][0]*500\n",
    "cost2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "108230"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost3 = cm3_cls[0][1]*10 + cm3_cls[1][0]*500\n",
    "cost3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cost değerlerini görüyoruz. Düşük olan daha iyi olduğu için Kmeans daha iyi sonuç verdiğini söyleyebiliriz."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAesAAAE/CAYAAABmXOuYAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAX5klEQVR4nO3dfbRldX3f8fdHRiJqEZBZRGeIgzqtRRIfmIWkRGskxQGM0OX4wCJhtBSaitZ2tYnoSiX1IdEmrQ2N0hCZAjYREU0gOjqdgEQT5WFGnkHDDYgMBRkdBNH4AH77x/5dOV7vHWbuucz93bnv11pnnd/+7t/e+3fOPnM/Z++zz5lUFZIkqV+Pm+8BSJKk7TOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzi2Z7wHMtf33379WrFgx38OQJGmnbN68+RtVtXS6ebtdWK9YsYJNmzbN9zAkSdopSe6YaZ6nwSVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUud3uP/KQ5tOK0z8130PQGL763mN36fZ8vSxsu/L14pG1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXOGtSRJnTOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzjxrWSdYluTfJjSO1/ZJsTHJru9+31ZPkzCQTSa5P8sKRZda2/rcmWTtSPzTJDW2ZM5Nke9uQJGmx2ZEj63OB1VNqpwOXVtVK4NI2DXA0sLLdTgXOgiF4gTOAFwGHAWeMhO9ZwCkjy61+lG1IkrSoPGpYV9XngG1TyscB57X2ecDxI/Xza3AFsE+SpwEvBzZW1baqug/YCKxu8/auqiuqqoDzp6xrum1IkrSozPYz6wOq6u7Wvgc4oLWXAXeO9NvSaturb5mmvr1t/JQkpybZlGTT1q1bZ/FwJEnq19gXmLUj4pqDscx6G1V1dlWtqqpVS5cufSyHIknSLjfbsP56O4VNu7+31e8CDhzpt7zVtldfPk19e9uQJGlRmW1YXwJMXtG9Frh4pH5Suyr8cOD+dip7A3BUkn3bhWVHARvavAeSHN6uAj9pyrqm24YkSYvKkkfrkOQjwEuB/ZNsYbiq+73AhUlOBu4AXtO6rweOASaA7wJvAKiqbUneBVzd+r2zqiYvWnsjwxXnewGfbje2sw1JkhaVRw3rqjphhllHTtO3gNNmWM86YN009U3AIdPUvzndNiRJWmz8BTNJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzhrUkSZ0zrCVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXOGtSRJnTOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzhrUkSZ0zrCVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOjdWWCf5D0luSnJjko8keUKSg5JcmWQiyUeT7Nn6/kybnmjzV4ys522t/pUkLx+pr261iSSnjzNWSZIWqlmHdZJlwL8DVlXVIcAewOuA9wHvr6pnA/cBJ7dFTgbua/X3t34kObgt91xgNfDBJHsk2QP4AHA0cDBwQusrSdKiMu5p8CXAXkmWAE8E7gZeBlzU5p8HHN/ax7Vp2vwjk6TVL6iq71fV7cAEcFi7TVTVbVX1A+CC1leSpEVl1mFdVXcBfwB8jSGk7wc2A9+qqodaty3AstZeBtzZln2o9X/qaH3KMjPVJUlaVMY5Db4vw5HuQcDTgScxnMbe5ZKcmmRTkk1bt26djyFIkvSYGec0+K8At1fV1qr6IfAJ4Ahgn3ZaHGA5cFdr3wUcCNDmPwX45mh9yjIz1X9KVZ1dVauqatXSpUvHeEiSJPVnnLD+GnB4kie2z56PBG4GPgusaX3WAhe39iVtmjb/sqqqVn9du1r8IGAlcBVwNbCyXV2+J8NFaJeMMV5JkhakJY/eZXpVdWWSi4AvAQ8B1wBnA58CLkjy7lY7py1yDvDhJBPANobwpapuSnIhQ9A/BJxWVQ8DJHkTsIHhSvN1VXXTbMcrSdJCNeuwBqiqM4AzppRvY7iSe2rf7wGvnmE97wHeM019PbB+nDFKkrTQ+QtmkiR1zrCWJKlzhrUkSZ0zrCVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXOGtSRJnTOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzhrUkSZ0zrCVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXNjhXWSfZJclOTLSW5J8otJ9kuyMcmt7X7f1jdJzkwykeT6JC8cWc/a1v/WJGtH6ocmuaEtc2aSjDNeSZIWonGPrP8Q+ExVPQd4HnALcDpwaVWtBC5t0wBHAyvb7VTgLIAk+wFnAC8CDgPOmAz41ueUkeVWjzleSZIWnFmHdZKnAC8BzgGoqh9U1beA44DzWrfzgONb+zjg/BpcAeyT5GnAy4GNVbWtqu4DNgKr27y9q+qKqirg/JF1SZK0aIxzZH0QsBX430muSfKhJE8CDqiqu1ufe4ADWnsZcOfI8ltabXv1LdPUJUlaVMYJ6yXAC4GzquoFwHd45JQ3AO2IuMbYxg5JcmqSTUk2bd269bHenCRJu9Q4Yb0F2FJVV7bpixjC++vtFDbt/t42/y7gwJHll7fa9urLp6n/lKo6u6pWVdWqpUuXjvGQJEnqz6zDuqruAe5M8k9a6UjgZuASYPKK7rXAxa19CXBSuyr8cOD+drp8A3BUkn3bhWVHARvavAeSHN6uAj9pZF2SJC0aS8Zc/s3AnybZE7gNeAPDG4ALk5wM3AG8pvVdDxwDTADfbX2pqm1J3gVc3fq9s6q2tfYbgXOBvYBPt5skSYvKWGFdVdcCq6aZdeQ0fQs4bYb1rAPWTVPfBBwyzhglSVro/AUzSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSercuD+Ksltbcfqn5nsIGtNX33vsfA9BksbmkbUkSZ0zrCVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXOGtSRJnTOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzhrUkSZ0zrCVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSerc2GGdZI8k1yT5ZJs+KMmVSSaSfDTJnq3+M216os1fMbKOt7X6V5K8fKS+utUmkpw+7lglSVqI5uLI+i3ALSPT7wPeX1XPBu4DTm71k4H7Wv39rR9JDgZeBzwXWA18sL0B2AP4AHA0cDBwQusrSdKiMlZYJ1kOHAt8qE0HeBlwUetyHnB8ax/Xpmnzj2z9jwMuqKrvV9XtwARwWLtNVNVtVfUD4ILWV5KkRWXcI+v/AfwW8KM2/VTgW1X1UJveAixr7WXAnQBt/v2t/4/rU5aZqf5TkpyaZFOSTVu3bh3zIUmS1JdZh3WSVwD3VtXmORzPrFTV2VW1qqpWLV26dL6HI0nSnFoyxrJHAK9McgzwBGBv4A+BfZIsaUfPy4G7Wv+7gAOBLUmWAE8BvjlSnzS6zEx1SZIWjVkfWVfV26pqeVWtYLhA7LKqOhH4LLCmdVsLXNzal7Rp2vzLqqpa/XXtavGDgJXAVcDVwMp2dfmebRuXzHa8kiQtVOMcWc/krcAFSd4NXAOc0+rnAB9OMgFsYwhfquqmJBcCNwMPAadV1cMASd4EbAD2ANZV1U2PwXglSeranIR1VV0OXN7atzFcyT21z/eAV8+w/HuA90xTXw+sn4sxSpK0UPkLZpIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXOGtSRJnTOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzhrUkSZ0zrCVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXOGtSRJnTOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzsw7rJAcm+WySm5PclOQtrb5fko1Jbm33+7Z6kpyZZCLJ9UleOLKuta3/rUnWjtQPTXJDW+bMJBnnwUqStBCNc2T9EPAfq+pg4HDgtCQHA6cDl1bVSuDSNg1wNLCy3U4FzoIh3IEzgBcBhwFnTAZ863PKyHKrxxivJEkL0qzDuqrurqovtfa3gVuAZcBxwHmt23nA8a19HHB+Da4A9knyNODlwMaq2lZV9wEbgdVt3t5VdUVVFXD+yLokSVo05uQz6yQrgBcAVwIHVNXdbdY9wAGtvQy4c2SxLa22vfqWaerTbf/UJJuSbNq6det4D0aSpM6MHdZJngx8HPj3VfXA6Lx2RFzjbuPRVNXZVbWqqlYtXbr0sd6cJEm71FhhneTxDEH9p1X1iVb+ejuFTbu/t9XvAg4cWXx5q22vvnyauiRJi8o4V4MHOAe4par++8isS4DJK7rXAheP1E9qV4UfDtzfTpdvAI5Ksm+7sOwoYEOb90CSw9u2ThpZlyRJi8aSMZY9Avh14IYk17ba24H3AhcmORm4A3hNm7ceOAaYAL4LvAGgqrYleRdwdev3zqra1tpvBM4F9gI+3W6SJC0qsw7rqvobYKbvPR85Tf8CTpthXeuAddPUNwGHzHaMkiTtDvwFM0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXOGtSRJnTOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzhrUkSZ0zrCVJ6pxhLUlS5wxrSZI6Z1hLktQ5w1qSpM4Z1pIkdc6wliSpc4a1JEmdM6wlSeqcYS1JUucMa0mSOmdYS5LUOcNakqTOGdaSJHXOsJYkqXOGtSRJnTOsJUnqnGEtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlzhrUkSZ0zrCVJ6pxhLUlS5wxrSZI6131YJ1md5CtJJpKcPt/jkSRpV+s6rJPsAXwAOBo4GDghycHzOypJknatrsMaOAyYqKrbquoHwAXAcfM8JkmSdqnew3oZcOfI9JZWkyRp0Vgy3wOYC0lOBU5tkw8m+cp8jmeB2R/4xnwP4rGS9833CHY7vl60M3y97JxnzDSj97C+CzhwZHp5q/2EqjobOHtXDWp3kmRTVa2a73FoYfD1op3h62Xu9H4a/GpgZZKDkuwJvA64ZJ7HJEnSLtX1kXVVPZTkTcAGYA9gXVXdNM/DkiRpl+o6rAGqaj2wfr7HsRvz4wPtDF8v2hm+XuZIqmq+xyBJkraj98+sJUla9AzrBSjJgyPtY5L8XZIZL/nX3EtyfJJK8pwx1/P6JH80V+MaR3tMB49MvzPJr8zRuv9Tki8nuTbJ1UlOavXLk+z01cJJnp/kmFks9/QkF+3scotdkofbvrsuyZeS/LNWX5HkxtZ+aZL7W78vJ/mDkeWfnOSPk/x9ks1tv7+ozXtwyra6+TfRE8N6AUtyJHAmcHRV3THf41lkTgD+pt0vGO0nfGdyPMPP+gJQVe+oqr+ag23+BvAvgMOq6vnAkUDGXO3zgZ0K6yRLqur/VdWaMbe9GP1DVT2/qp4HvA34vRn6fb7t4xcAr0hyRKt/CNgGrKyqQ4E3MHwHWzvIsF6gkrwE+BPgFVX19612bpKzklyR5Lb2TnddkluSnDuy7FFJvtjeIX8syZNb/R3tqOfGJGcnSatfnuR9Sa5qR/EvbvXnttq1Sa5PsnKXPxHzoD1fvwSczPB1wsn645J8sB1VbEyyPsmaNu+YVt+c5Mwkn5xmvSuSXNaey0uT/Fyrj7tfv9r235eAVyc5pe3n65J8PMkT25HSK4Hfb/vzWW27azL8ZzofG9nOSyfHP9M2p3g78G+r6gGAqnqgqs6b5vGPnjFaM/nYkry6vSavS/K5DF/jfCfw2jbW1yZ5UntOrkpyTZLj2rKvT3JJksuAS6ccCb4+ySeSfCbJrUn+68j2T26v9auS/IlHej9hb+C+7XWoqn8ArgWWJXkW8CLgt6vqR23+7VX1qcd6oLuVqvK2wG7ADxnepf7ClPq5DL+fHobfUH8A+HmGN2WbGY5G9gc+BzypLfNW4B2tvd/Iuj4M/GprXw78t9Y+Bvir1v6fwImtvSew13w/N7vo+T8ROKe1vwAc2tprGL658DjgZxn+oK0BnsDws7kHtX4fAT7Z2q8H/qi1/xJY29r/CviLOdqvXwV+a2T8Tx1pvxt488h21kx5Pa1h+NbI10bWfRbwa9vb5sg69gbu285zeTmwqrUfHKmvAc5t7RuAZa29z9TnrU3/LvBrk32AvwOe1PptmXxtAyuAG0fWcRvwlLaP7mD4Eaant+dsP+DxwOdHt7UYb8DDDOH7ZeB+HnnNjz6fLx15Xe/bXps/y/Am8M93YN2Tt68t9ud7uptH1gvTDxlC4uRp5v1lDf8CbgC+XlU31PBu9iaGf1iHM5zq/Nsk1wJreeQn7n45yZVJbgBeBjx3ZL2faPeb23oAvgi8PclbgWfU8G56MTiBITxp95Onwn8J+FhV/aiq7gE+2+rPAW6rqtvb9EdmWO8vAn/W2h9u65s0zn4F+OhI+5Akn2/7+UR+cj//lKp6CPgM8KtJlgDHAhfvwDbnyt8C5yY5heH3FqZzFHB6G8flDOH7c23exqraNsNyl1bV/VX1PeBmhvEfBvx1VW2rqh8CH5th2cVk8jT4c4DVwPmTZ96meHGS6xh+aXJD+3ewo+t+fg2n0N8xd8PefXT/PWtN60fAaxhO6729qn53ZN73R/p8f8oySxjexW6sqp/4rDXJE4APMhzl3Jnkdxj+4E1d78NtPVTVnyW5kuGP9/ok/6aqLpuLB9irJPsxvJH5+STFEB6V5Dcf403Par+O+M5I+1zg+Kq6LsnrGY6IHs0FwJsYzuhsqqpvtz/W29smVfVAkgeTPLOqbnuUbYx+j/THr72q+o0MFyMdC2xOcug0ywZ4VVX9xP8L0Jb7zjT9J40+lz9+bWtmVfXFJPsDS6eZ/fmqekWSg4ArklzI8IbyeUn2qKqHd+lgdyMeWS9QVfVdhj9eJyaZ7gh7JlcARyR5NkD7rO8f88gfx2+0zx0f9SKcJM9kOGI8k+FI6xd25jEsUGuAD1fVM6pqRVUdCNwOvJjhCPBV7bPrA3gkBL8CPDPJijb92hnW/QUe+Qz8RIbTrztqpv06nX8E3J3k8W07k77d5k3nr4EXAqfwyFmFHd3m7wEfSLJ36/fktKvBp/h6kn+a5HHAv5wsJnlWVV1ZVe8AtjKcqp461g3AmyeP9pK8YIbHsSOuBv55kn3bmYRXjbGu3U6Gb0DsAXxzpj7tLNJ7gbfWcE3NJuC/jOyfFUmO3RXj3V0Y1gtYO7W3GvjtJK/cwWW2MnxW95Ek1zOcyn5OVX2L4YK1Gxn+8F29A6t7DXBjO/V4CHD+Tj6EhegE4M+n1D7e6h9n+Hz0ZuD/AF8C7m8fD7wR+EySzQxBc/80634z8Ia2X34deMuODmqm/TpD9/8MXMnw5uLLI/ULgN9sF2g9a8r6HwY+CRzd7ndmm2cxfCRwdbu46/MMZwSmOr2t+wvA3SP1309yQ1v2C8B1bX0HT15gBryL4fPl65Pc1KZnparuYvgM/CqG5+irTL+/FpO92nN9LcNHKmt34Cj5fwEvaW9S/zVwADDR9uO5wL2P3XB3P/6CmTSHkjy5qh5M8lSGP/ZHVNU9I/UAHwBurar3z+9oNZOR/bWE4c3Zuqqa+iZN2mX8fEaaW59Msg/D1fHvGrnA5pQka1v9GuCP52l82jG/k+EHYZ4A/F/gL+Z3OFrsPLKWJKlzfmYtSVLnDGtJkjpnWEuS1DnDWpKkzhnWkiR1zrCWJKlz/x/+QZRS7UmbBAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_axes([0,0,1,1])\n",
    "ax.bar(['Kmeans', 'Agglomerative Clustering', 'BIRCH'],[cost1, cost2, cost3])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Presicion, Recall, Fscore, Support"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KMeans \n",
      "precision: [0.99320432 0.49180328]\n",
      "recall: [0.982144 0.72    ]\n",
      "fscore: [0.9876432  0.58441558]\n",
      "support: [15625   375]\n"
     ]
    }
   ],
   "source": [
    "precision, recall, fscore, support = score(y_test, y_cls_predict)\n",
    "print(\"KMeans \")\n",
    "print('precision: {}'.format(precision))\n",
    "print('recall: {}'.format(recall))\n",
    "print('fscore: {}'.format(fscore))\n",
    "print('support: {}'.format(support))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Agglomerative Clustering \n",
      "precision: [0.9863833  0.56690141]\n",
      "recall: [0.992128   0.42933333]\n",
      "fscore: [0.98924731 0.48861912]\n",
      "support: [15625   375]\n"
     ]
    }
   ],
   "source": [
    "precision, recall, fscore, support = score(y_test, y2_cls_predict)\n",
    "print(\"Agglomerative Clustering \")\n",
    "print('precision: {}'.format(precision))\n",
    "print('recall: {}'.format(recall))\n",
    "print('fscore: {}'.format(fscore))\n",
    "print('support: {}'.format(support))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "BIRCH \n",
      "precision: [0.9863833  0.56690141]\n",
      "recall: [0.992128   0.42933333]\n",
      "fscore: [0.98924731 0.48861912]\n",
      "support: [15625   375]\n"
     ]
    }
   ],
   "source": [
    "precision, recall, fscore, support = score(y_test, y3_cls_predict)\n",
    "print(\"BIRCH \")\n",
    "print('precision: {}'.format(precision))\n",
    "print('recall: {}'.format(recall))\n",
    "print('fscore: {}'.format(fscore))\n",
    "print('support: {}'.format(support))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  &&&&&&&&&&&&&&& Genel Sonuç &&&&&&&&&&&&&&&&&"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Genel olarak en iyi sonucu SGDClassifier'ın verdiğini söyleyebiliriz. Supervised ve tembel olmayan bir öğrenim yaptığından dolayı bu veri setinde doğal olarak kümeleme algoritmalarına karşı daha iyi sonuçlar verdi. Veri sayımızın yüksek olması SGDClassifier algoritmalarının daha iyi sonuçlar vermesini sağladı. Precision ve f1score olarakta en iyi sayılar SGDCLassifier'a ait."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}