Data Preprocessing.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import os\n",
    "import shutil"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get Datta from github : https://github.com/ieee8023/covid-chestxray-dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Description"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### clone the repository by using this link. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Here we have to do some preprocessing steps to get COVID-19 images.In the file chestxray/metadata.csv you have information about the patients having different diseases with corresponding chest X-rays."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### So Amoung from these we have retrive only diseases which is related to COVID-19.Here is the some preprocessing steps to follow to get those images  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Creating the data for positive samples (Github)\n",
    "\n",
    "File_Path = \"chestxray/metadata.csv\"\n",
    "Images_path = \"chestxray/images\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(File_Path)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Here we are having column finding.from that column we need to retrive finding== COVID-19 diseases"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "target_dir = 'Dataset/Covid'\n",
    "\n",
    "if not os.path.exists(target_dir):\n",
    "    os.mkdir(target_dir)\n",
    "    print('COVID folder created')\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "count=0 \n",
    "\n",
    "for(i,row) in df.iterrows():\n",
    "    if row['finding'] == 'COVID-19':\n",
    "        count + =1\n",
    "        \n",
    "print(count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### here we got count=278 which are related to COVID-19 cases.In that we have to different kinds of views of X-rays(i.e front view,top view etc...).so here we are considering only front view images."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### the view we can call it as posteroanterior"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "count=0 \n",
    "\n",
    "for(i,row) in df.iterrows():\n",
    "    if row['finding'] == 'COVID-19' and row['view']=='PA':\n",
    "        filename = row['filename']\n",
    "        image_path = os.path.join(Images_path,filename)\n",
    "        image_copy_path = os.path.join(target_dir,filename)\n",
    "        shutil.copy2(image_path,image_copy_path)\n",
    "        #print(\"Moving Image\",count)\n",
    "        count +=1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Get the normal patient X-ray images from the kaggle :https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Here were having three categories of diseases :Normal,BacteriaPneumonia,ViralPneumonia.Amoung there we want only the noraml X-ray images.To retrive those images we have to follw these preprocessing steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "Kaggle_file_path = 'chest_xray_kaggle/train/NORMAL' #set your file path\n",
    "Target_normal_dir = 'Dataset/Normal'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_names = os.listdir(Kaggle_file_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "random.shuffle(image_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### here we are having large amount of images i have taken only 142,why because we have COVID-19 images only 142"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(142): \n",
    "    image_name = image_names[i]\n",
    "    image_path = os.path.join(Kaggle_file_path,image_name)\n",
    "    target_path = os.path.join(Target_normal_dir,image_name)    \n",
    "    shutle.copy2(image_path,target_path)\n",
    "    print(\"Copying image\",i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now we are ready with our datset.lets build a CNN Model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### we are working with images so better to work on google-colab notebook for faster performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}