class3_file-io.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Session 3: File Input/Output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O: Basic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Data on a computer is usually stored in **files**\n",
    "* From the view of the operating system, a file is just a sequence of\n",
    "    bits that is given a name\n",
    "* What data is stored in a file and how exactly it is stored in a file\n",
    "    is defined in a **file format**\n",
    "* The file format defines what the bits mean to the program that is reading/writing\n",
    "    the file\n",
    "* ***Note:*** The **file extension** (e.g. whether the name of a file ends\n",
    "    in **.txt** or **.doc** does not determine the file format (it is\n",
    "    just a name) -- but it makes sense to name files according to their format"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O: Writing to a Text File"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* A very common and useful file format is one where the sequence of bits\n",
    "    is interpreted as sequence of characters\n",
    "* This conversion is performed with respect to a character set\n",
    "    (such as ASCII or UTF-8, but let's not worry about that here...)\n",
    "* In Python, such **text files** can be manipulated very easily, by\n",
    "    reading/writing their contents to/from strings\n",
    "* Using the `open()` function one can obtain a reference to a\n",
    "    **file** object that provides methods for reading and writing (e.g.\n",
    "    `read()` and `write()`)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O: Text Files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### File I/O: Writing to a text file:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Opening a text file for writing \n",
    "자세한 사항은 다음을 참조하세요\n",
    "https://wikidocs.net/26"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 309,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "path = \"C:\\\\Users\\\\user\\\\Dropbox\\\\Class\\\\2020_2\\\\programming\\\\week2\"\n",
    "os.chdir(path) \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 310,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = open('my_first_file.txt', 'w') # writing w:erase and write, r:read only, a: add to the previous version of the file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 311,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "11"
      ]
     },
     "execution_count": 311,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f.write('Hello world!\\n')\n",
    "f.write(\"Hello There\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 312,
   "metadata": {},
   "outputs": [],
   "source": [
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now read this file again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "ename": "FileNotFoundError",
     "evalue": "[Errno 2] No such file or directory: 'my_first_file.txt'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-4-21dd91463e9a>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mf\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mopen\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m'my_first_file.txt'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'r'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: 'my_first_file.txt'"
     ]
    }
   ],
   "source": [
    "f = open('my_first_file.txt', 'r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = open('my_first_file.txt', 'w')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 314,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello world!\n",
      "\n"
     ]
    }
   ],
   "source": [
    "line = f.readline() #한줄씩 읽음\n",
    "print(line)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 315,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello There\n"
     ]
    }
   ],
   "source": [
    "line2 = f.readline() # 그 다음줄\n",
    "print(line2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 316,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "\n",
    "line3 = f.readline()\n",
    "print(line3)\n",
    "#f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write can be called multiple times to write more data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I am Jaeyoun teaching this course. I am very kind Man.\n"
     ]
    }
   ],
   "source": [
    "print(\"I am {0} teaching this course. I am {2} kind {1}.\".format(\"Jaeyoun\", \"Man\", \"very\") )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I am Jaeyoun, and I hate music\n",
      "I am Jaeyoun\n",
      "I am 12\n"
     ]
    }
   ],
   "source": [
    "print(\"I am {0}, and I {1} music\".format(\"Jaeyoun\", \"hate\"))\n",
    "print(\"I am %s\" %\"Jaeyoun\")\n",
    "print(\"I am %d\" %12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Animal\tFood\n",
      "\n",
      "Sloth\tLeaves\n",
      "\n",
      "Chicken\tCorn\n",
      "\n",
      "Ant_eater\tAnts\n",
      "\n",
      "Penguin\tFish\n",
      "\n",
      "Armadillo\tIce_cream\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "f = open(\"animals.txt\", \"w\")\n",
    "for animal in [\"Animal\\tFood\",\"Sloth\\tLeaves\", \"Chicken\\tCorn\", \"Ant_eater\\tAnts\", \"Penguin\\tFish\", \"Armadillo\\tIce_cream\\n\"]:\n",
    "    print(\"%s\\n\" % animal)\n",
    "    f.write(\"%s\\n\" % animal)\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Animal\tFood\n",
      "\n",
      "Sloth\tLeaves\n",
      "\n",
      "Chicken\tCorn\n",
      "\n"
     ]
    }
   ],
   "source": [
    "f=open(\"animals.txt\", \"r\")\n",
    "line = f.readline()\n",
    "print(line)\n",
    "line2 = f.readline()\n",
    "print(line2)\n",
    "line3 = f.readline()\n",
    "print(line3)\n",
    "f.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O: Reading from a Text File:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Reading the content of a text file using the `readlines()` function:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `readlines()` function reads an entire text file into a list of strings, where each list entry corresponds to a line in the file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "f = open(\"animals.txt\", \"r\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "lines = f.readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Animal\\tFood\\n', 'Sloth\\tLeaves\\n', 'Chicken\\tCorn\\n', 'Ant_eater\\tAnts\\n', 'Penguin\\tFish\\n', 'Armadillo\\tIce_cream\\n', '\\n']\n"
     ]
    }
   ],
   "source": [
    "print(lines)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use of `strip` commands."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    a b c db\n",
      "a b c db\n"
     ]
    }
   ],
   "source": [
    "a=\"    a b c d\"+\"b\"\n",
    "print(a)\n",
    "print(a.lstrip(\" \"))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 325,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'abcd    b'"
      ]
     },
     "execution_count": 325,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a=\"abcd    \"   \n",
    "a+\"b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "abcdb\n"
     ]
    }
   ],
   "source": [
    "a=\"abcd    \"    \n",
    "b=a.rstrip(\" \")+\"b\"\n",
    "print(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now print each line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " A B C D \n",
      "A B C D \n",
      " A B C DE\n",
      "A B C DE\n"
     ]
    }
   ],
   "source": [
    "str = \" A B C D \"   \n",
    "print(str)\n",
    "print( str.lstrip())       \n",
    "print( str.rstrip() + \"E\") \n",
    "print( str.strip() + \"E\")  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `print` statement inserts `\\n` after automatically, without removing the already present `\\n` characters with `rstrip()` we end up with empty lines!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Reading the content of a text file line by line:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Because processing each line in a file is such a common operation,\n",
    "  Python provides the following simple syntax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 328,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Animal\\tFood\\n', 'Sloth\\tLeaves\\n', 'Chicken\\tCorn\\n', 'Ant_eater\\tAnts\\n', 'Penguin\\tFish\\n', 'Armadillo\\tIce_cream\\n', '\\n']\n"
     ]
    }
   ],
   "source": [
    "f = open(\"animals.txt\", \"r\")\n",
    "a=list(f)\n",
    "print(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 329,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Animal\\tFood\\n', 'Sloth\\tLeaves\\n', 'Chicken\\tCorn\\n', 'Ant_eater\\tAnts\\n', 'Penguin\\tFish\\n', 'Armadillo\\tIce_cream\\n', '\\n']\n"
     ]
    }
   ],
   "source": [
    "f = open(\"animals.txt\", \"r\")\n",
    "lines = f.readlines()\n",
    "print(lines)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 330,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Animal\tFood\n",
      "Sloth\tLeaves\n",
      "Chicken\tCorn\n",
      "Ant_eater\tAnts\n",
      "Penguin\tFish\n",
      "Armadillo\tIce_cream\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for line in a:\n",
    "    print(line.rstrip())\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This iterates over the file line by line instead of reading in the whole content in the beginning!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### And because python makes your life easy, here an even shorter version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 331,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Animal\tFood\n",
      "Sloth\tLeaves\n",
      "Chicken\tCorn\n",
      "Ant_eater\tAnts\n",
      "Penguin\tFish\n",
      "Armadillo\tIce_cream\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#R: print(aaa$bbb)  -->>    with(aaa, print(bbb)) \n",
    "with open(\"animals.txt\", \"r\") as infile10:  #infile = open(\"animals.txt\", \"r\")\n",
    "    for line in infile10:\n",
    "        print(line.rstrip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using `with` removes the necessity to call the `close()` function on your file object!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O: Transforming a File:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* When working with data provided by other programs (and/or other\n",
    "    people), it is often necessary to convert data from one format to another\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The file that we wrote contained columns separated by tabs; what if we\n",
    "    need commas? --> This is hard for Jaeyoun, hence leave it for exercise for student."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 336,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Animal\tFood\n",
      " ['Animal', 'Food'] Animal,Food\n",
      "Sloth\tLeaves\n",
      " ['Sloth', 'Leaves'] Sloth,Leaves\n",
      "Chicken\tCorn\n",
      " ['Chicken', 'Corn'] Chicken,Corn\n",
      "Ant_eater\tAnts\n",
      " ['Ant_eater', 'Ants'] Ant_eater,Ants\n",
      "Penguin\tFish\n",
      " ['Penguin', 'Fish'] Penguin,Fish\n",
      "Armadillo\tIce_cream\n",
      " ['Armadillo', 'Ice_cream'] Armadillo,Ice_cream\n",
      "\n",
      " [] \n"
     ]
    }
   ],
   "source": [
    "with open(\"animals.txt\", \"r\") as infile:\n",
    "    for line in infile:\n",
    "        print(line, line.split(), \",\".join(line.split())  )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "with open(\"animals.txt\", \"r\") as infile:\n",
    "    with open(\"animals.csv\", \"w\") as outfile:\n",
    "              for line in infile:\n",
    "                outfile.write(\",\".join(line.split()))\n",
    "                outfile.write('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets check everything worked..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Animal,Food\n",
      "Sloth,Leaves\n",
      "Chicken,Corn\n",
      "Ant_eater,Ants\n",
      "Penguin,Fish\n",
      "Armadillo,Ice_cream\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open(\"animals.csv\", \"r\") as infile:\n",
    "          for line in infile:\n",
    "              print(line.rstrip())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looking good!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 1.\n",
    "1> Open animals.csv by recognigizing the first line as header\n",
    "\n",
    "hint: Read https://thispointer.com/python-read-a-csv-file-line-by-line-with-or-without-header/\n",
    "\n",
    "2> Read animals.csv that you have made, and write the ADDITIONAL header, which is animal and food.\n",
    "\n",
    "hint: Use csv package to do this. Read https://jdhao.github.io/2018/05/13/read-write-csv-file-with-header/\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sloth,Leaves\n",
      "Chicken,Corn\n",
      "Ant_eater,Ants\n",
      "Penguin,Fish\n",
      "Armadillo,Ice_cream\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'Animal,Food\\n'"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#from csv import reader\n",
    "with open(\"animals.csv\", \"r\") as infile:\n",
    "          header = next(infile)\n",
    "          for line in infile:\n",
    "                print(line.rstrip())\n",
    "\n",
    "header"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Sloth', 'Leaves']\n",
      "['Chicken', 'Corn']\n",
      "['Ant_eater', 'Ants']\n",
      "['Penguin', 'Fish']\n",
      "['Armadillo', 'Ice_cream']\n",
      "[]\n"
     ]
    }
   ],
   "source": [
    "with open(\"animals.csv\", \"r\", newline='') as infile:\n",
    "    infile = csv.reader(infile, delimiter=\",\")\n",
    "    header = next(infile)\n",
    "    infile = list(infile)\n",
    "    for line in infile:\n",
    "        print(line)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Animal,Food\\n', 'Sloth,Leaves\\n', 'Chicken,Corn\\n', 'Ant_eater,Ants\\n', 'Penguin,Fish\\n', 'Armadillo,Ice_cream\\n', '\\n']\n"
     ]
    }
   ],
   "source": [
    "infile = open(\"animals.csv\", \"r\")\n",
    "ff = infile.readlines()\n",
    "print(ff)\n",
    "infile.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['Animal', 'Food'], ['Sloth', 'Leaves'], ['Chicken', 'Corn'], ['Ant_eater', 'Ants'], ['Penguin', 'Fish'], ['Armadillo', 'Ice_cream'], []]\n",
      "['Animal', 'Food']\n",
      "['Sloth', 'Leaves']\n",
      "['Chicken', 'Corn']\n",
      "['Ant_eater', 'Ants']\n",
      "['Penguin', 'Fish']\n",
      "['Armadillo', 'Ice_cream']\n",
      "[]\n"
     ]
    }
   ],
   "source": [
    "outfile = open(\"anmals2.csv\", \"w\", newline='')\n",
    "infile = open(\"animals.csv\", \"r\")\n",
    "infile2 =  csv.reader(infile)\n",
    "infile3 = list(infile2)\n",
    "print(infile3)\n",
    "header = ['Animal', 'Food']\n",
    "writer = csv.writer(outfile, delimiter=',')\n",
    "writer.writerow(header)\n",
    "for line in infile3:\n",
    "    print(line)\n",
    "    writer.writerow(line)\n",
    "infile.close()\n",
    "outfile.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## File I/O Pickling: (similar to .RData in R)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Text files are convenient when data needs to be exchanged with other\n",
    "    programs\n",
    "* However, getting the data in/out of text files can be tedious\n",
    "* If we know we only need the data within Python, there is a very easy\n",
    "    way to write arbitrary Python data structures to compact binary files\n",
    "* This is generally referred to as **serialization**, but in\n",
    "    Python-lingo it's called **pickling**\n",
    "* The **pickle** module and it's more efficient **cPickle**\n",
    "    version provide two functions, `dump()` and `load()`, that\n",
    "    allow writing and reading arbitrary Python objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "#from pickle import dump, load\n",
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['a', 'list', 'with', 'stuff', [42, 23, 3.14], True]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "l = [\"a\", \"list\", \"with\", \"stuff\", [42, 23, 3.14], True]\n",
    "print(l)\n",
    "len(l)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"my_list.pkl\", \"wb\") as f: #write wb\n",
    "    pickle.dump(l, f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a', 'list', 'with', 'stuff', [42, 23, 3.14], True]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with open(\"my_list.pkl\", \"rb\") as f: #read rb\n",
    "    l2 = pickle.load(f)\n",
    "l2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O Checking for Existence: Below is free Exercise!!!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Sometimes a program needs to check whether a file exists\n",
    "* The `os.path` module provides the `exists()` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "from os.path import exists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 248,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lockfile exists!\n"
     ]
    }
   ],
   "source": [
    "if exists(\"C:\\\\Users\\\\user\\\\Dropbox\\\\Class\\\\2020_2\\\\programming\\\\week2\\\\animals.csv\"):\n",
    "    print(\"Lockfile exists!\")\n",
    "else:\n",
    "    print(\"No lockfile found!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general, the  `os` and `os.path` modules provide functions for manipulating the file systems. Don't try to reinvent the wheel - most things exist already in the Python standard library!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "C:\\Users\\esthe\\python_env\n"
     ]
    }
   ],
   "source": [
    "#Print the current working directory\n",
    "import os\n",
    "print(os.getcwd())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O: Reading from the Web:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* In Python, there are several other objects that behave just like text\n",
    "    files\n",
    "* One particularly useful one provides file-like access to resources on\n",
    "    the web: the `urlopen()` method in the `urllib2` module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from urllib.request import urlopen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "URL = \"http://www.gutenberg.org/cache/epub/28885/pg28885.txt\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not exists(\"alice.txt\"):\n",
    "    #req=urllib.request\n",
    "    f = urlopen(URL)\n",
    "    with open(\"alice.txt\", \"wb\") as outfile:\n",
    "        outfile.write(f.read())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "middle of one! There ought to be a book written about me, that there\n",
      "ought! And when I grow up, I'll write one--but I'm grown up now,\" she\n",
      "added in a sorrowful tone; \"at least there's no room to grow up any more\n",
      "_here_.\"\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(''.join(open(\"alice.txt\").readlines()[970:975]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hand, and made a snatch in the air. She did not get hold of anything,\n",
      "but she heard a little shriek and a fall, and a crash of broken glass,\n",
      "from which she concluded that it was just possible it had fallen into a\n",
      "cucumber-frame, or something of the sort.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "with open(\"alice.txt\", \"r\") as infile: # rb\n",
    "    book = infile.readlines()\n",
    "    print(''.join(book[1000:1005]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O Multiple Files:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `glob` module provides an easy way to find all files with certain names (e.g. all files with names that end in `.txt`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_files = glob.glob(\"*.txt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "alice.txt\n",
      "animals.txt\n",
      "library_data.txt\n",
      "my_first_file.txt\n"
     ]
    }
   ],
   "source": [
    "for t in text_files:\n",
    "    print(t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## File I/O Terminal streams:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The terminal input/output streams can also be accessed like filesusing the `stdin` and `stdout` objects from the `sys` module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Another way to print!\n"
     ]
    }
   ],
   "source": [
    "sys.stdout.write(\"Another way to print!\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "aaa = \"a aa aa b b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['a', 'aa', 'aa', 'b', 'b']"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aaa.split()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}