Neural Networks - Intro, Tensorflow Basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural Networks - Reinforcement Learning\n",
    "\n",
    "Notes from https://www.youtube.com/watch?v=MotG3XI2qSs\n",
    "\n",
    "\n",
    "Uses a *Dataflow Graph*\n",
    "\n",
    "has both a Python and C++ API, but Python is more complete\n",
    "Faster compile times\n",
    "Supports CPUs, GPUs and Distributed Processing\n",
    "\n",
    "**DataFlow Graph**:  \n",
    "**Node** - represents a mathematical operation  \n",
    "**Edge** - multidimensional array (Tensor)\n",
    "\n",
    "Standard usage - build a graph and execute after the session is created by using the *run* and *eval* operations.\n",
    "\n",
    "There's an option for interactive sessions that run on demand. \n",
    "\n",
    "DSWB - Data Scientist's WorkBench\n",
    "\n",
    "\n",
    "DQN (Deep Q Network) patented by google\n",
    "- Experience Replay\n",
    "- Dueling Network Architecture\n",
    "\n",
    "----\n",
    "\n",
    "Reinforcement learning isn't just another way to say \"supervised learning\". *Supervised Learning* is all about making sense of the environment based on historical samples. \n",
    "\n",
    "Not always the best way to do things. \"Driving only looking at the rearview mirror\". \n",
    "\n",
    "*Reinforcement Learning* is all about *Reward*. You get points for your actions and can lose points through unhelpful behavior. The goal is to get the maximum number of points by accomplishing your goal based on the state of traffic around you.\n",
    "\n",
    "*Reinforcement Learning* recognizes that an *action* results in a change of the *state*. \n",
    "\n",
    "Exploration vs Exploitation:\n",
    "\n",
    "Most organizations operate in the realm of conventional wisdom. Which is about exploiting what is known to achieve finite rewards with known odds. Some groups venture into the unknown and explore new territory with the prospect of outsized rewards at long odds. And many of these organizations do fail. But some of them succeed and end up changing the world. \n",
    "\n",
    "With *Reinforcement Learning* a group can explore the trade-off between *Exploration and Exploitation* and choose the path to the maximum expected reward. \n",
    "\n",
    "AI encompasses Deep Learning and Reinforcement Learning (imagine a venn diagram with overlap). \n",
    "\n",
    "Goal setting, Planning and Perception. It can form a bridge between AI and the engineering disciplines. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Neural Networks Theory\n",
    "\n",
    "Stepping stone to approaching concepts like Deep Learning.\n",
    "\n",
    "Neural networks are modeled after *Biological Neural Networks* and attempt to allow computers to learn in a similar manner to humans - *Reinforcement Learning*. \n",
    "\n",
    "Use cases:\n",
    "- Pattern Recognition\n",
    "- Time Series Predictions\n",
    "- Signal Processing\n",
    "- Anomaly Detection\n",
    "- Control\n",
    "\n",
    "The human brain has interconnected *neurons* with *dendrites* that receive inputs and then, based on those inputs, produce and electrical signal output through the *axon*. \n",
    "\n",
    "**Artificial Neural Networks (ANN)**\n",
    "\n",
    "Why bother?  \n",
    "There are problems that are easy for computers but hard for humans (calculating large arithmetic problems)  \n",
    "There are problems that are hard for computers but easy for humans (recognizing a picture of a person from the side)\n",
    "\n",
    "*Neural Networks* attempt to solve problems that would be easy for humans but hard for computers. \n",
    "\n",
    "**The Perceptron** \n",
    "- simplest Neural Network\n",
    "\n",
    "Consists of:\n",
    "- one or more inputs\n",
    "- a processor\n",
    "- a single output\n",
    "\n",
    "A *perceptron* follows the *Feed Forward* model, meaning inputs are \n",
    "1. sent to the neuron\n",
    "2. processed\n",
    "3. result in an output\n",
    "\n",
    "The process follows **5 main steps**\n",
    "1. **Receive** inputs\n",
    "2. **Weight** inputs (add some sort of weighting - multiplying by some value (often a number between -1 and 1)\n",
    "3. **Sum** inputs\n",
    "4. **Activation Function** this tells the function whether to 'fire' or not\n",
    "5. **Generate output**\n",
    "\n",
    "*Activation Functions* (Logistic, Trigonometric, Sinusoidal, Step, ..., etc)\n",
    "\n",
    "Also consider **Bias** - in case the earlier functions would output *Nulls or Zeroes* in a non-helpful way. This overcomes the *Zero Issue*. \n",
    "\n",
    "*Training the Perceptron*\n",
    "1. Provide the perceptron inputs for which there is a *known answer*\n",
    "2. Ask the perceptron to guess the answer\n",
    "3. Compute the error (how far off from the correct answer?)\n",
    "4. Adjust all the weights according to the error \n",
    "5. Return to step 1 and repeat.\n",
    "\n",
    "We repeat the process until we reach an error that we're satisfied with. \n",
    "\n",
    "Eventually *Layer* a bunch of perceptrons together - this will create a Neural Net.\n",
    "\n",
    "Link a bunch of Perceptrons together in Layers.\n",
    "\n",
    "There will be an *Input Layer*, some number of layers and an *Output Layer*. \n",
    "The layers in-between are called *Hidden Layers* because you don't directly see the Input or Output.\n",
    "\n",
    "**Deep Learning** just means *A Neural Network with many Input and Output Layers*. Microsoft's Vision Recognition (circa 2016) is 152 layers. \n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TensorFlow\n",
    "\n",
    "Brief background on Tensorflow\n",
    "\n",
    "An open source library developed by Google.  \n",
    "Runs on either CPU or GPU.  \n",
    "Most popular library in the field  \n",
    "\n",
    "Typically Deep learning Networks run much faster on GPU\n",
    "\n",
    "The basic idea is the be able to create a *Data Flow Graph*  \n",
    "- the graphs have *Nodes* and *Edges* \n",
    "- arrays (data) passed along *From* a layer of nodes *To* a layer of nodes is called a **Tensor**\n",
    "\n",
    "Two ways to use TensorFlow:\n",
    "- Customizable Graph Session (typical)\n",
    "- SciKit-Learn type interface with *Contrib.Learn* (not as customizable, but easier to use)\n",
    "\n",
    "the *Session* mode can be overwhelming if you don't have a background in math\n",
    "\n",
    "\n",
    "----\n",
    "\n",
    "The MNIST database (Modified National Institute of Standards and Technology database) is a large database of handwritten digits that is commonly used for training various image processing systems.\n",
    "\n",
    "----\n",
    "\n",
    "### TensorFlow Basics\n",
    "\n",
    "Create a Simple Constant using TensorFlow\n",
    "\n",
    "**Tensor Objects**\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TensorFlow Objects**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'Const:0' shape=() dtype=string>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Constants will be stored as a Tensor Object\n",
    "tf.constant('Hello World')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "hello = tf.constant('Hello World')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensorflow.python.framework.ops.Tensor"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(hello)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = tf.constant(100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'Const_2:0' shape=() dtype=int32>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensorflow.python.framework.ops.Tensor"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Creating a *TensorFlow Session***\n",
    "\n",
    "The *Session Object* encapsulates the *Environment* where *Operation Objects* are executed.\n",
    "\n",
    "*Tensor Objects* are *Evaluated* in those Operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TensorFlow Sessions**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiating the Session Object\n",
    "sess = tf.Session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "b'Hello World'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# \"b\" indicates \"unicode\"\n",
    "sess.run(hello)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "100"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sess.run(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "numpy.int32"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(sess.run(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "bytes"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(sess.run(hello))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TensorFlow Operations**\n",
    "\n",
    "You can line up multiple Tensorflow Operations to be run during a session. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# e.g.\n",
    "x = tf.constant(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = tf.constant(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Operations With Constants\n",
      "Addition : 5\n",
      "Subtraction : -1\n",
      "Multiplication : 6\n",
      "Division : 0.6666666666666666\n"
     ]
    }
   ],
   "source": [
    "# Similar to opening a file\n",
    "with tf.Session() as sess:\n",
    "    print('Operations With Constants')\n",
    "    print('Addition :', sess.run(x + y))\n",
    "    print('Subtraction :',sess.run(x - y))\n",
    "    print('Multiplication :',sess.run(x * y))\n",
    "    print('Division :',sess.run(x / y))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using Placeholders**\n",
    "\n",
    "Used when you don't have the constants"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Must pass in object type. Tensorflow has a number of default types available\n",
    "# see also float16, float64, etc\n",
    "x = tf.placeholder(tf.int32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = tf.placeholder(tf.int32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor 'Placeholder:0' shape=<unknown> dtype=int32>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Notice the \"shape=<unknown>\"\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on machine type (64 or 32bit CPU or GPU, there may be errors with types)\n",
    "\n",
    "Now that we have Placeholders we can *Operations* that have *Variable Input*, compared to the earlier examples where we had vars with *Constant Inputs*\n",
    "\n",
    "These are somewhat analogous to python functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TensorFlow already has a number of built in operations for these, this is just to show what's possible\n",
    "add = tf.add(x,y)\n",
    "sub = tf.subtract(x,y)\n",
    "mul = tf.multiply(x,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adding a dictionary object\n",
    "d = {x:20,y:30}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Operations with placeholders\n",
      "addition:  50\n",
      "subtraction:  -10\n",
      "multiplication:  600\n"
     ]
    }
   ],
   "source": [
    "# Passing in 'Feed Dictionary' feed_dict. A dictionary of Values that the Operation will expect\n",
    "# note that the feed_dict is passing in x and y as raw var keys, not strings\n",
    "with tf.Session() as sess:\n",
    "    print('Operations with placeholders')\n",
    "    print('addition: ', sess.run(add,feed_dict={x:20,y:30}))\n",
    "    print('subtraction: ', sess.run(sub,feed_dict=d))\n",
    "    print('multiplication: ', sess.run(mul,feed_dict=d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**A more complex example** using *Matrix Multiplication*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Notice that A is a 1x2 shape and b is a 2x1 shape\n",
    "a = np.array([[5.0,5.0]])\n",
    "b = np.array([[2.0],[2.0]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1, 2)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 1)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now convert the matrices into tensor objects**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "mat1 = tf.constant(a)\n",
    "mat2 = tf.constant(b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Matrix Multiplication Operation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "matrix_multi = tf.matmul(mat1,mat2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now run the session to perform the operation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[20.]]\n"
     ]
    }
   ],
   "source": [
    "# Here we're not using a feed dictionary, since the inputs are already constants\n",
    "with tf.Session() as sess:\n",
    "    result = sess.run(matrix_multi)\n",
    "    print(result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}