DA0101EN-Review-Model-Development.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "    <a href=\"http://cocl.us/DA0101EN_NotbookLink_Top\">\n",
    "         <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/Images/TopAd.png\" width=\"750\" align=\"center\">\n",
    "    </a>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://www.bigdatauniversity.com\"><img src = \"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/Images/CCLog.png\" width = 300, align = \"center\"></a>\n",
    "\n",
    "<h1 align=center><font size=5>Data Analysis with Python</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Module 4: Model Development</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>In this section, we will develop several models that will predict the price of the car using the variables or features. This is just an estimate but should give us an objective idea of how much the car should cost.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some questions we want to ask in this module\n",
    "<ul>\n",
    "    <li>do I know if the dealer is offering fair value for my trade-in?</li>\n",
    "    <li>do I know if I put a fair value on my car?</li>\n",
    "</ul>\n",
    "<p>Data Analytics, we often use <b>Model Development</b> to help us predict future observations from the data we have.</p>\n",
    "\n",
    "<p>A Model will help us understand the exact relationship between different variables and how these variables are used to predict the result.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Setup</h4>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "load data and store in dataframe df:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dataset was hosted on IBM Cloud object click <a href=\"https://cocl.us/cognitive_class_DA0101EN_objectstorage\">HERE</a> for free storage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# path of data \n",
    "path = 'https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/automobileEDA.csv'\n",
    "df = pd.read_csv(path)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>1. Linear Regression and Multiple Linear Regression</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Linear Regression</h4>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<p>One example of a Data  Model that we will be using is</p>\n",
    "<b>Simple Linear Regression</b>.\n",
    "\n",
    "<br>\n",
    "<p>Simple Linear Regression is a method to help us understand the relationship between two variables:</p>\n",
    "<ul>\n",
    "    <li>The predictor/independent variable (X)</li>\n",
    "    <li>The response/dependent variable (that we want to predict)(Y)</li>\n",
    "</ul>\n",
    "\n",
    "<p>The result of Linear Regression is a <b>linear function</b> that predicts the response (dependent) variable as a function of the predictor (independent) variable.</p>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    " Y: Response \\ Variable\\\\\n",
    " X: Predictor \\ Variables\n",
    "$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " <b>Linear function:</b>\n",
    "$$\n",
    "Yhat = a + b  X\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<ul>\n",
    "    <li>a refers to the <b>intercept</b> of the regression line0, in other words: the value of Y when X is 0</li>\n",
    "    <li>b refers to the <b>slope</b> of the regression line, in other words: the value with which Y changes when X increases by 1 unit</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Lets load the modules for linear regression</h4>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Create the linear regression object</h4>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lm = LinearRegression()\n",
    "lm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>How could Highway-mpg help us predict car price?</h4>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this example, we want to look at how highway-mpg can help us predict car price.\n",
    "Using simple linear regression, we will create a linear function with \"highway-mpg\" as the predictor variable and the \"price\" as the response variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X = df[['highway-mpg']]\n",
    "Y = df['price']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fit the linear model using highway-mpg."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lm.fit(X,Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " We can output a prediction "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Yhat=lm.predict(X)\n",
    "Yhat[0:5]   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>What is the value of the intercept (a)?</h4>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lm.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>What is the value of the Slope (b)?</h4>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "lm.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>What is the final estimated linear model we get?</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we saw above, we should get a final linear model with the structure:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "Yhat = a + b  X\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plugging in the actual values we get:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>price</b> = 38423.31 - 821.73 x  <b>highway-mpg</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1>Question #1 a): </h1>\n",
    "\n",
    "<b>Create a linear regression object?</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "lm1 = LinearRegression()\n",
    "lm1 \n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1> Question #1 b): </h1>\n",
    "\n",
    "<b>Train the model using 'engine-size' as the independent variable and 'price' as the dependent variable?</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "lm1.fit(df[['highway-mpg']], df[['price']])\n",
    "lm1\n",
    "\n",
    "-->\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1>Question #1 c):</h1>\n",
    "\n",
    "<b>Find the slope and intercept of the model?</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Slope</h4>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Intercept</h4>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "# Slope \n",
    "lm1.coef_\n",
    "# Intercept\n",
    "lm1.intercept_\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1>Question #1 d): </h1>\n",
    "\n",
    "<b>What is the equation of the predicted line. You can use x and yhat or 'engine-size' or 'price'?</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# You can type you answer here\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "# using X and Y  \n",
    "Yhat=38423.31-821.733*X\n",
    "\n",
    "Price=38423.31-821.733*engine-size\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h4>Multiple Linear Regression</h4>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>What if we want to predict car price using more than one variable?</p>\n",
    "\n",
    "<p>If we want to use more variables in our model to predict car price, we can use <b>Multiple Linear Regression</b>.\n",
    "Multiple Linear Regression is very similar to Simple Linear Regression, but this method is used to explain the relationship between one continuous response (dependent) variable and <b>two or more</b> predictor (independent) variables.\n",
    "Most of the real-world regression models involve multiple predictors. We will illustrate the structure by using four predictor variables, but these results can generalize to any integer:</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "Y: Response \\ Variable\\\\\n",
    "X_1 :Predictor\\ Variable \\ 1\\\\\n",
    "X_2: Predictor\\ Variable \\ 2\\\\\n",
    "X_3: Predictor\\ Variable \\ 3\\\\\n",
    "X_4: Predictor\\ Variable \\ 4\\\\\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "a: intercept\\\\\n",
    "b_1 :coefficients \\ of\\ Variable \\ 1\\\\\n",
    "b_2: coefficients \\ of\\ Variable \\ 2\\\\\n",
    "b_3: coefficients \\ of\\ Variable \\ 3\\\\\n",
    "b_4: coefficients \\ of\\ Variable \\ 4\\\\\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The equation is given by"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "Yhat = a + b_1 X_1 + b_2 X_2 + b_3 X_3 + b_4 X_4\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>From the previous section  we know that other good predictors of price could be:</p>\n",
    "<ul>\n",
    "    <li>Horsepower</li>\n",
    "    <li>Curb-weight</li>\n",
    "    <li>Engine-size</li>\n",
    "    <li>Highway-mpg</li>\n",
    "</ul>\n",
    "Let's develop a model using these variables as the predictor variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Z = df[['horsepower', 'curb-weight', 'engine-size', 'highway-mpg']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fit the linear model using the four above-mentioned variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lm.fit(Z, df['price'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is the value of the intercept(a)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lm.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What are the values of the coefficients (b1, b2, b3, b4)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lm.coef_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " What is the final estimated linear model that we get?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we saw above, we should get a final linear function with the structure:\n",
    "\n",
    "$$\n",
    "Yhat = a + b_1 X_1 + b_2 X_2 + b_3 X_3 + b_4 X_4\n",
    "$$\n",
    "\n",
    "What is the linear function we get in this example?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Price</b> = -15678.742628061467 + 52.65851272 x <b>horsepower</b> + 4.69878948 x <b>curb-weight</b> + 81.95906216 x <b>engine-size</b> + 33.58258185 x <b>highway-mpg</b>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1> Question  #2 a): </h1>\n",
    "Create and train a Multiple Linear Regression model \"lm2\" where the response variable is price, and the predictor variable is 'normalized-losses' and  'highway-mpg'.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "lm2 = LinearRegression()\n",
    "lm2.fit(df[['normalized-losses' , 'highway-mpg']],df['price'])\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1>Question  #2 b): </h1>\n",
    "<b>Find the coefficient of the model?</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "lm2.coef_\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>2)  Model Evaluation using Visualization</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we've developed some models, how do we evaluate our models and how do we choose the best one? One way to do this is by using visualization."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "import the visualization package: seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# import the visualization package: seaborn\n",
    "import seaborn as sns\n",
    "%matplotlib inline "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Regression Plot</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>When it comes to simple linear regression, an excellent way to visualize the fit of our model is by using <b>regression plots</b>.</p>\n",
    "\n",
    "<p>This plot will show a combination of a scattered data points (a <b>scatter plot</b>), as well as the fitted <b>linear regression</b> line going through the data. This will give us a reasonable estimate of the relationship between the two variables, the strength of the correlation, as well as the direction (positive or negative correlation).</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Let's visualize Horsepower as potential predictor variable of price:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "width = 12\n",
    "height = 10\n",
    "plt.figure(figsize=(width, height))\n",
    "sns.regplot(x=\"highway-mpg\", y=\"price\", data=df)\n",
    "plt.ylim(0,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can see from this plot that price is negatively correlated to highway-mpg, since the regression slope is negative.\n",
    "One thing to keep in mind when looking at a regression plot is to pay attention to how scattered the data points are around the regression line. This will give you a good indication of the variance of the data, and whether a linear model would be the best fit or not. If the data is too far off from the line, this linear model might not be the best model for this data. Let's compare this plot to the regression plot of \"peak-rpm\".</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(width, height))\n",
    "sns.regplot(x=\"peak-rpm\", y=\"price\", data=df)\n",
    "plt.ylim(0,)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Comparing the regression plot of \"peak-rpm\" and \"highway-mpg\" we see that the points for \"highway-mpg\" are much closer to the generated line and on the average decrease. The points for \"peak-rpm\" have more spread around the predicted line, and it is much harder to determine if the points are decreasing or increasing as the \"highway-mpg\" increases.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1>Question #3:</h1>\n",
    "<b>Given the regression plots above is \"peak-rpm\" or \"highway-mpg\" more strongly correlated with \"price\". Use the method  \".corr()\" to verify your answer.</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "The variable \"peak-rpm\" has a stronger correlation with \"price\", it is approximate -0.704692  compared to   \"highway-mpg\" which is approximate     -0.101616. You can verify it using the following command:\n",
    "df[[\"peak-rpm\",\"highway-mpg\",\"price\"]].corr()\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Residual Plot</h3>\n",
    "\n",
    "<p>A good way to visualize the variance of the data is to use a residual plot.</p>\n",
    "\n",
    "<p>What is a <b>residual</b>?</p>\n",
    "\n",
    "<p>The difference between the observed value (y) and the predicted value (Yhat) is called the residual (e). When we look at a regression plot, the residual is the distance from the data point to the fitted regression line.</p>\n",
    "\n",
    "<p>So what is a <b>residual plot</b>?</p>\n",
    "\n",
    "<p>A residual plot is a graph that shows the residuals on the vertical y-axis and the independent variable on the horizontal x-axis.</p>\n",
    "\n",
    "<p>What do we pay attention to when looking at a residual plot?</p>\n",
    "\n",
    "<p>We look at the spread of the residuals:</p>\n",
    "\n",
    "<p>- If the points in a residual plot are <b>randomly spread out around the x-axis</b>, then a <b>linear model is appropriate</b> for the data. Why is that? Randomly spread out residuals means that the variance is constant, and thus the linear model is a good fit for this data.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "width = 12\n",
    "height = 10\n",
    "plt.figure(figsize=(width, height))\n",
    "sns.residplot(df['highway-mpg'], df['price'])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<i>What is this plot telling us?</i>\n",
    "\n",
    "<p>We can see from this residual plot that the residuals are not randomly spread around the x-axis, which leads us to believe that maybe a non-linear model is more appropriate for this data.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Multiple Linear Regression</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>How do we visualize a model for Multiple Linear Regression? This gets a bit more complicated because you can't visualize it with regression or residual plot.</p>\n",
    "\n",
    "<p>One way to look at the fit of the model is by looking at the <b>distribution plot</b>: We can look at the distribution of the fitted values that result from the model and compare it to the distribution of the actual values.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First lets make a prediction "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Y_hat = lm.predict(Z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "plt.figure(figsize=(width, height))\n",
    "\n",
    "\n",
    "ax1 = sns.distplot(df['price'], hist=False, color=\"r\", label=\"Actual Value\")\n",
    "sns.distplot(Yhat, hist=False, color=\"b\", label=\"Fitted Values\" , ax=ax1)\n",
    "\n",
    "\n",
    "plt.title('Actual vs Fitted Values for Price')\n",
    "plt.xlabel('Price (in dollars)')\n",
    "plt.ylabel('Proportion of Cars')\n",
    "\n",
    "plt.show()\n",
    "plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can see that the fitted values are reasonably close to the actual values, since the two distributions overlap a bit. However, there is definitely some room for improvement.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Part 3: Polynomial Regression and Pipelines</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p><b>Polynomial regression</b> is a particular case of the general linear regression model or multiple linear regression models.</p> \n",
    "<p>We get non-linear relationships by squaring or setting higher-order terms of the predictor variables.</p>\n",
    "\n",
    "<p>There are different orders of polynomial regression:</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><b>Quadratic - 2nd order</b></center>\n",
    "$$\n",
    "Yhat = a + b_1 X^2 +b_2 X^2 \n",
    "$$\n",
    "\n",
    "\n",
    "<center><b>Cubic - 3rd order</b></center>\n",
    "$$\n",
    "Yhat = a + b_1 X^2 +b_2 X^2 +b_3 X^3\\\\\n",
    "$$\n",
    "\n",
    "\n",
    "<center><b>Higher order</b>:</center>\n",
    "$$\n",
    "Y = a + b_1 X^2 +b_2 X^2 +b_3 X^3 ....\\\\\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We saw earlier that a linear model did not provide the best fit while using highway-mpg as the predictor variable. Let's see if we can try fitting a polynomial model to the data instead.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We will use the following function to plot the data:</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def PlotPolly(model, independent_variable, dependent_variabble, Name):\n",
    "    x_new = np.linspace(15, 55, 100)\n",
    "    y_new = model(x_new)\n",
    "\n",
    "    plt.plot(independent_variable, dependent_variabble, '.', x_new, y_new, '-')\n",
    "    plt.title('Polynomial Fit with Matplotlib for Price ~ Length')\n",
    "    ax = plt.gca()\n",
    "    ax.set_facecolor((0.898, 0.898, 0.898))\n",
    "    fig = plt.gcf()\n",
    "    plt.xlabel(Name)\n",
    "    plt.ylabel('Price of Cars')\n",
    "\n",
    "    plt.show()\n",
    "    plt.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lets get the variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "x = df['highway-mpg']\n",
    "y = df['price']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fit the polynomial using the function <b>polyfit</b>, then use the function <b>poly1d</b> to display the polynomial function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Here we use a polynomial of the 3rd order (cubic) \n",
    "f = np.polyfit(x, y, 3)\n",
    "p = np.poly1d(f)\n",
    "print(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Let's plot the function "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "PlotPolly(p, x, y, 'highway-mpg')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "np.polyfit(x, y, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>We can already see from plotting that this polynomial model performs better than the linear model. This is because the generated polynomial function  \"hits\" more of the data points.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1>Question  #4:</h1>\n",
    "<b>Create 11 order polynomial model with the variables x and y from above?</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "# calculate polynomial\n",
    "# Here we use a polynomial of the 3rd order (cubic) \n",
    "f1 = np.polyfit(x, y, 11)\n",
    "p1 = np.poly1d(f1)\n",
    "print(p)\n",
    "PlotPolly(p1,x,y, 'Length')\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>The analytical expression for Multivariate Polynomial function gets complicated. For example, the expression for a second-order (degree=2)polynomial with two variables is given by:</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "Yhat = a + b_1 X_1 +b_2 X_2 +b_3 X_1 X_2+b_4 X_1^2+b_5 X_2^2\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can perform a polynomial transform on multiple features. First, we import the module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create a <b>PolynomialFeatures</b> object of degree 2: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pr=PolynomialFeatures(degree=2)\n",
    "pr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Z_pr=pr.fit_transform(Z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The original data is of 201 samples and 4 features "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Z.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "after the transformation, there 201 samples and 15 features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Z_pr.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Pipeline</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Data Pipelines simplify the steps of processing the data. We use the module <b>Pipeline</b> to create a pipeline. We also use <b>StandardScaler</b> as a step in our pipeline.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import StandardScaler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We create the pipeline, by creating a list of tuples including the name of the model or estimator and its corresponding constructor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Input=[('scale',StandardScaler()), ('polynomial', PolynomialFeatures(include_bias=False)), ('model',LinearRegression())]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we input the list as an argument to the pipeline constructor "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pipe=Pipeline(Input)\n",
    "pipe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can normalize the data,  perform a transform and fit the model simultaneously. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "pipe.fit(Z,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Similarly,  we can normalize the data, perform a transform and produce a prediction  simultaneously"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ypipe=pipe.predict(Z)\n",
    "ypipe[0:4]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-danger alertdanger\" style=\"margin-top: 20px\">\n",
    "<h1>Question #5:</h1>\n",
    "<b>Create a pipeline that Standardizes the data, then perform prediction using a linear regression model using the features Z and targets y</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write your code below and press Shift+Enter to execute \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "</div>\n",
    "Double-click <b>here</b> for the solution.\n",
    "\n",
    "<!-- The answer is below:\n",
    "\n",
    "Input=[('scale',StandardScaler()),('model',LinearRegression())]\n",
    "\n",
    "pipe=Pipeline(Input)\n",
    "\n",
    "pipe.fit(Z,y)\n",
    "\n",
    "ypipe=pipe.predict(Z)\n",
    "ypipe[0:10]\n",
    "\n",
    "-->"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Part 4: Measures for In-Sample Evaluation</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>When evaluating our models, not only do we want to visualize the results, but we also want a quantitative measure to determine how accurate the model is.</p>\n",
    "\n",
    "<p>Two very important measures that are often used in Statistics to determine the accuracy of a model are:</p>\n",
    "<ul>\n",
    "    <li><b>R^2 / R-squared</b></li>\n",
    "    <li><b>Mean Squared Error (MSE)</b></li>\n",
    "</ul>\n",
    "    \n",
    "<b>R-squared</b>\n",
    "\n",
    "<p>R squared, also known as the coefficient of determination, is a measure to indicate how close the data is to the fitted regression line.</p>\n",
    "    \n",
    "<p>The value of the R-squared is the percentage of variation of the response variable (y) that is explained by a linear model.</p>\n",
    "\n",
    "\n",
    "\n",
    "<b>Mean Squared Error (MSE)</b>\n",
    "\n",
    "<p>The Mean Squared Error measures the average of the squares of errors, that is, the difference between actual value (y) and the estimated value (ŷ).</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Model 1: Simple Linear Regression</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's calculate the R^2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#highway_mpg_fit\n",
    "lm.fit(X, Y)\n",
    "# Find the R^2\n",
    "print('The R-square is: ', lm.score(X, Y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can say that ~ 49.659% of the variation of the price is explained by this simple linear model \"horsepower_fit\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's calculate the MSE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can predict the output i.e., \"yhat\" using the predict method, where X is the input variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Yhat=lm.predict(X)\n",
    "print('The output of the first four predicted value is: ', Yhat[0:4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "lets import the function <b>mean_squared_error</b> from the module <b>metrics</b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_squared_error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we compare the predicted results with the actual results "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "mse = mean_squared_error(df['price'], Yhat)\n",
    "print('The mean square error of price and predicted value is: ', mse)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Model 2: Multiple Linear Regression</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's calculate the R^2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# fit the model \n",
    "lm.fit(Z, df['price'])\n",
    "# Find the R^2\n",
    "print('The R-square is: ', lm.score(Z, df['price']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can say that ~ 80.896 % of the variation of price is explained by this multiple linear regression \"multi_fit\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's calculate the MSE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " we produce a prediction "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Y_predict_multifit = lm.predict(Z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " we compare the predicted results with the actual results "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print('The mean square error of price and predicted value using multifit is: ', \\\n",
    "      mean_squared_error(df['price'], Y_predict_multifit))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Model 3: Polynomial Fit</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's calculate the R^2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "let’s import the function <b>r2_score</b> from the module <b>metrics</b> as we are using a different function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import r2_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We apply the function to get the value of r^2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "r_squared = r2_score(y, p(x))\n",
    "print('The R-square value is: ', r_squared)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can say that ~ 67.419 % of the variation of price is explained by this polynomial fit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>MSE</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also calculate the MSE:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "mean_squared_error(df['price'], p(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Part 5: Prediction and Decision Making</h2>\n",
    "<h3>Prediction</h3>\n",
    "\n",
    "<p>In the previous section, we trained the model using the method <b>fit</b>. Now we will use the method <b>predict</b> to produce a prediction. Lets import <b>pyplot</b> for plotting; we will also be using some functions from numpy.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a new input "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "new_input=np.arange(1, 100, 1).reshape(-1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Fit the model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lm.fit(X, Y)\n",
    "lm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Produce a prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "yhat=lm.predict(new_input)\n",
    "yhat[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we can plot the data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "plt.plot(new_input, yhat)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Decision Making: Determining a Good Model Fit</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Now that we have visualized the different models, and generated the R-squared and MSE values for the fits, how do we determine a good model fit?\n",
    "<ul>\n",
    "    <li><i>What is a good R-squared value?</i></li>\n",
    "</ul>\n",
    "</p>\n",
    "\n",
    "<p>When comparing models, <b>the model with the higher R-squared value is a better fit</b> for the data.\n",
    "<ul>\n",
    "    <li><i>What is a good MSE?</i></li>\n",
    "</ul>\n",
    "</p>\n",
    "\n",
    "<p>When comparing models, <b>the model with the smallest MSE value is a better fit</b> for the data.</p>\n",
    "\n",
    "\n",
    "<h4>Let's take a look at the values for the different models.</h4>\n",
    "<p>Simple Linear Regression: Using Highway-mpg as a Predictor Variable of Price.\n",
    "<ul>\n",
    "    <li>R-squared: 0.49659118843391759</li>\n",
    "    <li>MSE: 3.16 x10^7</li>\n",
    "</ul>\n",
    "</p>\n",
    "    \n",
    "<p>Multiple Linear Regression: Using Horsepower, Curb-weight, Engine-size, and Highway-mpg as Predictor Variables of Price.\n",
    "<ul>\n",
    "    <li>R-squared: 0.80896354913783497</li>\n",
    "    <li>MSE: 1.2 x10^7</li>\n",
    "</ul>\n",
    "</p>\n",
    "    \n",
    "<p>Polynomial Fit: Using Highway-mpg as a Predictor Variable of Price.\n",
    "<ul>\n",
    "    <li>R-squared: 0.6741946663906514</li>\n",
    "    <li>MSE: 2.05 x 10^7</li>\n",
    "</ul>\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Simple Linear Regression model (SLR) vs Multiple Linear Regression model (MLR)</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Usually, the more variables you have, the better your model is at predicting, but this is not always true. Sometimes you may not have enough data, you may run into numerical problems, or many of the variables may not be useful and or even act as noise. As a result, you should always check the MSE and R^2.</p>\n",
    "\n",
    "<p>So to be able to compare the results of the MLR vs SLR models, we look at a combination of both the R-squared and MSE to make the best conclusion about the fit of the model.\n",
    "<ul>\n",
    "    <li><b>MSE</b>The MSE of SLR is  3.16x10^7  while MLR has an MSE of 1.2 x10^7.  The MSE of MLR is much smaller.</li>\n",
    "    <li><b>R-squared</b>: In this case, we can also see that there is a big difference between the R-squared of the SLR and the R-squared of the MLR. The R-squared for the SLR (~0.497) is very small compared to the R-squared for the MLR (~0.809).</li>\n",
    "</ul>\n",
    "</p>\n",
    "\n",
    "This R-squared in combination with the MSE show that MLR seems like the better model fit in this case, compared to SLR."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Simple Linear Model (SLR) vs Polynomial Fit</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<ul>\n",
    "    <li><b>MSE</b>: We can see that Polynomial Fit brought down the MSE, since this MSE is smaller than the one from the SLR.</li> \n",
    "    <li><b>R-squared</b>: The R-squared for the Polyfit is larger than the R-squared for the SLR, so the Polynomial Fit also brought up the R-squared quite a bit.</li>\n",
    "</ul>\n",
    "<p>Since the Polynomial Fit resulted in a lower MSE and a higher R-squared, we can conclude that this was a better fit model than the simple linear regression for predicting Price with Highway-mpg as a predictor variable.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>Multiple Linear Regression (MLR) vs Polynomial Fit</h3>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<ul>\n",
    "    <li><b>MSE</b>: The MSE for the MLR is smaller than the MSE for the Polynomial Fit.</li>\n",
    "    <li><b>R-squared</b>: The R-squared for the MLR is also much larger than for the Polynomial Fit.</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>Conclusion:</h2>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>Comparing these three models, we conclude that <b>the MLR model is the best model</b> to be able to predict price from our dataset. This result makes sense, since we have 27 variables in total, and we know that more than one of those variables are potential predictors of the final car price.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Thank you for completing this notebook</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\" style=\"margin-top: 20px\">\n",
    "\n",
    "    <p><a href=\"https://cocl.us/DA0101EN_NotbookLink_Top_bottom\"><img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DA0101EN/Images/BottomAd.png\" width=\"750\" align=\"center\"></a></p>\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>About the Authors:</h3>\n",
    "\n",
    "This notebook was written by <a href=\"https://www.linkedin.com/in/mahdi-noorian-58219234/\" target=\"_blank\">Mahdi Noorian PhD</a>, <a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\" target=\"_blank\">Joseph Santarcangelo</a>, Bahare Talayian, Eric Xiao, Steven Dong, Parizad, Hima Vsudevan and <a href=\"https://www.linkedin.com/in/fiorellawever/\" target=\"_blank\">Fiorella Wenver</a> and <a href=\" https://www.linkedin.com/in/yi-leng-yao-84451275/ \" target=\"_blank\" >Yi Yao</a>.\n",
    "\n",
    "<p><a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\" target=\"_blank\">Joseph Santarcangelo</a> is a Data Scientist at IBM, and holds a PhD in Electrical Engineering. His research focused on using Machine Learning, Signal Processing, and Computer Vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr>\n",
    "<p>Copyright &copy; 2018 IBM Developer Skills Network. This notebook and its source code are released under the terms of the <a href=\"https://cognitiveclass.ai/mit-license/\">MIT License</a>.</p>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}