{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [],
   "source": [
    "path_data = '../../data/'\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from scipy import stats\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Regression Line\n",
    "The correlation coefficient $r$ doesn't just measure how clustered the points in a scatter plot are about a straight line. It also helps identify the straight line about which the points are clustered. In this section we will retrace the path that Galton and Pearson took to discover that line.\n",
    "\n",
    "Galton's data on the heights of parents and their adult children showed a linear association. The linearity was confirmed when our predictions of the children's heights based on the midparent heights roughly followed a straight line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "galton = pd.read_csv(path_data + 'galton.csv')\n",
    "\n",
    "heights = pd.DataFrame(\n",
    "    {'MidParent':galton['midparentHeight'],\n",
    "    'Child':galton['childHeight']}\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict_child(mpht):\n",
    "    \"\"\"Return a prediction of the height of a child \n",
    "    whose parents have a midparent height of mpht.\n",
    "    \n",
    "    The prediction is the average height of the children \n",
    "    whose midparent height is in the range mpht plus or minus 0.5 inches.\n",
    "    \"\"\"\n",
    "    \n",
    "    close_points = heights[(heights['MidParent'] >= mpht-0.5) & (heights['MidParent'] < mpht+0.5)]\n",
    "    return close_points['Child'].mean()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MidParent</th>\n",
       "      <th>Child</th>\n",
       "      <th>Prediction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75.43</td>\n",
       "      <td>73.2</td>\n",
       "      <td>70.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.2</td>\n",
       "      <td>70.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.0</td>\n",
       "      <td>70.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.0</td>\n",
       "      <td>70.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>73.66</td>\n",
       "      <td>73.5</td>\n",
       "      <td>70.415789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>929</th>\n",
       "      <td>66.64</td>\n",
       "      <td>64.0</td>\n",
       "      <td>65.156579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>930</th>\n",
       "      <td>66.64</td>\n",
       "      <td>62.0</td>\n",
       "      <td>65.156579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>931</th>\n",
       "      <td>66.64</td>\n",
       "      <td>61.0</td>\n",
       "      <td>65.156579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>932</th>\n",
       "      <td>65.27</td>\n",
       "      <td>66.5</td>\n",
       "      <td>64.229630</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>933</th>\n",
       "      <td>65.27</td>\n",
       "      <td>57.0</td>\n",
       "      <td>64.229630</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>934 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     MidParent  Child  Prediction\n",
       "0        75.43   73.2   70.100000\n",
       "1        75.43   69.2   70.100000\n",
       "2        75.43   69.0   70.100000\n",
       "3        75.43   69.0   70.100000\n",
       "4        73.66   73.5   70.415789\n",
       "..         ...    ...         ...\n",
       "929      66.64   64.0   65.156579\n",
       "930      66.64   62.0   65.156579\n",
       "931      66.64   61.0   65.156579\n",
       "932      65.27   66.5   64.229630\n",
       "933      65.27   57.0   64.229630\n",
       "\n",
       "[934 rows x 3 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heights_with_predictions = heights.copy()\n",
    "\n",
    "heights_with_predictions['Prediction'] = heights['MidParent'].map(predict_child)\n",
    "\n",
    "heights_with_predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 46992 characters]\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(8,5))\n",
    "\n",
    "ax.scatter(heights_with_predictions['MidParent'], \n",
    "           heights_with_predictions['Child'], \n",
    "           label='Child', \n",
    "           color='darkblue')\n",
    "\n",
    "ax.scatter(heights_with_predictions['MidParent'], \n",
    "           heights_with_predictions['Prediction'], \n",
    "           label='Prediction', \n",
    "           color='gold')\n",
    "\n",
    "x_label = 'MidParent'\n",
    "\n",
    "y_label = ''\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "ax.legend(bbox_to_anchor=(1.04,1), loc=\"upper left\", frameon=False)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Measuring in Standard Units\n",
    "Let's see if we can find a way to identify this line. First, notice that linear association doesn't depend on the units of measurement – we might as well measure both variables in standard units. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def standard_units(xyz):\n",
    "    \"Convert any array of numbers to standard units.\"\n",
    "    return (xyz - np.mean(xyz))/np.std(xyz)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MidParent SU</th>\n",
       "      <th>Child SU</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3.454652</td>\n",
       "      <td>1.804156</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3.454652</td>\n",
       "      <td>0.686005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.454652</td>\n",
       "      <td>0.630097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.454652</td>\n",
       "      <td>0.630097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.472085</td>\n",
       "      <td>1.888017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>929</th>\n",
       "      <td>-1.424873</td>\n",
       "      <td>-0.767591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>930</th>\n",
       "      <td>-1.424873</td>\n",
       "      <td>-1.326667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>931</th>\n",
       "      <td>-1.424873</td>\n",
       "      <td>-1.606205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>932</th>\n",
       "      <td>-2.185390</td>\n",
       "      <td>-0.068747</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>933</th>\n",
       "      <td>-2.185390</td>\n",
       "      <td>-2.724356</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>934 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     MidParent SU  Child SU\n",
       "0        3.454652  1.804156\n",
       "1        3.454652  0.686005\n",
       "2        3.454652  0.630097\n",
       "3        3.454652  0.630097\n",
       "4        2.472085  1.888017\n",
       "..            ...       ...\n",
       "929     -1.424873 -0.767591\n",
       "930     -1.424873 -1.326667\n",
       "931     -1.424873 -1.606205\n",
       "932     -2.185390 -0.068747\n",
       "933     -2.185390 -2.724356\n",
       "\n",
       "[934 rows x 2 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heights_SU = pd.DataFrame(\n",
    "    {'MidParent SU':standard_units(heights['MidParent']),\n",
    "    'Child SU':standard_units(heights['Child'])}\n",
    ")\n",
    "heights_SU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On this scale, we can calculate our predictions exactly as before. But first we have to figure out how to convert our old definition of \"close\" points to a value on the new scale. We had said that midparent heights were \"close\" if they were within 0.5 inches of each other. Since standard units measure distances in units of SDs, we have to figure out how many SDs of midparent height correspond to 0.5 inches.\n",
    "\n",
    "One SD of midparent heights is about 1.8 inches. So 0.5 inches is about 0.28 SDs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.8014050969207576"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sd_midparent = np.std(heights.iloc[:,0])\n",
    "sd_midparent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.27756111096536695"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "0.5/sd_midparent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " We are now ready to modify our prediction function to make predictions on the standard units scale. All that has changed is that we are using the table of values in standard units, and defining \"close\" as above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict_child_su(mpht_su):\n",
    "    \"\"\"Return a prediction of the height (in standard units) of a child \n",
    "    whose parents have a midparent height of mpht_su in standard units.\n",
    "    \"\"\"\n",
    "    close = 0.5/sd_midparent\n",
    "    close_points = heights_SU[(heights_SU['MidParent SU'] >= mpht_su-close) & (heights_SU['MidParent SU'] <= mpht_su+close)]\n",
    "    return close_points['Child SU'].mean()   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MidParent SU</th>\n",
       "      <th>Child SU</th>\n",
       "      <th>Prediction SU</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3.454652</td>\n",
       "      <td>1.804156</td>\n",
       "      <td>0.937589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3.454652</td>\n",
       "      <td>0.686005</td>\n",
       "      <td>0.937589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.454652</td>\n",
       "      <td>0.630097</td>\n",
       "      <td>0.937589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.454652</td>\n",
       "      <td>0.630097</td>\n",
       "      <td>0.937589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2.472085</td>\n",
       "      <td>1.888017</td>\n",
       "      <td>1.025864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>929</th>\n",
       "      <td>-1.424873</td>\n",
       "      <td>-0.767591</td>\n",
       "      <td>-0.444284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>930</th>\n",
       "      <td>-1.424873</td>\n",
       "      <td>-1.326667</td>\n",
       "      <td>-0.444284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>931</th>\n",
       "      <td>-1.424873</td>\n",
       "      <td>-1.606205</td>\n",
       "      <td>-0.444284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>932</th>\n",
       "      <td>-2.185390</td>\n",
       "      <td>-0.068747</td>\n",
       "      <td>-0.703401</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>933</th>\n",
       "      <td>-2.185390</td>\n",
       "      <td>-2.724356</td>\n",
       "      <td>-0.703401</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>934 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     MidParent SU  Child SU  Prediction SU\n",
       "0        3.454652  1.804156       0.937589\n",
       "1        3.454652  0.686005       0.937589\n",
       "2        3.454652  0.630097       0.937589\n",
       "3        3.454652  0.630097       0.937589\n",
       "4        2.472085  1.888017       1.025864\n",
       "..            ...       ...            ...\n",
       "929     -1.424873 -0.767591      -0.444284\n",
       "930     -1.424873 -1.326667      -0.444284\n",
       "931     -1.424873 -1.606205      -0.444284\n",
       "932     -2.185390 -0.068747      -0.703401\n",
       "933     -2.185390 -2.724356      -0.703401\n",
       "\n",
       "[934 rows x 3 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heights_with_su_predictions = heights_SU.copy()\n",
    "\n",
    "heights_with_su_predictions['Prediction SU'] = heights_SU['MidParent SU'].map(predict_child_su)\n",
    "\n",
    "heights_with_su_predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 46936 characters]\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(8,5))\n",
    "\n",
    "ax.scatter(heights_with_su_predictions['MidParent SU'], \n",
    "           heights_with_su_predictions['Child SU'], \n",
    "           label='Child_SU', \n",
    "           color='darkblue')\n",
    "\n",
    "ax.scatter(heights_with_su_predictions['MidParent SU'], \n",
    "           heights_with_su_predictions['Prediction SU'], \n",
    "           label='Prediction_SU', \n",
    "           color='gold')\n",
    "\n",
    "x_label = 'MidParent SU'\n",
    "\n",
    "y_label = ''\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "ax.legend(bbox_to_anchor=(1.04,1), loc=\"upper left\", frameon=False)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This plot looks exactly like the plot drawn on the original scale. Only the numbers on the axes have changed. This confirms that we can understand the prediction process by just working in standard units."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Identifying the Line in Standard Units\n",
    "Galton's scatter plot has a *football* shape – that is, it is roughly oval like an American football. Not all scatter plots are football shaped, not even those that show linear association. But in this section we will pretend we are Galton and work only with football shaped scatter plots. In the next section, we will generalize our analysis to other shapes of plots.\n",
    "\n",
    "Here is a football shaped scatter plot with both variables measured in standard units. The 45 degree line is shown in red."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 86008 characters]\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "r = 0.5\n",
    "x_demo = np.random.normal(0, 1, 10000)\n",
    "z_demo = np.random.normal(0, 1, 10000)\n",
    "y_demo = r*x_demo + np.sqrt(1 - r**2)*z_demo\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "\n",
    "plt.xlim(-4, 4)\n",
    "\n",
    "plt.ylim(-4, 4)\n",
    "\n",
    "plt.scatter(x_demo, y_demo, s=10)\n",
    "\n",
    "plt.plot([-4,4],[-4,4], color='r', lw=2)\n",
    "\n",
    "plt.xlabel('x in standard units')\n",
    "\n",
    "plt.ylabel('y in standard units')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But the 45 degree line is not the line that picks off the centers of the vertical strips. You can see that in the figure below, where the vertical line at 1.5 standard units is shown in black. The points on the scatter plot near the black line all have heights roughly in the -2 to 3 range. The red line is too high to pick off the center."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 87004 characters]\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "r = 0.5\n",
    "x_demo = np.random.normal(0, 1, 10000)\n",
    "z_demo = np.random.normal(0, 1, 10000)\n",
    "y_demo = r*x_demo + np.sqrt(1 - r**2)*z_demo\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "\n",
    "plt.xlim(-4, 4)\n",
    "\n",
    "plt.ylim(-4, 4)\n",
    "\n",
    "plt.scatter(x_demo, y_demo, s=10)\n",
    "\n",
    "plt.plot([-4,4],[-4,4], color='r', lw=2)\n",
    "\n",
    "plt.plot([1.5,1.5], [-4,4], color='k', lw=2)\n",
    "\n",
    "plt.xlabel('x in standard units')\n",
    "\n",
    "plt.ylabel('y in standard units');\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the 45 degree line is not the \"graph of averages.\" That line is the green one shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 92660 characters]\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "r = 0.5\n",
    "x_demo = np.random.normal(0, 1, 10000)\n",
    "z_demo = np.random.normal(0, 1, 10000)\n",
    "y_demo = r*x_demo + np.sqrt(1 - r**2)*z_demo\n",
    "\n",
    "plt.figure(figsize=(6,6))\n",
    "\n",
    "plt.xlim(-4, 4)\n",
    "\n",
    "plt.ylim(-4, 4)\n",
    "\n",
    "plt.scatter(x_demo, y_demo, s=10)\n",
    "\n",
    "plt.plot([-4, 4], [-4*0.6,4*0.6], color='g', lw=2)\n",
    "\n",
    "plt.plot([-4,4],[-4,4], color='r', lw=2)\n",
    "\n",
    "plt.plot([1.5,1.5], [-4,4], color='k', lw=2)\n",
    "\n",
    "plt.xlabel('x in standard units')\n",
    "\n",
    "plt.ylabel('y in standard units')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both lines go through the origin (0, 0). The green line goes through the centers of the vertical strips (at least roughly), and is *flatter* than the red 45 degree line. \n",
    "\n",
    "The slope of the 45 degree line is 1. So the slope of the green \"graph of averages\" line is a value that is positive but less than 1. \n",
    "\n",
    "What value could that be? You've guessed it – it's $r$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Regression Line, in Standard Units\n",
    "The green \"graph of averages\" line is called the *regression line*, for reasons we will explain shortly. But first, let's simulate some football shaped scatter plots with different values of $r$, and see how the line changes. In each case, the red 45 degree line has been drawn for comparison.\n",
    "\n",
    "The function that performs the simulation is called `regression_line` and takes $r$ as its argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [],
   "source": [
    "\n",
    "def regression_line(r):\n",
    "    x = np.random.normal(0, 1, 10000)\n",
    "    z = np.random.normal(0, 1, 10000)\n",
    "    y = r*x + (np.sqrt(1-r**2))*z\n",
    "    plt.figure(figsize=(6, 6))\n",
    "    plt.xlim(-4, 4)\n",
    "    plt.ylim(-4, 4)\n",
    "    plt.scatter(x, y)\n",
    "    plt.plot([-4, 4], [-4*r,4*r], color='g', lw=2)\n",
    "    if r >= 0:\n",
    "        plt.plot([-4,4],[-4,4], lw=2, color='r')\n",
    "    else:\n",
    "        plt.plot([-4,4], [4,-4], lw=2, color='r')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 38124 characters]\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "regression_line(0.95)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 53636 characters]\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "regression_line(0.6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When $r$ is close to 1, the scatter plot, the 45 degree line, and the regression line are all very close to each other. But for more moderate values of $r$, the regression line is noticeably flatter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Regression Effect\n",
    "In terms of prediction, this means that for a parents whose midparent height is at 1.5 standard units, our prediction of the child's height is somewhat *less* than 1.5 standard units. If the midparent height is 2 standard units, we predict that the child's height will be somewhat less than 2 standard units.\n",
    "\n",
    "In other words, we predict that the child will be somewhat closer to average than the parents were.\n",
    "\n",
    "This didn't please Sir Francis Galton. He had been hoping that exceptionally tall parents would have children who were just as exceptionally tall. However, the data were clear, and Galton realized that the tall parents have children who are not quite as exceptionally tall, on average. Frustrated, Galton called this phenomenon \"regression to mediocrity.\" \n",
    "\n",
    "Galton also noticed that exceptionally short parents had children who were somewhat taller relative to their generation, on average. In general, individuals who are away from average on one variable are expected to be not quite as far away from average on the other. This is called the *regression effect*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Equation of the Regression Line ###\n",
    "In regression, we use the value of one variable (which we will call $x$) to predict the value of another (which we will call $y$). When the variables $x$ and $y$ are measured in standard units, the regression line for predicting $y$ based on $x$ has slope $r$ and passes through the origin. Thus the equation of the regression line can be written as:\n",
    "\n",
    "$$\n",
    "\\mbox{estimate of }y ~=~ r \\cdot x ~~~\n",
    "\\mbox{when both variables are measured in standard units}\n",
    "$$\n",
    "\n",
    "In the original units of the data, this becomes\n",
    "\n",
    "$$\n",
    "\\frac{\\mbox{estimate of}~y ~-~\\mbox{average of}~y}{\\mbox{SD of}~y}\n",
    "~=~ r \\times \n",
    "\\frac{\\mbox{the given}~x ~-~\\mbox{average of}~x}{\\mbox{SD of}~x}\n",
    "$$\n",
    "\n",
    "\n",
    "![regline](../../../images/regline.png)\n",
    "\n",
    "The slope and intercept of the regression line in original units can be derived from the diagram above. \n",
    "\n",
    "$$\n",
    "\\mathbf{\\mbox{slope of the regression line}} ~=~ r \\cdot\n",
    "\\frac{\\mbox{SD of }y}{\\mbox{SD of }x}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\mathbf{\\mbox{intercept of the regression line}} ~=~\n",
    "\\mbox{average of }y ~-~ \\mbox{slope} \\cdot \\mbox{average of }x\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The three functions below compute the correlation, slope, and intercept. All of them take three arguments: the name of the table, the label of the column containing $x$, and the label of the column containing $y$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def correlation(t, label_x, label_y):\n",
    "    return np.mean(standard_units(t[label_x])*standard_units(t[label_y]))\n",
    "\n",
    "def slope(t, label_x, label_y):\n",
    "    r = correlation(t, label_x, label_y)\n",
    "    return r*np.std(t[label_y])/np.std(t[label_x])\n",
    "\n",
    "def intercept(t, label_x, label_y):\n",
    "    return np.mean(t[label_y]) - slope(t, label_x, label_y)*np.mean(t[label_x])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Regression Line and Galton's Data\n",
    "The correlation between midparent height and child's height is 0.32:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.320949896063959"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "galton_r = correlation(heights, 'MidParent', 'Child')\n",
    "galton_r"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also find the equation of the regression line for predicting the child's height based on midparent height."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.6373608969694783, 22.636240549589772)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "galton_slope = slope(heights, 'MidParent', 'Child')\n",
    "galton_intercept = intercept(heights, 'MidParent', 'Child')\n",
    "galton_slope, galton_intercept"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The equation of the regression line is\n",
    "\n",
    "$$\n",
    "\\mbox{estimate of child's height} ~=~ 0.64 \\cdot \\mbox{midparent height} ~+~ 22.64\n",
    "$$\n",
    "\n",
    "This is also known as the *regression equation.* The principal use of the regression equation is to predict $y$ based on $x$.\n",
    "\n",
    "For example, for a midparent height of 70.48 inches, the regression equation predicts the child's height to be 67.56 inches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "67.55743656799861"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "galton_slope*70.48 + galton_intercept"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our original prediction, created by taking the average height of all children who had midparent heights close to 70.48, came out to be pretty close: 67.63 inches compared to the regression line's prediction of 67.55 inches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MidParent</th>\n",
       "      <th>Child</th>\n",
       "      <th>Prediction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>70.48</td>\n",
       "      <td>74.0</td>\n",
       "      <td>67.634239</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>70.48</td>\n",
       "      <td>70.0</td>\n",
       "      <td>67.634239</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>70.48</td>\n",
       "      <td>68.0</td>\n",
       "      <td>67.634239</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    MidParent  Child  Prediction\n",
       "33      70.48   74.0   67.634239\n",
       "34      70.48   70.0   67.634239\n",
       "35      70.48   68.0   67.634239"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heights_with_predictions[heights_with_predictions['MidParent'] == 70.48].head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are all of the rows in Galton's table, along with our original predictions and the new regression predictions of the children's heights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MidParent</th>\n",
       "      <th>Child</th>\n",
       "      <th>Prediction</th>\n",
       "      <th>Regression Prediction</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75.43</td>\n",
       "      <td>73.2</td>\n",
       "      <td>70.100000</td>\n",
       "      <td>70.712373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.2</td>\n",
       "      <td>70.100000</td>\n",
       "      <td>70.712373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.0</td>\n",
       "      <td>70.100000</td>\n",
       "      <td>70.712373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.0</td>\n",
       "      <td>70.100000</td>\n",
       "      <td>70.712373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>73.66</td>\n",
       "      <td>73.5</td>\n",
       "      <td>70.415789</td>\n",
       "      <td>69.584244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>929</th>\n",
       "      <td>66.64</td>\n",
       "      <td>64.0</td>\n",
       "      <td>65.156579</td>\n",
       "      <td>65.109971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>930</th>\n",
       "      <td>66.64</td>\n",
       "      <td>62.0</td>\n",
       "      <td>65.156579</td>\n",
       "      <td>65.109971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>931</th>\n",
       "      <td>66.64</td>\n",
       "      <td>61.0</td>\n",
       "      <td>65.156579</td>\n",
       "      <td>65.109971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>932</th>\n",
       "      <td>65.27</td>\n",
       "      <td>66.5</td>\n",
       "      <td>64.229630</td>\n",
       "      <td>64.236786</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>933</th>\n",
       "      <td>65.27</td>\n",
       "      <td>57.0</td>\n",
       "      <td>64.229630</td>\n",
       "      <td>64.236786</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>934 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     MidParent  Child  Prediction  Regression Prediction\n",
       "0        75.43   73.2   70.100000              70.712373\n",
       "1        75.43   69.2   70.100000              70.712373\n",
       "2        75.43   69.0   70.100000              70.712373\n",
       "3        75.43   69.0   70.100000              70.712373\n",
       "4        73.66   73.5   70.415789              69.584244\n",
       "..         ...    ...         ...                    ...\n",
       "929      66.64   64.0   65.156579              65.109971\n",
       "930      66.64   62.0   65.156579              65.109971\n",
       "931      66.64   61.0   65.156579              65.109971\n",
       "932      65.27   66.5   64.229630              64.236786\n",
       "933      65.27   57.0   64.229630              64.236786\n",
       "\n",
       "[934 rows x 4 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heights_with_predictions['Regression Prediction'] =  galton_slope*heights['MidParent'] + galton_intercept\n",
    "\n",
    "heights_with_predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 52996 characters]\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize=(8,5))\n",
    "\n",
    "ax.scatter(heights_with_predictions['MidParent'], \n",
    "           heights_with_predictions['Child'], \n",
    "           label='Child', \n",
    "           color='darkblue')\n",
    "\n",
    "ax.scatter(heights_with_predictions['MidParent'], \n",
    "           heights_with_predictions['Prediction'], \n",
    "           label='Prediction', \n",
    "           color='gold')\n",
    "\n",
    "ax.scatter(heights_with_predictions['MidParent'], \n",
    "           heights_with_predictions['Regression Prediction'], \n",
    "           label='Regression Prediction', \n",
    "           color='lightblue')\n",
    "\n",
    "x_label = 'MidParent'\n",
    "\n",
    "y_label = ''\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "ax.legend(bbox_to_anchor=(1.04,1), loc=\"upper left\", frameon=False)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The grey dots show the regression predictions, all on the regression line. Notice how the line is very close to the gold graph of averages. For these data, the regression line does a good job of approximating the centers of the vertical strips."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fitted Values\n",
    "The predictions all lie on the line and are known as the \"fitted values\". The function `fit` takes the name of the table and the labels of $x$ and $y$, and returns an array of fitted values, one fitted value for each point in the scatter plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit(table, x, y):\n",
    "    \"\"\"Return the height of the regression line at each x value.\"\"\"\n",
    "    a = slope(table, x, y)\n",
    "    b = intercept(table, x, y)\n",
    "    return a * table[x] + b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>MidParent</th>\n",
       "      <th>Child</th>\n",
       "      <th>Fitted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>75.43</td>\n",
       "      <td>73.2</td>\n",
       "      <td>70.712373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.2</td>\n",
       "      <td>70.712373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.0</td>\n",
       "      <td>70.712373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>75.43</td>\n",
       "      <td>69.0</td>\n",
       "      <td>70.712373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>73.66</td>\n",
       "      <td>73.5</td>\n",
       "      <td>69.584244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>929</th>\n",
       "      <td>66.64</td>\n",
       "      <td>64.0</td>\n",
       "      <td>65.109971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>930</th>\n",
       "      <td>66.64</td>\n",
       "      <td>62.0</td>\n",
       "      <td>65.109971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>931</th>\n",
       "      <td>66.64</td>\n",
       "      <td>61.0</td>\n",
       "      <td>65.109971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>932</th>\n",
       "      <td>65.27</td>\n",
       "      <td>66.5</td>\n",
       "      <td>64.236786</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>933</th>\n",
       "      <td>65.27</td>\n",
       "      <td>57.0</td>\n",
       "      <td>64.236786</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>934 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     MidParent  Child     Fitted\n",
       "0        75.43   73.2  70.712373\n",
       "1        75.43   69.2  70.712373\n",
       "2        75.43   69.0  70.712373\n",
       "3        75.43   69.0  70.712373\n",
       "4        73.66   73.5  69.584244\n",
       "..         ...    ...        ...\n",
       "929      66.64   64.0  65.109971\n",
       "930      66.64   62.0  65.109971\n",
       "931      66.64   61.0  65.109971\n",
       "932      65.27   66.5  64.236786\n",
       "933      65.27   57.0  64.236786\n",
       "\n",
       "[934 rows x 3 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heights['Fitted'] = fit(heights, 'MidParent', 'Child')\n",
    "\n",
    "heights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is easier to see the line in the graph below than in the one above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 44536 characters]\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#heights.with_column('Fitted', fit(heights, 'MidParent', 'Child')).scatter('MidParent')\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(8,5))\n",
    "\n",
    "ax.scatter(heights['MidParent'], \n",
    "           heights['Child'], \n",
    "           label='Child', \n",
    "           color='darkblue'\n",
    "          )\n",
    "\n",
    "ax.scatter(heights['MidParent'], \n",
    "           heights['Fitted'], \n",
    "           label='Fitted', \n",
    "           color='gold'\n",
    "          )\n",
    "\n",
    "x_label = 'MidParent'\n",
    "\n",
    "y_label = ''\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "ax.legend(bbox_to_anchor=(1.04,1), loc=\"upper left\", frameon=False)\n",
    "\n",
    "plt.show()\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to draw the line is to use the option `fit_line=True` with the Table method `scatter`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 50252 characters]\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#heights.scatter('MidParent', fit_line=True)\n",
    "\n",
    "x_data = heights['MidParent']\n",
    "\n",
    "y_data = heights['Child']\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(8,6))\n",
    "\n",
    "m, b = np.polyfit(x_data, y_data, 1)\n",
    "\n",
    "minx, maxx = np.min(x_data),np.max(x_data)\n",
    "\n",
    "ax.plot([minx,maxx],[m*minx+b,m*maxx+b], label='Best Fit Line', color='red')\n",
    "\n",
    "ax.scatter(heights['MidParent'], \n",
    "           heights['Child'],   \n",
    "           color='darkblue'\n",
    "          )\n",
    "\n",
    "\n",
    "x_label = 'MidParent'\n",
    "\n",
    "y_label = 'Child'\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "ax.legend(bbox_to_anchor=(1.04,1), loc=\"upper left\", frameon=False)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Units of Measurement of the Slope\n",
    "The slope is a ratio, and it worth taking a moment to study the units in which it is measured. Our example comes from the familiar dataset about mothers who gave birth in a hospital system. The scatter plot of pregnancy weights versus heights looks like a football that has been used in one game too many, but it's close enough to a football that we can justify putting our fitted line through it. In later sections we will see how to make such justifications more formal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "baby = pd.read_csv(path_data + 'baby.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 50468 characters]\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#baby.plot.scatter('Maternal Height', 'Maternal Pregnancy Weight', fit_line=True)\n",
    "\n",
    "x_data = baby['Maternal Height']\n",
    "\n",
    "y_data = baby['Maternal Pregnancy Weight']\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(8,6))\n",
    "\n",
    "m, b = np.polyfit(x_data, y_data, 1)\n",
    "\n",
    "minx, maxx = np.min(x_data),np.max(x_data)\n",
    "\n",
    "ax.plot([minx,maxx],[m*minx+b,m*maxx+b], label='Best Fit Line', color='red')\n",
    "\n",
    "ax.scatter(baby['Maternal Height'], \n",
    "           baby['Maternal Pregnancy Weight'],   \n",
    "           color='darkblue'\n",
    "          )\n",
    "\n",
    "\n",
    "x_label = 'Maternal Height'\n",
    "\n",
    "y_label = 'Maternal Pregnancy Weight'\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "ax.legend(bbox_to_anchor=(1.04,1), loc=\"upper left\", frameon=False)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.572846259275052"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "slope(baby, 'Maternal Height', 'Maternal Pregnancy Weight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The slope of the regression line is **3.57 pounds per inch**. This means that for two women who are 1 inch apart in height, our prediction of pregnancy weight will differ by 3.57 pounds. For a woman who is 2 inches taller than another, our prediction of pregnancy weight will be \n",
    "$$\n",
    "2 \\times 3.57 ~=~ 7.14\n",
    "$$\n",
    "pounds more than our prediction for the shorter woman.\n",
    "\n",
    "Notice that the successive vertical strips in the scatter plot are one inch apart, because the heights have been rounded to the nearest inch. Another way to think about the slope is to take any two consecutive strips (which are necessarily 1 inch apart), corresponding to two groups of women who are separated by 1 inch in height. The slope of 3.57 pounds per inch means that the average pregnancy weight of the taller group is about 3.57 pounds more than that of the shorter group."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example\n",
    "Suppose that our goal is to use regression to estimate the height of a basset hound based on its weight, using a sample that looks consistent with the regression model. Suppose the observed correlation $r$ is 0.5, and that the summary statistics for the two variables are as in the table below: \n",
    "\n",
    "|       |**average**    |**SD**     |   \n",
    "|------:|:-------------:|:---------:|\n",
    "|height |14 inches      |2 inches   |\n",
    "|weight |50 pounds      |5 pounds   |\n",
    "\n",
    "To calculate the equation of the regression line, we need the slope and the intercept.\n",
    "\n",
    "$$\n",
    "\\mbox{slope} ~=~ \\frac{r \\cdot \\mbox{SD of }y}{\\mbox{SD of }x} ~=~\n",
    "\\frac{0.5 \\cdot 2 \\mbox{ inches}}{5 \\mbox{ pounds}} ~=~ 0.2 ~\\mbox{inches per pound}\n",
    "$$\n",
    "\n",
    "\n",
    "$$\n",
    "\\mbox{intercept} ~=~ \\mbox{average of }y - \\mbox{slope}\\cdot \\mbox{average of } x\n",
    "~=~ 14 \\mbox{ inches} ~-~ 0.2 \\mbox{ inches per pound} \\cdot 50 \\mbox{ pounds}\n",
    "~=~ 4 \\mbox{ inches}\n",
    "$$\n",
    "\n",
    "The equation of the regression line allows us to calculate the estimated height, in inches,\n",
    "based on a given weight in pounds:\n",
    "\n",
    "$$\n",
    "\\mbox{estimated height} ~=~ 0.2 \\cdot \\mbox{given weight} ~+~ 4\n",
    "$$\n",
    "\n",
    "The slope of the line is measures the increase in the estimated height per unit increase in weight. The slope is positive, and it is important to note that this does not mean that we think basset hounds get taller if they put on weight. The slope reflects the difference in the average heights of two groups of dogs that are 1 pound apart in weight. Specifically, consider a group of dogs whose weight is $w$ pounds, and the group whose weight is $w+1$ pounds. The second group is estimated to be 0.2 inches taller, on average. This is true for all values of $w$ in the sample.\n",
    "\n",
    "In general, the slope of the regression line can be interpreted as the average increase in $y$ per unit increase in $x$. Note that if the slope is negative, then for every unit increase in $x$, the average of $y$ decreases."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Endnote\n",
    "Even though we won't establish the mathematical basis for the regression equation, we can see that it gives pretty good predictions when the scatter plot is football shaped. It is a surprising mathematical fact that no matter what the shape of the scatter plot, the same equation gives the \"best\" among all straight lines. That's the topic of the next section."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
