{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [],
   "source": [
    "path_data = '../../data/'\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confidence Intervals\n",
    "We have developed a method for estimating a parameter by using random sampling and the bootstrap. Our method produces an interval of estimates, to account for chance variability in the random sample. By providing an interval of estimates instead of just one estimate, we give ourselves some wiggle room.\n",
    "\n",
    "In the previous example we saw that our process of estimation produced a good interval about 95% of the time, a \"good\" interval being one that contains the parameter. We say that we are *95% confident* that the process results in a good interval. Our interval of estimates is called a *95% confidence interval* for the parameter, and 95% is called the *confidence level* of the interval.\n",
    "\n",
    "The situation in the previous example was a bit unusual. Because we happened to know the value of the parameter, we were able to check whether an interval was good or not so good, and this in turn helped us to see that our process of estimation captured the parameter about 95 out of every 100 times we used it.\n",
    "\n",
    "But usually, data scientists don't know the value of the parameter. That is the reason they want to estimate it in the first place. In such situations, they provide an interval of estimates for the unknown parameter by using methods like the one we have developed. Because of statistical theory and demonstrations like the one we have seen, data scientists can be confident that their process of generating the interval results in a good interval a known percent of the time."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Confidence Interval for a Population Median: Bootstrap Percentile Method ###\n",
    "\n",
    "We will now use the bootstrap method to estimate an unknown population median. The data come from a sample of newborns in a large hospital system; we will treat it as if it were a simple random sample though the sampling was done in multiple stages. [Stat Labs](https://www.stat.berkeley.edu/~statlabs/) by Deborah Nolan and Terry Speed has details about a larger dataset from which this set is drawn. \n",
    "\n",
    "The table `baby` contains the following variables for mother-baby pairs: the baby's birth weight in ounces, the number of gestational days, the mother's age in completed years, the mother's height in inches, pregnancy weight in pounds, and whether or not the mother smoked during pregnancy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "baby = pd.read_csv(path_data + 'baby.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Birth Weight</th>\n",
       "      <th>Gestational Days</th>\n",
       "      <th>Maternal Age</th>\n",
       "      <th>Maternal Height</th>\n",
       "      <th>Maternal Pregnancy Weight</th>\n",
       "      <th>Maternal Smoker</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>120</td>\n",
       "      <td>284</td>\n",
       "      <td>27</td>\n",
       "      <td>62</td>\n",
       "      <td>100</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>113</td>\n",
       "      <td>282</td>\n",
       "      <td>33</td>\n",
       "      <td>64</td>\n",
       "      <td>135</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>128</td>\n",
       "      <td>279</td>\n",
       "      <td>28</td>\n",
       "      <td>64</td>\n",
       "      <td>115</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>108</td>\n",
       "      <td>282</td>\n",
       "      <td>23</td>\n",
       "      <td>67</td>\n",
       "      <td>125</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>136</td>\n",
       "      <td>286</td>\n",
       "      <td>25</td>\n",
       "      <td>62</td>\n",
       "      <td>93</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1169</th>\n",
       "      <td>113</td>\n",
       "      <td>275</td>\n",
       "      <td>27</td>\n",
       "      <td>60</td>\n",
       "      <td>100</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1170</th>\n",
       "      <td>128</td>\n",
       "      <td>265</td>\n",
       "      <td>24</td>\n",
       "      <td>67</td>\n",
       "      <td>120</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1171</th>\n",
       "      <td>130</td>\n",
       "      <td>291</td>\n",
       "      <td>30</td>\n",
       "      <td>65</td>\n",
       "      <td>150</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1172</th>\n",
       "      <td>125</td>\n",
       "      <td>281</td>\n",
       "      <td>21</td>\n",
       "      <td>65</td>\n",
       "      <td>110</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1173</th>\n",
       "      <td>117</td>\n",
       "      <td>297</td>\n",
       "      <td>38</td>\n",
       "      <td>65</td>\n",
       "      <td>129</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1174 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Birth Weight  Gestational Days  Maternal Age  Maternal Height  \\\n",
       "0              120               284            27               62   \n",
       "1              113               282            33               64   \n",
       "2              128               279            28               64   \n",
       "3              108               282            23               67   \n",
       "4              136               286            25               62   \n",
       "...            ...               ...           ...              ...   \n",
       "1169           113               275            27               60   \n",
       "1170           128               265            24               67   \n",
       "1171           130               291            30               65   \n",
       "1172           125               281            21               65   \n",
       "1173           117               297            38               65   \n",
       "\n",
       "      Maternal Pregnancy Weight  Maternal Smoker  \n",
       "0                           100            False  \n",
       "1                           135            False  \n",
       "2                           115             True  \n",
       "3                           125             True  \n",
       "4                            93            False  \n",
       "...                         ...              ...  \n",
       "1169                        100            False  \n",
       "1170                        120            False  \n",
       "1171                        150             True  \n",
       "1172                        110            False  \n",
       "1173                        129            False  \n",
       "\n",
       "[1174 rows x 6 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "baby"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Birth weight is an important factor in the health of a newborn infant – smaller babies tend to need more medical care in their first days than larger newborns. It is therefore helpful to have an estimate of birth weight before the baby is born. One way to do this is to examine the relationship between birth weight and the number of gestational days. \n",
    "\n",
    "A simple measure of this relationship is the ratio of birth weight to the number of gestational days. The table `ratios` contains the first two columns of `baby`, as well as a column of the ratios. The first entry in that column was calcualted as follows:\n",
    "\n",
    "$$\n",
    "\\frac{120~\\mbox{ounces}}{284~\\mbox{days}} ~\\approx ~ 0.4225~ \\mbox{ounces per day}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratios = baby[['Birth Weight', 'Gestational Days']]\n",
    "\n",
    "ratios['Ratio BW/GD'] = baby['Birth Weight']/baby['Gestational Days']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Birth Weight</th>\n",
       "      <th>Gestational Days</th>\n",
       "      <th>Ratio BW/GD</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>120</td>\n",
       "      <td>284</td>\n",
       "      <td>0.422535</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>113</td>\n",
       "      <td>282</td>\n",
       "      <td>0.400709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>128</td>\n",
       "      <td>279</td>\n",
       "      <td>0.458781</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>108</td>\n",
       "      <td>282</td>\n",
       "      <td>0.382979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>136</td>\n",
       "      <td>286</td>\n",
       "      <td>0.475524</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1169</th>\n",
       "      <td>113</td>\n",
       "      <td>275</td>\n",
       "      <td>0.410909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1170</th>\n",
       "      <td>128</td>\n",
       "      <td>265</td>\n",
       "      <td>0.483019</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1171</th>\n",
       "      <td>130</td>\n",
       "      <td>291</td>\n",
       "      <td>0.446735</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1172</th>\n",
       "      <td>125</td>\n",
       "      <td>281</td>\n",
       "      <td>0.444840</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1173</th>\n",
       "      <td>117</td>\n",
       "      <td>297</td>\n",
       "      <td>0.393939</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1174 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Birth Weight  Gestational Days  Ratio BW/GD\n",
       "0              120               284     0.422535\n",
       "1              113               282     0.400709\n",
       "2              128               279     0.458781\n",
       "3              108               282     0.382979\n",
       "4              136               286     0.475524\n",
       "...            ...               ...          ...\n",
       "1169           113               275     0.410909\n",
       "1170           128               265     0.483019\n",
       "1171           130               291     0.446735\n",
       "1172           125               281     0.444840\n",
       "1173           117               297     0.393939\n",
       "\n",
       "[1174 rows x 3 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratios"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a histogram of the ratios."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 20872 characters]\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "unit = ''\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(8,6))\n",
    "\n",
    "ax.hist(ratios['Ratio BW/GD'], density=True, color='blue', alpha=0.8, ec='white')\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Ratio BW/GD'\n",
    "\n",
    "ax.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.xticks(rotation=90)\n",
    "\n",
    "plt.title('');\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At first glance the histogram looks quite symmetric, with the density at its maximum over the interval 4 ounces per day to 4.5 ounces per day. But a closer look reveals that some of the ratios were quite large by comparison. The maximum value of the ratios was just over 0.78 ounces per day, almost double the typical value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Birth Weight</th>\n",
       "      <th>Gestational Days</th>\n",
       "      <th>Ratio BW/GD</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>238</th>\n",
       "      <td>116</td>\n",
       "      <td>148</td>\n",
       "      <td>0.783784</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Birth Weight  Gestational Days  Ratio BW/GD\n",
       "238           116               148     0.783784"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratios_1 = ratios.sort_values(by=['Ratio BW/GD'], ascending=False)\n",
    "\n",
    "ratios_1.iloc[[0]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The median gives a sense of the typical ratio because it is unaffected by the very large or very small ratios. The median ratio in the sample is about 0.429 ounces per day."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.42907801418439717"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.median(ratios.iloc[:,2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But what was the median in the population? We don't know, so we will estimate it. \n",
    "\n",
    "Our method will be exactly the same as in the previous section. We will bootstrap the sample 5,000 times resulting in 5,000 estimates of the median. Our 95% confidence interval will be the \"middle 95%\" of all of our estimates.\n",
    "\n",
    "Recall the function `bootstrap_median` defined in the previous section. We will call this function and construct a 95% confidence interval for the median ratio in the population. Remember that the table `ratios` contains the relevant data from our original sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bootstrap_median(original_sample, label, replications):\n",
    "    \n",
    "    \"\"\"Returns an array of bootstrapped sample medians:\n",
    "    original_sample: table containing the original sample\n",
    "    label: label of column containing the variable\n",
    "    replications: number of bootstrap samples\n",
    "    \"\"\"\n",
    "    just_one_column = original_sample[[label]]\n",
    "    medians = np.array([])\n",
    "    for i in np.arange(replications):\n",
    "        bootstrap_sample = just_one_column.sample(len(just_one_column), replace=True)\n",
    "        resampled_median = np.percentile(bootstrap_sample, 50)\n",
    "        medians = np.append(medians, resampled_median)\n",
    "        \n",
    "    return medians"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Generate the medians from 5000 bootstrap samples\n",
    "bstrap_medians = bootstrap_median(ratios, 'Ratio BW/GD', 5000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.42545455, 0.43262411])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get the endpoints of the 95% confidence interval\n",
    "left = np.percentile(bstrap_medians, 2.5, interpolation='nearest')\n",
    "right = np.percentile(bstrap_medians, 97.5, interpolation='nearest')\n",
    "\n",
    "np.array([left, right])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 95% confidence interval goes from about 0.425 ounces per day to about 0.433 ounces per day. We are estimating the median \"birth weight to gestational days\" ratio in the population is somewhere in the interval 0.425 ounces per day to 0.433 ounces per day.\n",
    "\n",
    "The estimate of 0.429 based on the original sample happens to be exactly half-way in between the two ends of the interval, though that need not be true in general.\n",
    "\n",
    "To visualize our results, let us draw the empirical histogram of our bootstrapped medians and place the confidence interval on the horizontal axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 25104 characters]\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "resampled_medians = pd.DataFrame({'Bootstrap Sample Median':bstrap_medians})\n",
    "\n",
    "unit = ''\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "\n",
    "ax.hist(resampled_medians, bins=15, density=True, color='blue', alpha=0.8, ec='white', zorder=5)\n",
    "\n",
    "ax.plot(np.array([left, right]), np.array([0,0]), color='yellow', lw=8, zorder=10)\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Bootstrap Sample Median'\n",
    "\n",
    "ax.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.xticks(rotation=90)\n",
    "\n",
    "plt.title('');\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This histogram and interval resembles those we drew in the previous section, with one big difference – there is no red dot showing where the parameter is. We don't know where that dot should be, or whether it is even in the interval.\n",
    "\n",
    "We just have an interval of estimates. It is a 95% confidence interval of estimates, because the process that generates it produces a good interval about 95% of the time. That certainly beats guessing at random!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keep in mind that this interval is an approximate 95% confidence interval. There are many approximations involved in its computation. The approximation is not bad, but it is not exact."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Confidence Interval for a Population Mean: Bootstrap Percentile Method ###\n",
    "What we have done for medians can be done for means as well. Suppose we want to estimate the average age of the mothers in the population. A natural estimate is the average age of the mothers in the sample. Here is the distribution of their ages, and their average age which was about 27.2 years."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 15844 characters]\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "unit = ''\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(8,5))\n",
    "\n",
    "ax.hist(baby['Maternal Age'], density=True, color='blue', alpha=0.8, ec='white', zorder=5)\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Maternal Age'\n",
    "\n",
    "ax.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.xticks(rotation=90)\n",
    "\n",
    "plt.title('');\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "27.228279386712096"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(baby['Maternal Age'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What was the average age of the mothers in the population? We don't know the value of this parameter.\n",
    "\n",
    "Let's estimate the unknown parameter by the bootstrap method. To do this, we will edit the code for `bootstrap_median` to instead define the function `bootstrap_mean`. The code is the same except that the statistics are means instead of medians, and are collected in an array called `means` instead of `medians`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bootstrap_mean(original_sample, label, replications):\n",
    "    \n",
    "    \"\"\"Returns an array of bootstrapped sample means:\n",
    "    original_sample: table containing the original sample\n",
    "    label: label of column containing the variable\n",
    "    replications: number of bootstrap samples\n",
    "    \"\"\"\n",
    "    \n",
    "    just_one_column = original_sample[[label]]\n",
    "    means = np.array([])\n",
    "    for i in np.arange(replications):\n",
    "        bootstrap_sample = just_one_column.sample(len(just_one_column), replace=True)\n",
    "        resampled_mean = np.mean(bootstrap_sample.iloc[:,0])\n",
    "        means = np.append(means, resampled_mean)\n",
    "        \n",
    "    return means\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([26.89437819, 27.57410562])"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generate the means from 5000 bootstrap samples\n",
    "bstrap_means = bootstrap_mean(baby, 'Maternal Age', 5000)\n",
    "\n",
    "# Get the endpoints of the 95% confidence interval\n",
    "left = np.percentile(bstrap_means, 2.5, interpolation='nearest')\n",
    "right = np.percentile(bstrap_means, 97.5, interpolation='nearest')\n",
    "\n",
    "np.array([left, right])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 95% confidence interval goes from about 26.9 years to about 27.6 years. That is, we are estimating that the average age of the mothers in the population is somewhere in the interval 26.9 years to 27.6 years. \n",
    "\n",
    "Notice how close the two ends are to the average of about 27.2 years in the original sample. The sample size is very large – 1,174 mothers – and so the sample averages don't vary much. We will explore this observation further in the next chapter.\n",
    "\n",
    "The empirical histogram of the 5,000 bootstrapped means is shown below, along with the 95% confidence interval for the population mean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 20684 characters]\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "resampled_means = pd.DataFrame({'Bootstrap Sample Mean':bstrap_means})\n",
    "\n",
    "unit = ''\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "\n",
    "ax.hist(resampled_means, bins=15, density=True, color='blue', alpha=0.8, ec='white', zorder=5)\n",
    "\n",
    "ax.plot(np.array([left, right]), np.array([0,0]), color='yellow', lw=8, zorder=10)\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Bootstrap Sample Median'\n",
    "\n",
    "ax.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.xticks(rotation=90)\n",
    "\n",
    "plt.title('');\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, the average of the original sample (27.23 years) is close to the center of the interval. That's not very surprising, because each bootstrapped sample is drawn from that same original sample. The averages of the bootstrapped samples are about symmetrically distributed on either side of the average of the sample from which they were drawn."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice also that the empirical histogram of the resampled means has roughly a symmetric bell shape, even though the histogram of the sampled ages was not symmetric at all:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 15632 characters]\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "unit = ''\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(8,5))\n",
    "\n",
    "ax.hist(baby['Maternal Age'], density=True, color='blue', alpha=0.8, ec='white', zorder=5)\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Maternal Age'\n",
    "\n",
    "ax.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.title('');\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a consequence of the Central Limit Theorem of probability and statistics. In later sections, we will see what the theorem says."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### An 80% Confidence Interval\n",
    "You can use the bootstrapped sample means to construct an interval of any level of confidence. For example, to construct an 80% confidence interval for the mean age in the population, you would take the \"middle 80%\" of the resampled means. So you would want 10% of the distribution in each of the two tails, and hence the endpoints would be the 10th and 90th percentiles of the resampled means."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([27.01192504, 27.45059625])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "left_80 = np.percentile(bstrap_means, 10, interpolation='nearest')\n",
    "right_80 = np.percentile(bstrap_means, 90, interpolation='nearest')\n",
    "np.array([left_80, right_80])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 20640 characters]\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "unit = ''\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "\n",
    "ax.hist(resampled_means, bins=15, density=True, color='blue', alpha=0.8, ec='white', zorder=5)\n",
    "\n",
    "ax.plot(np.array([left_80, right_80]), np.array([0,0]), color='yellow', lw=8, zorder=10)\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Bootstrap Sample Mean'\n",
    "\n",
    "ax.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.xticks(rotation=90)\n",
    "\n",
    "plt.title('');\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This 80% confidence interval is much shorter than the 95% confidence interval. It only goes from about 27.0 years to about 27.4 years. While that's a tight set of estimates, you know that this process only produces a good interval about 80% of the time. \n",
    "\n",
    "The earlier process produced a wider interval but we had more confidence in the process that generated it.\n",
    "\n",
    "To get a narrow confidence interval at a high level of confidence, you'll have to start with a larger sample. We'll see why in the next chapter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Confidence Interval for a Population Proportion: Bootstrap Percentile Method\n",
    "In the sample, 39% of the mothers smoked during pregnancy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3909710391822828"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(baby[baby['Maternal Smoker'] == True]) / len(baby)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For what follows, it is useful to observe that this proportion can also be calculated by an array operation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.3909710391822828"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "smoking = baby['Maternal Smoker']\n",
    "np.count_nonzero(smoking)/len(smoking)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What percent of mothers in the population smoked during pregnancy? This is an unknown parameter which we can estimate by a bootstrap confidence interval. The steps in the process are analogous to those we took to estimate the population mean and median.\n",
    "\n",
    "We will start by defining a function `bootstrap_proportion` that returns an array of bootstrapped sampled proportions. Once again, we will achieve this by editing our definition of `bootstrap_median`. The only change in computation is in replacing the median of the resample by the proportion of smokers in it. The code assumes that the column of data consists of Boolean values. The other changes are only to the names of arrays, to help us read and understand our code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def bootstrap_proportion(original_sample, label, replications):\n",
    "    \n",
    "    \"\"\"Returns an array of bootstrapped sample proportions:\n",
    "    original_sample: table containing the original sample\n",
    "    label: label of column containing the Boolean variable\n",
    "    replications: number of bootstrap samples\n",
    "    \"\"\"\n",
    "    \n",
    "    just_one_column = original_sample[[label]]\n",
    "    proportions = np.array([])\n",
    "    for i in np.arange(replications):\n",
    "        bootstrap_sample = just_one_column.sample(len(just_one_column), replace=True)\n",
    "        resample_array = bootstrap_sample.iloc[:,0]\n",
    "        resampled_proportion = np.count_nonzero(resample_array)/len(resample_array)\n",
    "        proportions = np.append(proportions, resampled_proportion)\n",
    "        \n",
    "    return proportions\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us use `bootstrap_proportion` to construct an approximate 95% confidence interval for the percent of smokers among the mothers in the population. The code is analogous to the corresponding code for the mean and median."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.36286201, 0.41993186])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generate the proportions from 5000 bootstrap samples\n",
    "bstrap_props = bootstrap_proportion(baby, 'Maternal Smoker', 5000)\n",
    "\n",
    "# Get the endpoints of the 95% confidence interval\n",
    "left = np.percentile(bstrap_props, 2.5, interpolation='nearest')\n",
    "right = np.percentile(bstrap_props, 97.5, interpolation='nearest')\n",
    "\n",
    "np.array([left, right])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The confidence interval goes from about 36% to about 42%. The original sample percent of 39% is very close to the center of the interval, as you can see below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 23920 characters]\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "resampled_proportions = pd.DataFrame({'Bootstrap Sample Proportion':bstrap_props})\n",
    "\n",
    "unit = ''\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10,5))\n",
    "\n",
    "ax.hist(resampled_proportions, bins=15, density=True, color='blue', alpha=0.8, ec='white', zorder=5)\n",
    "\n",
    "ax.plot(np.array([left, right]), np.array([0,0]), color='yellow', lw=8, zorder=10)\n",
    "\n",
    "y_vals = ax.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Bootstrap Sample Proportion'\n",
    "\n",
    "ax.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.xticks(rotation=90)\n",
    "\n",
    "plt.title('');\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Care in Using the Bootstrap\n",
    "The bootstrap is an elegant and powerful method. Before using it, it is important to keep some points in mind.\n",
    "\n",
    "- Start with a large random sample. If you don't, the method might not work. Its success is based on large random samples (and hence also resamples from the sample) resembling the population. The Law of Averages says that this is likely to be true provided the random sample is large.\n",
    "\n",
    "- To approximate the probability distribution of a statistic, it is a good idea to replicate the resampling procedure as many times as possible. A few thousand replications will result in decent approximations to the distribution of sample median, especially if the distribution of the population has one peak and is not very asymmetric. We used 5,000 replications in our examples but would recommend 10,000 in general.\n",
    "\n",
    "- The bootstrap percentile method works well for estimating the population median or mean based on a large random sample. However, it has limitations, as do all methods of estimation. For example, it is not expected to do well in the following situations.\n",
    "    - The goal is to estimate the minimum or maximum value in the population, or a very low or very high percentile, or parameters that are greatly influenced by rare elements of the population.\n",
    "    - The probability distribution of the statistic is not roughly bell shaped.\n",
    "    - The original sample is very small, say less than 10 or 15.\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
