{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [],
   "source": [
    "path_data = '../../data/'\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [],
   "source": [
    "# As of Jan 2017, this census file is online here: \n",
    "data = 'http://www2.census.gov/programs-surveys/popest/datasets/2010-2015/national/asrh/nc-est2015-agesex-res.csv'\n",
    "\n",
    "# A local copy can be accessed here in case census.gov moves the file:\n",
    "# data = path_data + 'nc-est2015-agesex-res.csv'\n",
    "\n",
    "full_census_table = pd.read_csv(data)\n",
    "# pandas dataframe nuilt upon the 'data' file\n",
    "\n",
    "partial_census_table = full_census_table[['SEX', 'AGE', 'POPESTIMATE2010', 'POPESTIMATE2014']]\n",
    "# df built on 4 columns from full_census_table \n",
    "\n",
    "us_pop = partial_census_table.rename(columns={'POPESTIMATE2010':'2010','POPESTIMATE2014':'2014'})\n",
    "# partial census df with renamed columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example: Trends in Gender\n",
    "\n",
    "We are now equipped with enough coding skills to examine features and trends in subgroups of the U.S. population. In this example, we will look at the distribution of males and females across age groups. We will continue using the `us_pop` table from the previous section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SEX</th>\n",
       "      <th>AGE</th>\n",
       "      <th>2010</th>\n",
       "      <th>2014</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3951330</td>\n",
       "      <td>3949775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>3957888</td>\n",
       "      <td>3949776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>4090862</td>\n",
       "      <td>3959664</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>4111920</td>\n",
       "      <td>4007079</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>4077551</td>\n",
       "      <td>4005716</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   SEX  AGE     2010     2014\n",
       "0    0    0  3951330  3949775\n",
       "1    0    1  3957888  3949776\n",
       "2    0    2  4090862  3959664\n",
       "3    0    3  4111920  4007079\n",
       "4    0    4  4077551  4005716"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "us_pop.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we know from having examined this dataset earlier, a [description of the table](http://www2.census.gov/programs-surveys/popest/datasets/2010-2015/national/asrh/nc-est2015-agesex-res.pdf) appears online. Here is a reminder of what the table contains. \n",
    "\n",
    "Each row represents an age group. The `SEX` column contains numeric codes: `0` stands for the total, `1` for male, and `2` for female. The `AGE` column contains ages in completed years, but the special value `999` represents the entire population regardless of age. The rest of the columns contain estimates of the US population."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understanding `AGE` = 100\n",
    "As a preliminary, let's interpret data in the final age category in the table, where `AGE` is 100. The code below extracts the rows for the combined group of men and women (`SEX` code 0) for the highest ages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SEX</th>\n",
       "      <th>AGE</th>\n",
       "      <th>2010</th>\n",
       "      <th>2014</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>97</th>\n",
       "      <td>0</td>\n",
       "      <td>97</td>\n",
       "      <td>68893</td>\n",
       "      <td>83089</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>0</td>\n",
       "      <td>98</td>\n",
       "      <td>47037</td>\n",
       "      <td>59726</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>0</td>\n",
       "      <td>99</td>\n",
       "      <td>32178</td>\n",
       "      <td>41468</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>0</td>\n",
       "      <td>100</td>\n",
       "      <td>54410</td>\n",
       "      <td>71626</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     SEX  AGE   2010   2014\n",
       "97     0   97  68893  83089\n",
       "98     0   98  47037  59726\n",
       "99     0   99  32178  41468\n",
       "100    0  100  54410  71626"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# note use of boolean '&' operator - not 'and'\n",
    "# boolean 'or' = | (aka a 'pipe' character)\n",
    "\n",
    "us_pop[(us_pop['SEX']== 0) & ((us_pop['AGE'] >= 97) & (us_pop['AGE'] <= 101))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not surprisingly, the numbers of people are smaller at higher ages – for example, there are fewer 99-year-olds than 98-year-olds. \n",
    "\n",
    "It does come as a surprise, though, that the numbers for `AGE` 100 are quite a bit larger than those for age 99. A closer examination of the documentation shows that it's because the Census Bureau used 100 as the code for everyone aged 100 or more. \n",
    "\n",
    "The row with `AGE` 100 doesn't just represent 100-year-olds – it also includes those who are older than 100. That is why the numbers in that row are larger than in the row for the 99-year-olds."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Overall Proportions of Males and Females\n",
    "We will now begin looking at gender ratios in 2014. First, let's look at all the age groups together. Remember that this means looking at the rows where the \"age\" is coded 999. The table `all_ages` contains this information. There are three rows: one for the total of both genders, one for males (`SEX` code 1), and one for females (`SEX` code 2)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SEX</th>\n",
       "      <th>AGE</th>\n",
       "      <th>2014</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>101</th>\n",
       "      <td>0</td>\n",
       "      <td>999</td>\n",
       "      <td>318907401</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>203</th>\n",
       "      <td>1</td>\n",
       "      <td>999</td>\n",
       "      <td>156955337</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>305</th>\n",
       "      <td>2</td>\n",
       "      <td>999</td>\n",
       "      <td>161952064</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     SEX  AGE       2014\n",
       "101    0  999  318907401\n",
       "203    1  999  156955337\n",
       "305    2  999  161952064"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "us_pop_2014 = us_pop.drop(columns=['2010'])\n",
    "all_ages = us_pop_2014[us_pop_2014['AGE'] == 999]\n",
    "all_ages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Row 0 of `all_ages` contains the total U.S. population in each of the two years. The United States had just under 319 million in 2014.\n",
    "\n",
    "Row 1 contains the counts for males and Row 2 for females. Compare these two rows to see that in 2014, there were more females than males in the United States. \n",
    "\n",
    "The population counts in Row 1 and Row 2 add up to the total population in Row 0. \n",
    "\n",
    "For comparability with other quantities, we will need to convert these counts to percents out of the total population. Let's access the total for 2014 and name it. Then, we'll show a population table with a proportion column. Consistent with our earlier observation that there were more females than males, about 50.8% of the population in 2014 was female and about 49.2% male in each of the two years. \n",
    "\n",
    "**N.B.** Try - in this case we are setting a value based on a 'slice' of a 'copy' which could lead to problems at some point in further calculations. As this is a practice example we can select the 'mode.chained_assignment' to value None otherwise Pandas will helpfully warn us that there may be a problem now or later. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_8705a65e_5428_11eb_b710_685b35b96a23\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >SEX</th>        <th class=\"col_heading level0 col1\" >AGE</th>        <th class=\"col_heading level0 col2\" >2014</th>        <th class=\"col_heading level0 col3\" >Proportion</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_8705a65e_5428_11eb_b710_685b35b96a23level0_row0\" class=\"row_heading level0 row0\" >101</th>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row0_col0\" class=\"data row0 col0\" >0</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row0_col1\" class=\"data row0 col1\" >999</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row0_col2\" class=\"data row0 col2\" >318907401</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row0_col3\" class=\"data row0 col3\" >100.00%</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_8705a65e_5428_11eb_b710_685b35b96a23level0_row1\" class=\"row_heading level0 row1\" >203</th>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row1_col0\" class=\"data row1 col0\" >1</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row1_col1\" class=\"data row1 col1\" >999</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row1_col2\" class=\"data row1 col2\" >156955337</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row1_col3\" class=\"data row1 col3\" >49.22%</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_8705a65e_5428_11eb_b710_685b35b96a23level0_row2\" class=\"row_heading level0 row2\" >305</th>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row2_col0\" class=\"data row2 col0\" >2</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row2_col1\" class=\"data row2 col1\" >999</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row2_col2\" class=\"data row2 col2\" >161952064</td>\n",
       "                        <td id=\"T_8705a65e_5428_11eb_b710_685b35b96a23row2_col3\" class=\"data row2 col3\" >50.78%</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7fba533fab00>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pop_2014 = all_ages['2014'].iloc[0]\n",
    "\n",
    "all_ages['Proportion'] = all_ages['2014']/pop_2014\n",
    "all_ages.style.format({'Proportion':\"{:.2%}\"})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Proportions of Boys and Girls among Infants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we look at infants, however, the opposite is true. Let's define infants to be babies who have not yet completed one year, represented in the row corresponding to `AGE` 0. Here are their numbers in the population. You can see that male infants outnumbered female infants."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SEX</th>\n",
       "      <th>AGE</th>\n",
       "      <th>2014</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3949775</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>102</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2020326</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>204</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1929449</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     SEX  AGE     2014\n",
       "0      0    0  3949775\n",
       "102    1    0  2020326\n",
       "204    2    0  1929449"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "infants = us_pop_2014[us_pop_2014['AGE'] == 0]\n",
    "infants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As before, we can convert these counts to percents out of the total numbers of infants. The resulting table shows that in 2014, just over 51% of infants in the U.S. were male. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_8ac32468_5428_11eb_8070_685b35b96a23\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >SEX</th>        <th class=\"col_heading level0 col1\" >AGE</th>        <th class=\"col_heading level0 col2\" >2014</th>        <th class=\"col_heading level0 col3\" >Proportion</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_8ac32468_5428_11eb_8070_685b35b96a23level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row0_col0\" class=\"data row0 col0\" >0</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row0_col1\" class=\"data row0 col1\" >0</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row0_col2\" class=\"data row0 col2\" >3949775</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row0_col3\" class=\"data row0 col3\" >100.00%</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_8ac32468_5428_11eb_8070_685b35b96a23level0_row1\" class=\"row_heading level0 row1\" >102</th>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row1_col0\" class=\"data row1 col0\" >1</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row1_col1\" class=\"data row1 col1\" >0</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row1_col2\" class=\"data row1 col2\" >2020326</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row1_col3\" class=\"data row1 col3\" >51.15%</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_8ac32468_5428_11eb_8070_685b35b96a23level0_row2\" class=\"row_heading level0 row2\" >204</th>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row2_col0\" class=\"data row2 col0\" >2</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row2_col1\" class=\"data row2 col1\" >0</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row2_col2\" class=\"data row2 col2\" >1929449</td>\n",
       "                        <td id=\"T_8ac32468_5428_11eb_8070_685b35b96a23row2_col3\" class=\"data row2 col3\" >48.85%</td>\n",
       "            </tr>\n",
       "    </tbody></table>"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7fba5372c438>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "infants_2014 = infants['2014'].loc[0]\n",
    "\n",
    "infants['Proportion'] = infants['2014']/infants_2014\n",
    "\n",
    "infants.style.format({'Proportion':\"{:.2%}\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In fact, it has long been observed that the proportion of boys among newborns is slightly more than 1/2. The reason for this is not thoroughly understood, and [scientists are still working on it](http://www.npr.org/sections/health-shots/2015/03/30/396384911/why-are-more-baby-boys-born-than-girls)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Female:Male Gender Ratio at Each Age"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have seen that while there are more baby boys than baby girls, there are more females than males overall. So it's clear that the split between genders must vary across age groups.\n",
    "\n",
    "To study this variation, we will separate out the data for the females and the males, and eliminate the row where all the ages are aggregated and `AGE` is coded as 999.\n",
    "\n",
    "The dfs `females` and `males` contain the data for each the two genders.  \n",
    "\n",
    "**N.B.** in the following cases we will reset the DataFrame index so that we may eventually carry out a comparison."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>SEX</th>\n",
       "      <th>AGE</th>\n",
       "      <th>2014</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>204</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1929449</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>205</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1931375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>206</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>1935991</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>207</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1957483</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>208</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>1961199</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index  SEX  AGE     2014\n",
       "0    204    2    0  1929449\n",
       "1    205    2    1  1931375\n",
       "2    206    2    2  1935991\n",
       "3    207    2    3  1957483\n",
       "4    208    2    4  1961199"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "females_all_rows = us_pop_2014[us_pop_2014['SEX'] == 2]\n",
    "\n",
    "females = females_all_rows[females_all_rows['AGE'] != 999]\n",
    "\n",
    "females = females.reset_index()\n",
    "\n",
    "females.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>SEX</th>\n",
       "      <th>AGE</th>\n",
       "      <th>2014</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>102</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2020326</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>103</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2018401</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>104</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2023673</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>105</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2049596</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>106</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>2044517</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   index  SEX  AGE     2014\n",
       "0    102    1    0  2020326\n",
       "1    103    1    1  2018401\n",
       "2    104    1    2  2023673\n",
       "3    105    1    3  2049596\n",
       "4    106    1    4  2044517"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "males_all_rows = us_pop_2014[us_pop_2014['SEX'] == 1]\n",
    "\n",
    "males = males_all_rows[males_all_rows['AGE'] != 999]\n",
    "\n",
    "males = males.reset_index()\n",
    "\n",
    "males.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plan now is to compare the number of women and the number of men at each age, for each of the two years. Array and Table methods give us straightforward ways to do this. Both of these DataFrames have one row for each age."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   AGE\n",
       "0    0\n",
       "1    1\n",
       "2    2\n",
       "3    3\n",
       "4    4"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "males[['AGE']].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   AGE\n",
       "0    0\n",
       "1    1\n",
       "2    2\n",
       "3    3\n",
       "4    4"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "females[['AGE']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For any given age, we can get the Female:Male gender ratio by dividing the number of females by the number of males. To do this in one step, we can use `column` to extract the array of female counts and the corresponding array of male counts, and then simply divide one array by the other. Elementwise division will create an array of gender ratios for all the years."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1929449\n",
       "1    1931375\n",
       "2    1935991\n",
       "3    1957483\n",
       "4    1961199\n",
       "5    1962561\n",
       "6    2024870\n",
       "7    2032494\n",
       "8    2015285\n",
       "9    2010659\n",
       "Name: 2014, dtype: int64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "females['2014'][0:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    2020326\n",
       "1    2018401\n",
       "2    2023673\n",
       "3    2049596\n",
       "4    2044517\n",
       "5    2044339\n",
       "6    2111060\n",
       "7    2122832\n",
       "8    2105618\n",
       "9    2097690\n",
       "Name: 2014, dtype: int64"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "males['2014'][0:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.955019\n",
       "1    0.956884\n",
       "2    0.956672\n",
       "3    0.955058\n",
       "4    0.959248\n",
       "5    0.959998\n",
       "6    0.959172\n",
       "7    0.957445\n",
       "8    0.957099\n",
       "9    0.958511\n",
       "Name: 2014, dtype: float64"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(females['2014']/males['2014'])[0:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "      <th>2014 F:M RATIO</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.955019</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.956884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.956672</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0.955058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0.959248</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>0.959998</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>0.959172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>0.957445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>0.957099</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>0.958511</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   AGE  2014 F:M RATIO\n",
       "0    0        0.955019\n",
       "1    1        0.956884\n",
       "2    2        0.956672\n",
       "3    3        0.955058\n",
       "4    4        0.959248\n",
       "5    5        0.959998\n",
       "6    6        0.959172\n",
       "7    7        0.957445\n",
       "8    8        0.957099\n",
       "9    9        0.958511"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratios = pd.DataFrame({\n",
    "    'AGE': females['AGE'],\n",
    "    '2014 F:M RATIO': (females['2014']/males['2014'])})\n",
    "\n",
    "ratios.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see from the display that the ratios are all around 0.96 for children aged nine or younger. When the Female:Male ratio is less than 1, there are fewer females than males. Thus what we are seeing is that there were fewer girls than boys in each of the age groups 0, 1, 2, and so on through 9. Moreover, in each of these age groups, there were about 96 girls for every 100 boys."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So how can the overall proportion of females in the population be higher than the males? \n",
    "\n",
    "Something extraordinary happens when we examine the other end of the age range. Here are the Female:Male ratios for people aged more than 75."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGE</th>\n",
       "      <th>2014 F:M RATIO</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>76</th>\n",
       "      <td>76</td>\n",
       "      <td>1.234867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77</th>\n",
       "      <td>77</td>\n",
       "      <td>1.257965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>78</th>\n",
       "      <td>78</td>\n",
       "      <td>1.282442</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>79</th>\n",
       "      <td>79</td>\n",
       "      <td>1.316273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80</th>\n",
       "      <td>80</td>\n",
       "      <td>1.341383</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>81</th>\n",
       "      <td>81</td>\n",
       "      <td>1.379669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>82</td>\n",
       "      <td>1.419317</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83</th>\n",
       "      <td>83</td>\n",
       "      <td>1.465520</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>84</th>\n",
       "      <td>84</td>\n",
       "      <td>1.520484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85</th>\n",
       "      <td>85</td>\n",
       "      <td>1.575604</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    AGE  2014 F:M RATIO\n",
       "76   76        1.234867\n",
       "77   77        1.257965\n",
       "78   78        1.282442\n",
       "79   79        1.316273\n",
       "80   80        1.341383\n",
       "81   81        1.379669\n",
       "82   82        1.419317\n",
       "83   83        1.465520\n",
       "84   84        1.520484\n",
       "85   85        1.575604"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(ratios[ratios['AGE'] > 75]).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "10Not only are all of these ratios greater than 1, signifying more women than men in all of these age groups, many of them are considerably greater than 1. \n",
    "\n",
    "- At ages 89 and 90 the ratios are close to 2, meaning that there were about twice as many women as men at those ages in 2014.\n",
    "- At ages 98 and 99, there were about 3.5 to 4 times as many women as men. \n",
    "\n",
    "If you are wondering how many people there were at these advanced ages, you can use Python to find out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>SEX</th>\n",
       "      <th>AGE</th>\n",
       "      <th>2014</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>200</td>\n",
       "      <td>1</td>\n",
       "      <td>98</td>\n",
       "      <td>13518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>201</td>\n",
       "      <td>1</td>\n",
       "      <td>99</td>\n",
       "      <td>8951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>202</td>\n",
       "      <td>1</td>\n",
       "      <td>100</td>\n",
       "      <td>13618</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     index  SEX  AGE   2014\n",
       "98     200    1   98  13518\n",
       "99     201    1   99   8951\n",
       "100    202    1  100  13618"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "males[(males['AGE'] >= 98) & (males['AGE'] <= 100)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>index</th>\n",
       "      <th>SEX</th>\n",
       "      <th>AGE</th>\n",
       "      <th>2014</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>98</th>\n",
       "      <td>302</td>\n",
       "      <td>2</td>\n",
       "      <td>98</td>\n",
       "      <td>46208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>99</th>\n",
       "      <td>303</td>\n",
       "      <td>2</td>\n",
       "      <td>99</td>\n",
       "      <td>32517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>304</td>\n",
       "      <td>2</td>\n",
       "      <td>100</td>\n",
       "      <td>58008</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     index  SEX  AGE   2014\n",
       "98     302    2   98  46208\n",
       "99     303    2   99  32517\n",
       "100    304    2  100  58008"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "females[(females['AGE'] >= 98) & (females['AGE'] <= 100)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The graph below shows the gender ratios plotted against age. The blue curve shows the 2014 ratio by age.\n",
    "\n",
    "The ratios are almost 1 (signifying close to equal numbers of males and females) for ages 0 through 60, but they start shooting up dramatically (more females than males) starting at about age 65.\n",
    "\n",
    "That females outnumber males in the U.S. is partly due to the marked gender imbalance in favor of women among senior citizens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 19104 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ratios.plot('AGE')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
