{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [],
   "source": [
    "path_data = '../../data/'\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualizing Numerical Distributions\n",
    "\n",
    "Many of the variables that data scientists study are *quantitative* or *numerical*. Their values are numbers on which you can perform arithmetic. Examples that we have seen include the number of periods in chapters of a book, the amount of money made by movies, and the age of people in the United States.\n",
    "\n",
    "The values of a categorical variable can be given numerical codes, but that doesn't make the variable quantitative. In the example in which we studied Census data broken down by age group, the categorial variable `SEX` had the numerical codes `1` for 'Male,' `2` for 'Female,' and `0` for the aggregate of both groups `1` and `2`. While 0, 1, and 2 are numbers, in this context it doesn't make sense to subtract 1 from 2, or take the average of 0, 1, and 2, or perform other arithmetic on the three values. `SEX` is a categorical variable even though the values have been given a numerical code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For our main example, we will return to a dataset that we studied when we were visualizing categorical data. It is the table `top`, which consists of data from U.S.A.'s top grossing movies of all time. For convenience, here is the description of the table again.\n",
    "\n",
    "The first column contains the title of the movie. The second column contains the name of the studio that produced the movie. The third contains the domestic box office gross in dollars, and the fourth contains the  gross amount that would have been earned from ticket sales at 2016 prices. The fifth contains the release year of the movie. \n",
    "\n",
    "There are 200 movies on the list. Here are the top ten according to the unadjusted gross receipts in the column `Gross`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>Studio</th>\n",
       "      <th>Gross</th>\n",
       "      <th>Gross (Adjusted)</th>\n",
       "      <th>Year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Star Wars: The Force Awakens</td>\n",
       "      <td>Buena Vista (Disney)</td>\n",
       "      <td>906723418</td>\n",
       "      <td>906723400</td>\n",
       "      <td>2015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>Fox</td>\n",
       "      <td>760507625</td>\n",
       "      <td>846120800</td>\n",
       "      <td>2009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Titanic</td>\n",
       "      <td>Paramount</td>\n",
       "      <td>658672302</td>\n",
       "      <td>1178627900</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Jurassic World</td>\n",
       "      <td>Universal</td>\n",
       "      <td>652270625</td>\n",
       "      <td>687728000</td>\n",
       "      <td>2015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Marvel's The Avengers</td>\n",
       "      <td>Buena Vista (Disney)</td>\n",
       "      <td>623357910</td>\n",
       "      <td>668866600</td>\n",
       "      <td>2012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>195</th>\n",
       "      <td>The Caine Mutiny</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>21750000</td>\n",
       "      <td>386173500</td>\n",
       "      <td>1954</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>The Bells of St. Mary's</td>\n",
       "      <td>RKO</td>\n",
       "      <td>21333333</td>\n",
       "      <td>545882400</td>\n",
       "      <td>1945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>197</th>\n",
       "      <td>Duel in the Sun</td>\n",
       "      <td>Selz.</td>\n",
       "      <td>20408163</td>\n",
       "      <td>443877500</td>\n",
       "      <td>1946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>198</th>\n",
       "      <td>Sergeant York</td>\n",
       "      <td>Warner Bros.</td>\n",
       "      <td>16361885</td>\n",
       "      <td>418671800</td>\n",
       "      <td>1941</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>The Four Horsemen of the Apocalypse</td>\n",
       "      <td>MPC</td>\n",
       "      <td>9183673</td>\n",
       "      <td>399489800</td>\n",
       "      <td>1921</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>200 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   Title                Studio      Gross  \\\n",
       "0           Star Wars: The Force Awakens  Buena Vista (Disney)  906723418   \n",
       "1                                 Avatar                   Fox  760507625   \n",
       "2                                Titanic             Paramount  658672302   \n",
       "3                         Jurassic World             Universal  652270625   \n",
       "4                  Marvel's The Avengers  Buena Vista (Disney)  623357910   \n",
       "..                                   ...                   ...        ...   \n",
       "195                     The Caine Mutiny              Columbia   21750000   \n",
       "196              The Bells of St. Mary's                   RKO   21333333   \n",
       "197                      Duel in the Sun                 Selz.   20408163   \n",
       "198                        Sergeant York          Warner Bros.   16361885   \n",
       "199  The Four Horsemen of the Apocalypse                   MPC    9183673   \n",
       "\n",
       "     Gross (Adjusted)  Year  \n",
       "0           906723400  2015  \n",
       "1           846120800  2009  \n",
       "2          1178627900  1997  \n",
       "3           687728000  2015  \n",
       "4           668866600  2012  \n",
       "..                ...   ...  \n",
       "195         386173500  1954  \n",
       "196         545882400  1945  \n",
       "197         443877500  1946  \n",
       "198         418671800  1941  \n",
       "199         399489800  1921  \n",
       "\n",
       "[200 rows x 5 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top = pd.read_csv(path_data + 'top_movies.csv')\n",
    "\n",
    "top"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>Studio</th>\n",
       "      <th>Gross</th>\n",
       "      <th>Gross (Adjusted)</th>\n",
       "      <th>Year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Star Wars: The Force Awakens</td>\n",
       "      <td>Buena Vista (Disney)</td>\n",
       "      <td>906,723,418</td>\n",
       "      <td>906,723,400</td>\n",
       "      <td>2015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>Fox</td>\n",
       "      <td>760,507,625</td>\n",
       "      <td>846,120,800</td>\n",
       "      <td>2009</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Titanic</td>\n",
       "      <td>Paramount</td>\n",
       "      <td>658,672,302</td>\n",
       "      <td>1,178,627,900</td>\n",
       "      <td>1997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Jurassic World</td>\n",
       "      <td>Universal</td>\n",
       "      <td>652,270,625</td>\n",
       "      <td>687,728,000</td>\n",
       "      <td>2015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Marvel's The Avengers</td>\n",
       "      <td>Buena Vista (Disney)</td>\n",
       "      <td>623,357,910</td>\n",
       "      <td>668,866,600</td>\n",
       "      <td>2012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>195</th>\n",
       "      <td>The Caine Mutiny</td>\n",
       "      <td>Columbia</td>\n",
       "      <td>21,750,000</td>\n",
       "      <td>386,173,500</td>\n",
       "      <td>1954</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>The Bells of St. Mary's</td>\n",
       "      <td>RKO</td>\n",
       "      <td>21,333,333</td>\n",
       "      <td>545,882,400</td>\n",
       "      <td>1945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>197</th>\n",
       "      <td>Duel in the Sun</td>\n",
       "      <td>Selz.</td>\n",
       "      <td>20,408,163</td>\n",
       "      <td>443,877,500</td>\n",
       "      <td>1946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>198</th>\n",
       "      <td>Sergeant York</td>\n",
       "      <td>Warner Bros.</td>\n",
       "      <td>16,361,885</td>\n",
       "      <td>418,671,800</td>\n",
       "      <td>1941</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>The Four Horsemen of the Apocalypse</td>\n",
       "      <td>MPC</td>\n",
       "      <td>9,183,673</td>\n",
       "      <td>399,489,800</td>\n",
       "      <td>1921</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>200 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   Title                Studio        Gross  \\\n",
       "0           Star Wars: The Force Awakens  Buena Vista (Disney)  906,723,418   \n",
       "1                                 Avatar                   Fox  760,507,625   \n",
       "2                                Titanic             Paramount  658,672,302   \n",
       "3                         Jurassic World             Universal  652,270,625   \n",
       "4                  Marvel's The Avengers  Buena Vista (Disney)  623,357,910   \n",
       "..                                   ...                   ...          ...   \n",
       "195                     The Caine Mutiny              Columbia   21,750,000   \n",
       "196              The Bells of St. Mary's                   RKO   21,333,333   \n",
       "197                      Duel in the Sun                 Selz.   20,408,163   \n",
       "198                        Sergeant York          Warner Bros.   16,361,885   \n",
       "199  The Four Horsemen of the Apocalypse                   MPC    9,183,673   \n",
       "\n",
       "    Gross (Adjusted)  Year  \n",
       "0        906,723,400  2015  \n",
       "1        846,120,800  2009  \n",
       "2      1,178,627,900  1997  \n",
       "3        687,728,000  2015  \n",
       "4        668,866,600  2012  \n",
       "..               ...   ...  \n",
       "195      386,173,500  1954  \n",
       "196      545,882,400  1945  \n",
       "197      443,877,500  1946  \n",
       "198      418,671,800  1941  \n",
       "199      399,489,800  1921  \n",
       "\n",
       "[200 rows x 5 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Make the numbers in the Gross and Gross (Adjusted) columns look nicer:\n",
    "# Separate '000s with comma\n",
    "# When using an original data set it is often good practice to work on a copy of the original\n",
    "\n",
    "top1 = top.copy()\n",
    "\n",
    "top1['Gross'] = top1['Gross'].apply('{:,}'.format)\n",
    "top1['Gross (Adjusted)'] = top1['Gross (Adjusted)'].apply('{:,}'.format)\n",
    "\n",
    "top1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing the Distribution of the Adjusted Receipts\n",
    "\n",
    "In this section we will draw graphs of the distribution of the numerical variable in the column `Gross (Adjusted)`. For simplicity, let's create a smaller table that has the information that we need. And since three-digit numbers are easier to work with than nine-digit numbers, let's measure the `Adjusted Gross` receipts in millions of dollars. Note how `round` is used to retain only two decimal places."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Title</th>\n",
       "      <th>Adjusted Gross</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Star Wars: The Force Awakens</td>\n",
       "      <td>906.72</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Avatar</td>\n",
       "      <td>846.12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Titanic</td>\n",
       "      <td>1178.63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Jurassic World</td>\n",
       "      <td>687.73</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Marvel's The Avengers</td>\n",
       "      <td>668.87</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>195</th>\n",
       "      <td>The Caine Mutiny</td>\n",
       "      <td>386.17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>196</th>\n",
       "      <td>The Bells of St. Mary's</td>\n",
       "      <td>545.88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>197</th>\n",
       "      <td>Duel in the Sun</td>\n",
       "      <td>443.88</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>198</th>\n",
       "      <td>Sergeant York</td>\n",
       "      <td>418.67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>The Four Horsemen of the Apocalypse</td>\n",
       "      <td>399.49</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>200 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                   Title  Adjusted Gross\n",
       "0           Star Wars: The Force Awakens          906.72\n",
       "1                                 Avatar          846.12\n",
       "2                                Titanic         1178.63\n",
       "3                         Jurassic World          687.73\n",
       "4                  Marvel's The Avengers          668.87\n",
       "..                                   ...             ...\n",
       "195                     The Caine Mutiny          386.17\n",
       "196              The Bells of St. Mary's          545.88\n",
       "197                      Duel in the Sun          443.88\n",
       "198                        Sergeant York          418.67\n",
       "199  The Four Horsemen of the Apocalypse          399.49\n",
       "\n",
       "[200 rows x 2 columns]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "millions = pd.DataFrame({'Title': top['Title'], 'Adjusted Gross': np.round((top['Gross (Adjusted)']/1e6), 2)})\n",
    "\n",
    "millions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A Histogram\n",
    "A *histogram* of a numerical dataset looks very much like a bar chart, though it has some important differences that we will examine in this section. First, let's just draw a histogram of the adjusted receipts.\n",
    "\n",
    "The `hist` method generates a histogram of the values in a column. The histogram shows the distribution of the adjusted gross amounts, in millions of 2016 dollars. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 21340 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "unit = 'Million Dollars'\n",
    "\n",
    "fig, ax1 = plt.subplots()\n",
    "\n",
    "ax1.hist(millions['Adjusted Gross'], 10, density=True, ec='white')\n",
    "\n",
    "y_vals = ax1.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Adjusted Gross (' + (unit if unit else 'unit') + ')'\n",
    "\n",
    "ax1.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Horizontal Axis\n",
    "\n",
    "The amounts have been grouped into contiguous intervals called *bins*. Although in this dataset no movie grossed an amount that is exactly on the edge between two bins, `hist` does have to account for situations where there might have been values at the edges. So `hist` has an *endpoint convention*: bins include the data at their left endpoint, but not the data at their right endpoint. \n",
    "\n",
    "We will use the notation **[*a*, *b*)** for the bin that starts at *a* and ends at *b* but doesn't include *b*.\n",
    "\n",
    "Sometimes, adjustments have to be made in the first or last bin, to ensure that the smallest and largest values of the variable are included. You saw an example of such an adjustment in the Census data studied earlier, where an age of \"100\" years actually meant \"100 years old or older.\"\n",
    "\n",
    "We can see that there are 10 bins (some bars are so low that they are hard to see), and that they all have the same width. We can also see that none of the movies grossed fewer than 300 million dollars; that is because we are considering only the top grossing movies of all time. \n",
    "\n",
    "It is a little harder to see exactly where the ends of the bins are situated. For example, it is not easy to pinpoint exactly where the value 500 lies on the horizontal axis. So it is hard to judge exactly where one bar ends and the next begins.\n",
    "\n",
    "The optional argument `bins` can be used with `hist` to specify the endpoints of the bins. It must consist of a sequence of numbers that starts with the left end of the first bin and ends with the right end of the last bin. We will start by setting the numbers in `bins` to be 300, 400, 500, and so on, ending with 2000.  \n",
    "\n",
    "**Note** defining bin values the y-axis scale has changed to 0.00 - 0.04"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 21240 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "unit = 'Million Dollars'\n",
    "\n",
    "fig, ax1 = plt.subplots()\n",
    "\n",
    "ax1.hist(millions['Adjusted Gross'], bins=np.arange(300,2001,100), density=True, ec='white')\n",
    "\n",
    "y_vals = ax1.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Adjusted Gross (' + (unit if unit else 'unit') + ')'\n",
    "\n",
    "ax1.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The horizontal axis of this figure is easier to read. The labels 200, 400, 600, and so on are centered at the corresponding values. The tallest bar is for movies that grossed between 300 million and 400 million dollars. \n",
    "\n",
    "A very small number of movies grossed 800 million dollars or more. This results in the figure being \"skewed to the right,\" or, less formally, having \"a long right hand tail.\" Distributions of variables like income or rent in large populations also often have this kind of shape."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Counts in the Bins\n",
    "\n",
    "The counts of values in the bins can be computed from a table using the `.value_counts()` method, which takes a column label or index and an optional sequence or number of bins. The result is a tabular form of a histogram. The first column lists the bin ranges (but see the note about the final value, below). The second column contains the counts of all values in the `Adjusted Gross` column that are in the corresponding bin. That is, it counts all the `Adjusted Gross` values that are greater than or equal to the value in `bin`, but less than the next value in `bin`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Adjusted Gross</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>(299.999, 400.0]</th>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(400.0, 500.0]</th>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(500.0, 600.0]</th>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(600.0, 700.0]</th>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(700.0, 800.0]</th>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(800.0, 900.0]</th>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(900.0, 1000.0]</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1100.0, 1200.0]</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1200.0, 1300.0]</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1000.0, 1100.0]</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1500.0, 1600.0]</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1700.0, 1800.0]</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1800.0, 1900.0]</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1300.0, 1400.0]</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1400.0, 1500.0]</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1600.0, 1700.0]</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1900.0, 2000.0]</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Adjusted Gross\n",
       "(299.999, 400.0]              81\n",
       "(400.0, 500.0]                52\n",
       "(500.0, 600.0]                28\n",
       "(600.0, 700.0]                16\n",
       "(700.0, 800.0]                 7\n",
       "(800.0, 900.0]                 5\n",
       "(900.0, 1000.0]                3\n",
       "(1100.0, 1200.0]               3\n",
       "(1200.0, 1300.0]               2\n",
       "(1000.0, 1100.0]               1\n",
       "(1500.0, 1600.0]               1\n",
       "(1700.0, 1800.0]               1\n",
       "(1800.0, 1900.0]               0\n",
       "(1300.0, 1400.0]               0\n",
       "(1400.0, 1500.0]               0\n",
       "(1600.0, 1700.0]               0\n",
       "(1900.0, 2000.0]               0"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bin_counts = millions['Adjusted Gross']\n",
    "\n",
    "bin_counts = pd.DataFrame(bin_counts.value_counts(bins=(np.arange(300,2001,100))))\n",
    "\n",
    "bin_counts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Adjusted Gross</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>(299.999, 400.0]</th>\n",
       "      <td>0.405</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(400.0, 500.0]</th>\n",
       "      <td>0.260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(500.0, 600.0]</th>\n",
       "      <td>0.140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(600.0, 700.0]</th>\n",
       "      <td>0.080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(700.0, 800.0]</th>\n",
       "      <td>0.035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(800.0, 900.0]</th>\n",
       "      <td>0.025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(900.0, 1000.0]</th>\n",
       "      <td>0.015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1100.0, 1200.0]</th>\n",
       "      <td>0.015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1200.0, 1300.0]</th>\n",
       "      <td>0.010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1000.0, 1100.0]</th>\n",
       "      <td>0.005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1500.0, 1600.0]</th>\n",
       "      <td>0.005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1700.0, 1800.0]</th>\n",
       "      <td>0.005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1800.0, 1900.0]</th>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1300.0, 1400.0]</th>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1400.0, 1500.0]</th>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1600.0, 1700.0]</th>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1900.0, 2000.0]</th>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Adjusted Gross\n",
       "(299.999, 400.0]           0.405\n",
       "(400.0, 500.0]             0.260\n",
       "(500.0, 600.0]             0.140\n",
       "(600.0, 700.0]             0.080\n",
       "(700.0, 800.0]             0.035\n",
       "(800.0, 900.0]             0.025\n",
       "(900.0, 1000.0]            0.015\n",
       "(1100.0, 1200.0]           0.015\n",
       "(1200.0, 1300.0]           0.010\n",
       "(1000.0, 1100.0]           0.005\n",
       "(1500.0, 1600.0]           0.005\n",
       "(1700.0, 1800.0]           0.005\n",
       "(1800.0, 1900.0]           0.000\n",
       "(1300.0, 1400.0]           0.000\n",
       "(1400.0, 1500.0]           0.000\n",
       "(1600.0, 1700.0]           0.000\n",
       "(1900.0, 2000.0]           0.000"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bin_counts_norm = millions['Adjusted Gross']\n",
    "\n",
    "bin_counts_norm = pd.DataFrame(bin_counts_norm.value_counts(normalize=True,bins=(np.arange(300,2001,100))))\n",
    "\n",
    "bin_counts_norm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Splitting data\n",
    "in the next step we take the index of 'bin_counts_norm', reset the index so that we create an 'intervals' column which can then be split into 'left' and 'right' elements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>intervals</th>\n",
       "      <th>left</th>\n",
       "      <th>right</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>(299.999, 400.0]</td>\n",
       "      <td>299.999</td>\n",
       "      <td>400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>(400.0, 500.0]</td>\n",
       "      <td>400.000</td>\n",
       "      <td>500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>(500.0, 600.0]</td>\n",
       "      <td>500.000</td>\n",
       "      <td>600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>(600.0, 700.0]</td>\n",
       "      <td>600.000</td>\n",
       "      <td>700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>(700.0, 800.0]</td>\n",
       "      <td>700.000</td>\n",
       "      <td>800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>(800.0, 900.0]</td>\n",
       "      <td>800.000</td>\n",
       "      <td>900.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>(900.0, 1000.0]</td>\n",
       "      <td>900.000</td>\n",
       "      <td>1000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>(1100.0, 1200.0]</td>\n",
       "      <td>1100.000</td>\n",
       "      <td>1200.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>(1200.0, 1300.0]</td>\n",
       "      <td>1200.000</td>\n",
       "      <td>1300.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>(1000.0, 1100.0]</td>\n",
       "      <td>1000.000</td>\n",
       "      <td>1100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>(1500.0, 1600.0]</td>\n",
       "      <td>1500.000</td>\n",
       "      <td>1600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>(1700.0, 1800.0]</td>\n",
       "      <td>1700.000</td>\n",
       "      <td>1800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>(1800.0, 1900.0]</td>\n",
       "      <td>1800.000</td>\n",
       "      <td>1900.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>(1300.0, 1400.0]</td>\n",
       "      <td>1300.000</td>\n",
       "      <td>1400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>(1400.0, 1500.0]</td>\n",
       "      <td>1400.000</td>\n",
       "      <td>1500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>(1600.0, 1700.0]</td>\n",
       "      <td>1600.000</td>\n",
       "      <td>1700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>(1900.0, 2000.0]</td>\n",
       "      <td>1900.000</td>\n",
       "      <td>2000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           intervals      left   right\n",
       "0   (299.999, 400.0]   299.999   400.0\n",
       "1     (400.0, 500.0]   400.000   500.0\n",
       "2     (500.0, 600.0]   500.000   600.0\n",
       "3     (600.0, 700.0]   600.000   700.0\n",
       "4     (700.0, 800.0]   700.000   800.0\n",
       "5     (800.0, 900.0]   800.000   900.0\n",
       "6    (900.0, 1000.0]   900.000  1000.0\n",
       "7   (1100.0, 1200.0]  1100.000  1200.0\n",
       "8   (1200.0, 1300.0]  1200.000  1300.0\n",
       "9   (1000.0, 1100.0]  1000.000  1100.0\n",
       "10  (1500.0, 1600.0]  1500.000  1600.0\n",
       "11  (1700.0, 1800.0]  1700.000  1800.0\n",
       "12  (1800.0, 1900.0]  1800.000  1900.0\n",
       "13  (1300.0, 1400.0]  1300.000  1400.0\n",
       "14  (1400.0, 1500.0]  1400.000  1500.0\n",
       "15  (1600.0, 1700.0]  1600.000  1700.0\n",
       "16  (1900.0, 2000.0]  1900.000  2000.0"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "interval_split = pd.DataFrame({'intervals': bin_counts_norm.index})\n",
    "\n",
    "interval_split['left'] = interval_split['intervals'].array.left\n",
    "\n",
    "interval_split['right'] = interval_split['intervals'].array.right\n",
    "\n",
    "interval_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[300,\n",
       " 400,\n",
       " 500,\n",
       " 600,\n",
       " 700,\n",
       " 800,\n",
       " 900,\n",
       " 1100,\n",
       " 1200,\n",
       " 1000,\n",
       " 1500,\n",
       " 1700,\n",
       " 1800,\n",
       " 1300,\n",
       " 1400,\n",
       " 1600,\n",
       " 1900]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lower_limit = np.round(interval_split['left'],0)\n",
    "\n",
    "lower_limit = lower_limit.astype(int)\n",
    "\n",
    "lower_limit = list(lower_limit)\n",
    "\n",
    "lower_limit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the `bin` value 2000 in the last row. That's not the left end-point of any bar – it's the right end point of the last bar. By the endpoint convention, the data there are not included. So the corresponding `count` is recorded as 0, and would have been recorded as 0 even if there had been movies that made more than \\$2,000$ million dollars. When either `bin` or `hist` is called with a `bins` argument, the graph only considers values that are in the specified bins.\n",
    "\n",
    "Once values have been binned, the resulting counts can be used to generate a **bar chart** using df `lower_limit` *typed* as a list. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 21652 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# using matplotlib\n",
    "\n",
    "width = lower_limit[1] - lower_limit[0]\n",
    "\n",
    "plt.bar(lower_limit, bin_counts_norm['Adjusted Gross'], align='center', width=width, ec='white')\n",
    "\n",
    "plt.xlabel('Adjusted Gross (Million Dollars)')\n",
    "\n",
    "plt.ylabel('Percent per Million Dollars')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Vertical Axis: Density Scale\n",
    "\n",
    "The horizontal axis of a histogram is straightforward to read, once we have taken care of details like the ends of the bins. The features of the vertical axis require a little more attention. We will go over them one by one.\n",
    "\n",
    "Let's start by examining how to calculate the numbers on the vertical axis. If the calculation seems a little strange, have patience – the rest of the section will explain the reasoning.\n",
    "\n",
    "**Calculation.** The height of each bar is the **percent of elements that fall into the corresponding bin, relative to the width of the bin**. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Count</th>\n",
       "      <th>Percent</th>\n",
       "      <th>Height</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>(299.999, 400.0]</th>\n",
       "      <td>81</td>\n",
       "      <td>40.5</td>\n",
       "      <td>0.405</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(400.0, 500.0]</th>\n",
       "      <td>52</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0.260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(500.0, 600.0]</th>\n",
       "      <td>28</td>\n",
       "      <td>14.0</td>\n",
       "      <td>0.140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(600.0, 700.0]</th>\n",
       "      <td>16</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.080</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(700.0, 800.0]</th>\n",
       "      <td>7</td>\n",
       "      <td>3.5</td>\n",
       "      <td>0.035</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(800.0, 900.0]</th>\n",
       "      <td>5</td>\n",
       "      <td>2.5</td>\n",
       "      <td>0.025</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(900.0, 1000.0]</th>\n",
       "      <td>3</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1100.0, 1200.0]</th>\n",
       "      <td>3</td>\n",
       "      <td>1.5</td>\n",
       "      <td>0.015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1200.0, 1300.0]</th>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1000.0, 1100.0]</th>\n",
       "      <td>1</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1500.0, 1600.0]</th>\n",
       "      <td>1</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1700.0, 1800.0]</th>\n",
       "      <td>1</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1800.0, 1900.0]</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1300.0, 1400.0]</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1400.0, 1500.0]</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1600.0, 1700.0]</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(1900.0, 2000.0]</th>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Count  Percent  Height\n",
       "(299.999, 400.0]     81     40.5   0.405\n",
       "(400.0, 500.0]       52     26.0   0.260\n",
       "(500.0, 600.0]       28     14.0   0.140\n",
       "(600.0, 700.0]       16      8.0   0.080\n",
       "(700.0, 800.0]        7      3.5   0.035\n",
       "(800.0, 900.0]        5      2.5   0.025\n",
       "(900.0, 1000.0]       3      1.5   0.015\n",
       "(1100.0, 1200.0]      3      1.5   0.015\n",
       "(1200.0, 1300.0]      2      1.0   0.010\n",
       "(1000.0, 1100.0]      1      0.5   0.005\n",
       "(1500.0, 1600.0]      1      0.5   0.005\n",
       "(1700.0, 1800.0]      1      0.5   0.005\n",
       "(1800.0, 1900.0]      0      0.0   0.000\n",
       "(1300.0, 1400.0]      0      0.0   0.000\n",
       "(1400.0, 1500.0]      0      0.0   0.000\n",
       "(1600.0, 1700.0]      0      0.0   0.000\n",
       "(1900.0, 2000.0]      0      0.0   0.000"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "counts = bin_counts\n",
    "\n",
    "counts = counts.rename(columns={'Adjusted Gross': 'Count'})\n",
    "\n",
    "percents = counts\n",
    "\n",
    "percents['Percent'] = (counts['Count']/200)*100\n",
    "\n",
    "percents\n",
    "\n",
    "heights = percents\n",
    "\n",
    "heights['Height'] = percents['Percent']/100\n",
    "\n",
    "heights"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Go over the numbers on the vertical axis of the histogram above to check that the column `Heights` looks correct."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The calculations will become clear if we just examine the first row of the table. \n",
    "\n",
    "Remember that there are 200 movies in the dataset. The [300, 400) bin contains 81 movies. That's 40.5% of all the movies:\n",
    "\n",
    "$$\n",
    "\\mbox{Percent} = \\frac{81}{200} \\cdot 100 = 40.5\n",
    "$$\n",
    "\n",
    "The width of the [300, 400) bin is $ 400 - 300 = 100$. So\n",
    "\n",
    "$$\n",
    "\\mbox{Height} = \\frac{40.5}{100} = 0.405\n",
    "$$\n",
    "\n",
    "The code for calculating the heights used the facts that there are 200 movies in all and that the width of each bin is 100."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Units.** The height of the bar is 40.5% divided by 100 million dollars, and so the height is 0.405% per million dollars. \n",
    "\n",
    "This method of drawing histograms creates a vertical axis that is said to be ***on the density scale***. The height of bar is **not** the percent of entries in the bin; it is the percent of entries in the bin relative to the amount of space in the bin. That is why the height measures crowdedness or *density*.\n",
    "\n",
    "Let's see why this matters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Unequal Bins\n",
    "An advantage of the histogram over a bar chart is that a histogram can contain bins of unequal width. Below, the values in the `Millions` column are binned into three uneven categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 20388 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "uneven = np.array([300, 400, 600, 1500])\n",
    "\n",
    "fig, ax1 = plt.subplots()\n",
    "\n",
    "ax1.hist(millions['Adjusted Gross'], bins=uneven, density=True, ec='white')\n",
    "\n",
    "y_vals = ax1.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Adjusted Gross (' + (unit if unit else 'unit') + ')'\n",
    "\n",
    "ax1.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are the counts in the three bins."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Adjusted Gross</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>(299.999, 400.0]</th>\n",
       "      <td>81</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(400.0, 600.0]</th>\n",
       "      <td>80</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(600.0, 1500.0]</th>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Adjusted Gross\n",
       "(299.999, 400.0]              81\n",
       "(400.0, 600.0]                80\n",
       "(600.0, 1500.0]               37"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bin_counts_uneven = millions['Adjusted Gross']\n",
    "\n",
    "bin_counts_uneven = pd.DataFrame(bin_counts_uneven.value_counts(bins=uneven))\n",
    "\n",
    "bin_counts_uneven"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Although the ranges [300, 400) and [400, 600) have nearly identical counts, the bar over the former is twice as tall as the latter because it is only half as wide. The density of values in the [300, 400) is twice as much as the density in [400, 600). \n",
    "\n",
    "Histograms help us visualize where on the number line the data are most concentrated, especially when the bins are uneven."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Problem with Simply Plotting Counts\n",
    "It is possible to display counts directly in a chart, using the `normed=False` option of the `hist` method. The resulting chart has the same shape as a histogram when the bins all have equal widths, though the numbers on the vertical axis are different."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 18484 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "millions.hist('Adjusted Gross', density=False, bins=np.arange(300,2001,100), ec='white')\n",
    "\n",
    "plt.xlabel('Adjusted Gross')\n",
    "\n",
    "plt.ylabel('Count')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the count scale is perhaps more natural to interpret than the density scale, the chart becomes highly misleading when bins have different widths. Below, it appears (due to the count scale) that high-grossing movies are quite common, when in fact we have seen that they are relatively rare."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 17628 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "millions.hist('Adjusted Gross', density=False, bins=uneven, ec='white')\n",
    "\n",
    "plt.xlabel('Adjusted Gross')\n",
    "\n",
    "plt.ylabel('Count')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Even though the method used is called `hist`, **the figure above is NOT A HISTOGRAM.** It misleadingly exaggerates the proportion of movies grossing at least 600 million dollars. The height of each bar is simply plotted at the number of movies in the bin, *without accounting for the difference in the widths of the bins*. \n",
    "\n",
    "The picture becomes even more absurd if the last two bins are combined."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 19212 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "very_uneven = np.array([300, 400, 1500])\n",
    "\n",
    "millions.hist('Adjusted Gross', density=False, bins=very_uneven, ec='white')\n",
    "\n",
    "plt.xlabel('Adjusted Gross')\n",
    "\n",
    "plt.ylabel('Count')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this count-based figure, the shape of the distribution of movies is lost entirely."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Histogram: General Principles and Calculation\n",
    "\n",
    "The figure above shows that what the eye perceives as \"big\" is area, not just height. This observation becomes particularly important when the bins have different widths.\n",
    "\n",
    "That is why a histogram has two defining properties:\n",
    "\n",
    "1. The bins are drawn to scale and are contiguous (though some might be empty), because the values on the horizontal axis are numerical.\n",
    "2. The **area** of each bar is proportional to the number of entries in the bin. \n",
    "\n",
    "Property 2 is the key to drawing a histogram, and is usually achieved as follows:\n",
    "\n",
    "$$\n",
    "\\mbox{area of bar} ~=~ \\mbox{percent of entries in bin}\n",
    "$$\n",
    "\n",
    "The calculation of the heights just uses the fact that the bar is a rectangle:\n",
    "\n",
    "$$\n",
    "\\mbox{area of bar} = \\mbox{height of bar} \\times \\mbox{width of bin}\n",
    "$$\n",
    "\n",
    "and so\n",
    "\n",
    "$$\n",
    "\\mbox{height of bar} ~=~ \n",
    "\\frac{\\mbox{area of bar}}{\\mbox{width of bin}} ~=~\n",
    "\\frac{\\mbox{percent of entries in bin}}{\\mbox{width of bin}}\n",
    "$$\n",
    "\n",
    "The units of height are \"percent per unit on the horizontal axis.\"\n",
    "\n",
    "When drawn using this method, the histogram is said to be drawn on the density scale. On this scale:\n",
    "- The area of each bar is equal to the percent of data values that are in the corresponding bin.\n",
    "- The total area of all the bars in the histogram is 100%. Speaking in terms of proportions, we say that the areas of all the bars in a histogram \"sum to 1\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Flat Tops and the Level of Detail\n",
    "\n",
    "Even though the density scale correctly represents percents using area, some detail is lost by grouping values into bins.\n",
    "\n",
    "Take another look at the [300, 400) bin in the figure below. The flat top of the bar, at the level 0.405% per million dollars, hides the fact that the movies are somewhat unevenly distributed across that bin. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 20388 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax1 = plt.subplots()\n",
    "\n",
    "ax1.hist(millions['Adjusted Gross'], bins=uneven, density=True, ec='white')\n",
    "\n",
    "y_vals = ax1.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Adjusted Gross (' + (unit if unit else 'unit') + ')'\n",
    "\n",
    "ax1.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To see this, let us split the [300, 400) bin into 10 narrower bins, each of width 10 million dollars."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 22732 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "some_tiny_bins = np.array([300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 600, 1500])\n",
    "\n",
    "fig, ax1 = plt.subplots()\n",
    "\n",
    "ax1.hist(millions['Adjusted Gross'], bins=some_tiny_bins, density=True, ec='white')\n",
    "\n",
    "y_vals = ax1.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Adjusted Gross (' + (unit if unit else 'unit') + ')'\n",
    "\n",
    "ax1.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some of the skinny bars are taller than 0.405 and others are shorter; the first two have heights of 0 because there are no data between 300 and 320. By putting a flat top at the level 0.405 across the whole bin, we are deciding to ignore the finer detail and are using the flat level as a rough approximation. Often, though not always, this is sufficient for understanding the general shape of the distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**The height as a rough approximation.**\n",
    "This observation gives us a different way of thinking about the height.\n",
    "Look again at the [300, 400) bin in the earlier histograms. As we have seen, the bin is 100 million dollars wide and contains 40.5% of the data. Therefore the height of the corresponding bar is 0.405% per million dollars.\n",
    "\n",
    "Now think of the bin as consisting of 100 narrow bins that are each 1 million dollars wide. The bar's height of \"0.405% per million dollars\" means that as a rough approximation, 0.405% of the movies are in each of those 100 skinny bins of width 1 million dollars."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that because we have the entire dataset that is being used to draw the histograms, we can draw the histograms to as fine a level of detail as the data and our patience will allow. However, if you are looking at a histogram in a book or on a website, and you don't have access to the underlying dataset, then it becomes important to have a clear understanding of the \"rough approximation\" created by the flat tops."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Histograms Q&A\n",
    "Let's draw the histogram again, this time with four bins, and check our understanding of the concepts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 21132 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "uneven_again = np.array([300, 350, 400, 450, 1500])\n",
    "\n",
    "fig, ax1 = plt.subplots()\n",
    "\n",
    "ax1.hist(millions['Adjusted Gross'], bins=uneven_again, density=True, ec='white')\n",
    "\n",
    "y_vals = ax1.get_yticks()\n",
    "\n",
    "y_label = 'Percent per ' + (unit if unit else 'unit')\n",
    "\n",
    "x_label = 'Adjusted Gross (' + (unit if unit else 'unit') + ')'\n",
    "\n",
    "ax1.set_yticklabels(['{:g}'.format(x * 100) for x in y_vals])\n",
    "\n",
    "plt.ylabel(y_label)\n",
    "\n",
    "plt.xlabel(x_label)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Adjusted Gross</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>(450.0, 1500.0]</th>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(350.0, 400.0]</th>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(299.999, 350.0]</th>\n",
       "      <td>32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>(400.0, 450.0]</th>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  Adjusted Gross\n",
       "(450.0, 1500.0]               92\n",
       "(350.0, 400.0]                49\n",
       "(299.999, 350.0]              32\n",
       "(400.0, 450.0]                25"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bin_counts_uneven_again = millions['Adjusted Gross']\n",
    "\n",
    "bin_counts_uneven_again = pd.DataFrame(bin_counts_uneven_again.value_counts(bins=uneven_again))\n",
    "\n",
    "bin_counts_uneven_again"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look again at the histogram, and compare the [400, 450) bin with the [450, 1500) bin.\n",
    "\n",
    "**Q**: Which has more movies in it? \n",
    "\n",
    "**A**: The [450, 1500) bin. It has 92 movies, compared with 25 movies in the [400, 450) bin.\n",
    "\n",
    "**Q**: Then why is the [450, 1500) bar so much shorter than the [400, 450) bar?\n",
    "\n",
    "**A**: Because height represents density per unit of space in the bin, not the number of movies in the bin. The [450, 1500) bin does have more movies than the [400, 450) bin, but it is also a whole lot wider. So it is less crowded. The density of movies in it is much lower."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Differences Between Bar Charts and Histograms\n",
    "\n",
    "- Bar charts display one quantity per category. They are often used to display the distributions of categorical variables. Histograms display the distributions of quantitative variables. \n",
    "- All the bars in a bar chart have the same width, and there is an equal amount of space between consecutive bars. The bars of a histogram can have different widths, and they are contiguous.\n",
    "- The lengths (or heights, if the bars are drawn vertically) of the bars in a bar chart are proportional to the value for each category. The heights of bars in a histogram measure densities; the *areas* of bars in a histogram are proportional to the numbers of entries in the bins."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
