{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [],
   "source": [
    "path_data = '../../../data/'\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "\n",
    "import warnings\n",
    "warnings.simplefilter(action=\"ignore\", category=FutureWarning)\n",
    "\n",
    "from urllib.request import urlopen \n",
    "import re\n",
    "def read_url(url): \n",
    "    return re.sub('\\\\s+', ' ', urlopen(url).read().decode())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "tags": [
     "remove_input"
    ]
   },
   "outputs": [],
   "source": [
    "# Read two books, fast (again)!\n",
    "\n",
    "huck_finn_path = path_data + 'huck_finn.txt'\n",
    "with open(huck_finn_path, 'r') as ff:\n",
    "    huck_finn_text = ff.read()\n",
    "huck_finn_chapters = huck_finn_text.split('CHAPTER ')[44:]\n",
    "\n",
    "little_women_path = path_data + 'little_women.txt'\n",
    "with open(little_women_path, 'r') as ff:\n",
    "    little_women_text = ff.read()\n",
    "little_women_chapters = little_women_text.split('CHAPTER ')[1:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Literary Characters\n",
    "*The Adventures of Huckleberry Finn* describes a journey that Huck and Jim take along the Mississippi River. Tom Sawyer joins them towards the end as the action heats up. Having loaded the text, we can quickly visualize how many times these characters have each been mentioned at any point in the book."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Jim  Tom  Huck\n",
      "0    0    6     3\n",
      "1   16   30     5\n",
      "2   16   35     7\n",
      "3   24   35     8\n",
      "4   24   35     8\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 21956 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "varA = np.char.count(huck_finn_chapters, 'Jim')\n",
    "varB = np.char.count(huck_finn_chapters, 'Tom')\n",
    "varC = np.char.count(huck_finn_chapters, 'Huck')\n",
    "\n",
    "chapter_mentions = pd.DataFrame({'Jim': varA.cumsum(), 'Tom': varB.cumsum(), 'Huck':varC.cumsum()})\n",
    "print(chapter_mentions.head())\n",
    "chapter_mentions.plot()\n",
    "\n",
    "plt.show()\n",
    "\n",
    "# when creating this chart we have used both pandas 'plot' and matplotlib 'plt'. This is quite common."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the plot above, the horizontal axis shows chapter numbers and the vertical axis shows how many times each character has been mentioned up to and including that chapter. \n",
    "\n",
    "You can see that Jim is a central character by the large number of times his name appears. Notice how Tom is hardly mentioned for much of the book until he arrives and joins Huck and Jim, after Chapter 30. His curve and Jim's rise sharply at that point, as the action involving both of them intensifies. As for Huck, his name hardly appears at all, because he is the narrator. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Little Women* is a story of four sisters growing up together during the civil war. In this book, chapter numbers are spelled out and chapter titles are written in all capital letters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Chapters</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ONE\\n\\nPLAYING PILGRIMS\\n\\n\"Christmas won't be...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>TWO\\n\\nA MERRY CHRISTMAS\\n\\nJo was the first t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>THREE\\n\\nTHE LAURENCE BOY\\n\\n\"Jo!  Jo!  Where ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>FOUR\\n\\nBURDENS\\n\\n\"Oh, dear, how hard it does...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>FIVE\\n\\nBEING NEIGHBORLY\\n\\n\"What in the world...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>SIX\\n\\nBETH FINDS THE PALACE BEAUTIFUL\\n\\nThe ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>SEVEN\\n\\nAMY'S VALLEY OF HUMILIATION\\n\\n\"That ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>EIGHT\\n\\nJO MEETS APOLLYON\\n\\n\"Girls, where ar...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>NINE\\n\\nMEG GOES TO VANITY FAIR\\n\\n\"I do think...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>TEN\\n\\nTHE P.C. AND P.O.\\n\\nAs spring came on,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            Chapters\n",
       "0  ONE\\n\\nPLAYING PILGRIMS\\n\\n\"Christmas won't be...\n",
       "1  TWO\\n\\nA MERRY CHRISTMAS\\n\\nJo was the first t...\n",
       "2  THREE\\n\\nTHE LAURENCE BOY\\n\\n\"Jo!  Jo!  Where ...\n",
       "3  FOUR\\n\\nBURDENS\\n\\n\"Oh, dear, how hard it does...\n",
       "4  FIVE\\n\\nBEING NEIGHBORLY\\n\\n\"What in the world...\n",
       "5  SIX\\n\\nBETH FINDS THE PALACE BEAUTIFUL\\n\\nThe ...\n",
       "6  SEVEN\\n\\nAMY'S VALLEY OF HUMILIATION\\n\\n\"That ...\n",
       "7  EIGHT\\n\\nJO MEETS APOLLYON\\n\\n\"Girls, where ar...\n",
       "8  NINE\\n\\nMEG GOES TO VANITY FAIR\\n\\n\"I do think...\n",
       "9  TEN\\n\\nTHE P.C. AND P.O.\\n\\nAs spring came on,..."
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The chapters of Little Women, as a DataFrame - index, column 0\n",
    "\n",
    "pd.DataFrame({'Chapters':little_women_chapters}).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can track the mentions of main characters to learn about the plot of this book as well.  The protagonist Jo interacts with her sisters Meg, Beth, and Amy regularly, up until Chapter 27 when she moves to New York alone."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   Amy  Beth   Jo  Meg  Laurie\n",
      "0   23    26   44   26       0\n",
      "1   36    38   65   46       0\n",
      "2   38    40  127   82      16\n",
      "3   52    58  161   99      16\n",
      "4   58    72  216  112      51\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 34496 characters]\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Counts of names in the chapters of Little Women\n",
    "\n",
    "varD = np.char.count(little_women_chapters,'Amy')\n",
    "varE = np.char.count(little_women_chapters,'Beth')\n",
    "varF = np.char.count(little_women_chapters,'Jo')\n",
    "varG = np.char.count(little_women_chapters,'Meg')\n",
    "varH = np.char.count(little_women_chapters,'Laurie')\n",
    "\n",
    "chapter_mentions = pd.DataFrame({'Amy': varD.cumsum(), \n",
    "                                 'Beth': varE.cumsum(), \n",
    "                                 'Jo':varF.cumsum(), \n",
    "                                'Meg':varG.cumsum(),\n",
    "                                'Laurie':varH.cumsum()})\n",
    "\n",
    "# cumulative sum\n",
    "\n",
    "# cumsum - numpy\n",
    "# https://numpy.org/doc/stable/reference/generated/numpy.cumsum.html#numpy-cumsum\n",
    "\n",
    "# or\n",
    "\n",
    "# cumsum - pandas\n",
    "# https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.cumsum.html#pandas-dataframe-cumsum\n",
    "\n",
    "print(chapter_mentions.head())\n",
    "chapter_mentions.plot()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Laurie is a young man who marries one of the girls in the end. See if you can use the plots to guess which one."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
