wk3-d1-ch0-01.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gipsi's 1 Million Women to Tech: Summer Of Code 2018\n",
    "\n",
    "## week 3\n",
    "\n",
    "### Natural language processing with Python\n",
    "\n",
    "[NLTK book, ch0](http://www.nltk.org/book/ch00.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loving\n",
      "#testing\n",
      "commaing\n",
      "coding\n",
      "typing\n",
      "innovating-waterskiing!smiling.panicking\n",
      "pythoning\n"
     ]
    }
   ],
   "source": [
    "for line in open(\"file.txt\"):\n",
    "    for word in line.split():\n",
    "        if word.endswith('ing'):\n",
    "            print(word)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Chapter1. - Language Processing and Python\n",
    "[NLTK book CH1](http://www.nltk.org/book/ch01.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "*** Introductory Examples for the NLTK Book ***\n",
      "Loading text1, ..., text9 and sent1, ..., sent9\n",
      "Type the name of the text or sentence to view it.\n",
      "Type: 'texts()' or 'sents()' to list the materials.\n",
      "text1: Moby Dick by Herman Melville 1851\n",
      "text2: Sense and Sensibility by Jane Austen 1811\n",
      "text3: The Book of Genesis\n",
      "text4: Inaugural Address Corpus\n",
      "text5: Chat Corpus\n",
      "text6: Monty Python and the Holy Grail\n",
      "text7: Wall Street Journal\n",
      "text8: Personals Corpus\n",
      "text9: The Man Who Was Thursday by G . K . Chesterton 1908\n"
     ]
    }
   ],
   "source": [
    "import nltk\n",
    "from nltk.book import *\n",
    "# * means everything or all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "text1: Moby Dick by Herman Melville 1851\n",
      "text2: Sense and Sensibility by Jane Austen 1811\n",
      "text3: The Book of Genesis\n",
      "text4: Inaugural Address Corpus\n",
      "text5: Chat Corpus\n",
      "text6: Monty Python and the Holy Grail\n",
      "text7: Wall Street Journal\n",
      "text8: Personals Corpus\n",
      "text9: The Man Who Was Thursday by G . K . Chesterton 1908\n"
     ]
    }
   ],
   "source": [
    "texts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sent1: Call me Ishmael .\n",
      "sent2: The family of Dashwood had long been settled in Sussex .\n",
      "sent3: In the beginning God created the heaven and the earth .\n",
      "sent4: Fellow - Citizens of the Senate and of the House of Representatives :\n",
      "sent5: I have a problem with people PMing me to lol JOIN\n",
      "sent6: SCENE 1 : [ wind ] [ clop clop clop ] KING ARTHUR : Whoa there !\n",
      "sent7: Pierre Vinken , 61 years old , will join the board as a nonexecutive director Nov. 29 .\n",
      "sent8: 25 SEXY MALE , seeks attrac older single lady , for discreet encounters .\n",
      "sent9: THE suburb of Saffron Park lay on the sunset side of London , as red and ragged as a cloud of sunset .\n"
     ]
    }
   ],
   "source": [
    "sents()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Displaying 25 of 25 matches:\n",
      " [ angels sing ] Arthur , this is the Holy Grail . Look well , Arthur , for it i\n",
      "urpose , Arthur ... the quest for the Holy Grail . [ boom ] [ singing stops ] LA\n",
      "t he can join us in our quest for the Holy Grail . FRENCH GUARD : Well , I ' ll \n",
      "egy was required if the quest for the Holy Grail were to be brought to a success\n",
      " GALAHAD : You are the keepers of the Holy Grail ? ZOOT : The what ? GALAHAD : T\n",
      "NCELOT : No , we ' ve got to find the Holy Grail . Come on ! GALAHAD : Oh , let \n",
      "ould be the sign that leads us to the Holy Grail ! Brave , brave Concorde , you \n",
      "oung people in the joyful bond of the holy wedlock . Unfortunately , one of them\n",
      " ' ve not given up your quest for the Holy Grail ? MINSTREL : [ singing ] He is \n",
      "? TIM : I do . [ zoosh ] You seek the Holy Grail ! ARTHUR : That is our quest . \n",
      " , we ' re -- we ' re looking for the Holy Grail . Our quest is to find the Holy\n",
      "Holy Grail . Our quest is to find the Holy Grail . KNIGHTS : Yeah . Yes . It is \n",
      "- TIM : Yes , I can help you find the Holy Grail . KNIGHTS : Oh , thank you . Oh\n",
      "in the last resting place of the most Holy Grail . ARTHUR : Where could we find \n",
      "ARTHUR : No . LAUNCELOT : We have the Holy Hand Grenade . ARTHUR : Yes , of cour\n",
      "nade . ARTHUR : Yes , of course ! The Holy Hand Grenade of Antioch ! ' Tis one o\n",
      " him ! Brother Maynard ! Bring up the Holy Hand Grenade ! MONKS : [ chanting ] P\n",
      "ing , ' First shalt thou take out the Holy Pin . Then , shalt thou count to thre\n",
      " , be reached , then lobbest thou thy Holy Hand Grenade of Antioch towards thy f\n",
      "liant and pure of spirit may find the Holy Grail in the Castle of uuggggggh '. A\n",
      "oon peril was no more . The quest for Holy Grail could continue . SCENE 23 : [ g\n",
      " your quest ? LAUNCELOT : To seek the Holy Grail . BRIDGEKEEPER : What is your f\n",
      "t is your quest ? ROBIN : To seek the Holy Grail . BRIDGEKEEPER : What is the ca\n",
      " is your quest ? ARTHUR : To seek the Holy Grail . BRIDGEKEEPER : What is the ai\n",
      "t Thou hast vouchsafed to us the most holy -- [ twong ] [ baaaa ] Jesus Christ !\n"
     ]
    }
   ],
   "source": [
    "text6.concordance(\"Holy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Displaying 3 of 3 matches:\n",
      "ou do ? ARTHUR : How do you do , good lady . I am Arthur , King of the Britons .\n",
      "d you become king then ? ARTHUR : The Lady of the Lake , ... [ angels sing ] ...\n",
      "e thing is , I thought your son was a lady . FATHER : I can understand that . HE\n"
     ]
    }
   ],
   "source": [
    "text6.concordance(\"lady\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Displaying 2 of 2 matches:\n",
      "minutes you ' re getting married to a girl whose father owns the biggest tracts \n",
      "BERT : I know , but I want the -- the girl that I marry to have ... [ music ] ..\n"
     ]
    }
   ],
   "source": [
    "text6.concordance(\"girl\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Displaying 24 of 24 matches:\n",
      "stops ] ARTHUR : Old woman ! DENNIS : Man ! ARTHUR : Man . Sorry . What knight \n",
      ": Old woman ! DENNIS : Man ! ARTHUR : Man . Sorry . What knight live in that ca\n",
      "UR : Well , I can ' t just call you ' Man '. DENNIS : Well , you could say ' De\n",
      " aside ! BLACK KNIGHT : I move for no man . ARTHUR : So be it ! ARTHUR and BLAC\n",
      "ble . It ' s a busy life in Camelot . MAN : I have to push the pram a lot [ out\n",
      "on . ARTHUR : Now look here , my good man -- FRENCH GUARD : I don ' t wanna tal\n",
      " t just a string of pussy jokes . OLD MAN : Get on with it . TIM THE ENCHANTER \n",
      "ear a starling -- oooh SCENE 12 : OLD MAN : Heh , hee ha ha hee hee ! Hee hee h\n",
      "u speak , he has seen the Grail ? OLD MAN : ... Ha ha ha ha ! Heh , hee ha ha h\n",
      "... ARTHUR : Where does he live ? OLD MAN : ... Heh heh heh heh ... ARTHUR : Ol\n",
      " ... Heh heh heh heh ... ARTHUR : Old man , where does he live ? OLD MAN : ... \n",
      " : Old man , where does he live ? OLD MAN : ... Hee ha ha ha . He knows of a ca\n",
      " He knows of a cave , a cave which no man has entered . ARTHUR : And the Grail \n",
      " the Grail . The Grail is there ? OLD MAN : There is much danger , for beyond t\n",
      "the Gorge of Eternal Peril , which no man has ever crossed . ARTHUR : But the G\n",
      "the Grail ! Where is the Grail ?! OLD MAN : Seek you the Bridge of Death . ARTH\n",
      "eath , which leads to the Grail ? OLD MAN : Heh , hee hee hee hee ! Ha ha ha ha\n",
      " . GUEST # 1 : He ' s killed the best man ! GUESTS : [ yelling ] FATHER : Hold \n",
      "to find the enchanter of whom the old man had spoken in scene twenty - four . B\n",
      " boom boom boom boom ] What manner of man are you that can summon up fire witho\n",
      " ARTHUR : Look , um , you ' re a busy man , uh -- TIM : Yes , I can help you fi\n",
      "a creature so foul , so cruel that no man yet has fought with it and lived ! Bo\n",
      "t . ARTHUR : Look ! There ' s the old man from scene twenty - four ! BEDEVERE :\n",
      "n . Come along . INSPECTOR : Put this man in the van . OFFICER # 1 : Clear off \n"
     ]
    }
   ],
   "source": [
    "text6.concordance(\"man\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Displaying 3 of 3 matches:\n",
      "ideal republic , where every man and woman is called under the flag for assignm\n",
      "e have proclaimed that every man and woman on this earth has rights , and digni\n",
      "riend of each nation and every man , woman , and child who seeks a future of pe\n"
     ]
    }
   ],
   "source": [
    "text4.concordance(\"woman\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ4AAAEWCAYAAABWn/G6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcXFWd9/HPFxqNCCQgGUUhaUARETGQBgWBbtxRYPAZHUAdCSODOG7oRAaeqN046AAyAq4QfTQ64ooww7iBom0QBekgEEAYtqCRLYhBAsj6e/6459I3N7V31e1u832/XvWqW+ece87vnrpVv9wl1YoIzMzMqrLBZAdgZmbrFyceMzOrlBOPmZlVyonHzMwq5cRjZmaVcuIxM7NKOfHYekvSDyUdPsE+Fkj6xQT7uFbS0ET66KZuzEsHY45I+lqVY9rkceKxaUHSCkmv7GafEbF/RHylm30WSeqXFJLWpMddkr4n6VWlOF4YEaO9iqNdvZoXSUskPZLm4l5JP5a0Ywf9dH1fsGo58Zj13qyI2AR4MfBj4DxJCyYrGEl9kzU2cEqai62Bu4ElkxiLTRInHpv2JB0g6UpJqyX9UtIuqXz79C/r3dLrZ0u6Jz+tJWlU0pGFfv5J0m8l3S/pusJ6x0m6uVD+hk7ijIg7I+IMYAQ4WdIGqf8n/wUvaQ9JY5L+nI6QPpnK86OnoyTdLukOSf9SiH2DQpx/lPRtSVuU1n27pN8BP5U0Q9LXUtvVki6X9MzyvKR+PyTpNkl3S/qqpJmlfg+X9Ls0t4tanIsHga8DO9eql3RQOgW5OsXzglT+n8Ac4H/SkdOx7b4PNvmceGxaS8nhS8A7gGcAZwHnS3pqRNwM/CtwtqSNgS8DS2qd1pL0JrKE8DZgM+Ag4I+p+mZgH2AmcALwNUlbTSDsc4G/AZ5fo+4M4IyI2AzYHvh2qX4/4HnAq4HjCqec3gscDAwCzwb+BHy2tO4g8ALgNcDhaXu2IZu3o4GHasSzID32A7YDNgE+U2qzd9qWVwAfyZNEI5I2Ad4C/KZG3Q7AN4BjgNnAD8gSzVMi4h+A3wEHRsQmEXFKs7Fs6nHisenun4CzIuKyiHg8XZt4GHgpQER8AbgRuAzYCqj3L/IjyU4DXR6ZmyLittTHdyLi9oh4IiK+lfrbYwIx356et6hR9yjwXElbRsSaiLi0VH9CRDwQEcvJEulhqfwdwKKIWBkRD5Ml0TeWTquNpHUfSuM8A3humrdlEfHnGvG8BfhkRNwSEWuA44FDS/2eEBEPRcRVwFVkpxTrWShpNXATWRJbUKPNIcD3I+LHEfEocCrwNGCvBv3aNOLEY9PdXOBf0imZ1elLbRuyf/XnvkB2SufT6Uu5lm3IjmzWIelthVN5q1NfW04g5uek53tr1L0d2AG4Pp3+OqBU//vC8m2Mb+dcsmtHeYy/BR4Hnlln3f8ELgC+mU7dnSJpoxrxPDuNUxyzr9TvnYXlB8kSSj2nRsSsiHhWRByUjkobjhkRT6TYn1OjrU1DTjw23f0e+Fj6MssfG0fEN+DJUzqnA/8PGMmve9TpZ/tyoaS5ZInr3cAzImIWcA2gCcT8BrIL6zeUKyLixog4jOxU3MnAOZKeXmiyTWF5DuNHT78H9i/Nw4yI+EOx+8I4j0bECRGxE9mRxAFkpxnLbidLasUxHwPuanFbO7HWmJJEtt35tvgn9ac5Jx6bTjZKF8XzRx9ZUjha0kuUebqk10vaNK1zBrAsIo4Evg+cWafvL5KdBpqf+nluSjpPJ/uiWwUg6QjqXBBvRtIzJb0bGAaOT/+SL7d5q6TZqW51Kn680OTDkjaW9ELgCOBbqfxM4GMpZiTNlvS3DWLZT9KLJG0I/Jns1NvjNZp+A3i/pG1TEv848K2IeKydbW/Tt4HXS3pFOgr7F7LTp79M9XeRXW+yacqJx6aTH5BdAM8fIxExRnad5zNkF9RvIl03SF+8ryW7cA7wAWA3SW8pdxwR3wE+Rnan1f3AfwFbRMR1wH8AvyL7wnsRcEmbca+W9ACwHHgd8KaI+FKdtq8FrpW0hixpHhoRfynU/zxt40Vkp60uTOVnAOcDF0q6H7gUeEmDmJ4FnEOWdH6b+q31Hzi/RHZabilwK/AX4D2NN3diIuIG4K3Ap4F7gAPJbiZ4JDX5d+BD6bTiwl7GYr0h/yE4s6lPUj/ZF/9GPT7aMOs5H/GYmVmlnHjMzKxSPtVmZmaV8hGPmZlVajJ/LHDK2nLLLaO/v3+ywzAzm1aWLVt2T0TMbtbOiaeG/v5+xsbGJjsMM7NpRdJtzVv5VJuZmVXMicfMzCrlxGNmZpVy4jEzs0o58ZiZWaWceMzMrFJOPGZmViknHjMzq5QTj5mZVcqJx8zMKuXEY2ZmlXLiMTOzSjnxmJlZpZx4zMysUk48ZmZWKSceMzOrlBOPmZlVyonHzMwq5cRjZmaVcuIxM7NKOfGYmVmlnHjMzKxSTjxmZlYpJ55JMjICs2aNL5frymW9GL8XYwwNTWz9Ykx5jJ32WauvetvdbC7y+vJzOzG00qbe9k61faJW21b342JZvW1t9LoVtfoojzUyAv392aPR+vl6eR+zZmXrDA2tXZf31c685OuV2+ZleX07c1CvbaM+ittRBUVENSNNgEQ/8CPgF8BLgauALwMnAH8DvCU1PR14GvAQcEQEN0gsAA4CNga2B86L4NhG4w0MDMTY2Fj3N6RAyp4jsuXi21Csq2L8bvc7kT6L6+cxQmd91uurVn/N4s7ry8/txNBOnLXiq1XeTe2MUattq/txeXubvRed7FOtxNJo/6oVY3kfKq7baP9qFEs5rlr7Wb0Y66k3X43msd39uv7YWhYRA83aTacjnucCZwC7ADsCbwb2BhYC/xe4Htg3gl2BjwAfL6w7DzgEeBFwiMQ2FcZtZmYFfZMdQBtujWA5gMS1wEURhMRyoB+YCXxF4nlAABsV1r0ogvvSutcBc4HfFzuXdBRwFMCcOXN6vClmZuuv6XTE83Bh+YnC6yfIEui/AT+LYGfgQGBGnXUfp0bCjYjFETEQEQOzZ8/uauBmZjZuOiWeZmYCf0jLCyYxDjMza2A6nWpr5hSyU20fAH462cE0MzwMp58+vlyuq2L8XhgcnNj6xbjy5dHR7vXVSttG9eXndmJopc3wcO3tnWr7RK22re7HxfJa+0s3Pg+1+ijP6/AwLFnSfP08xryPK68cv7OtVl8LFjSPpV4Med3cuWvXl/tspJV5LytuYxWmxV1tVavirjYzs782f413tZmZ2V8BJx4zM6uUE4+ZmVXKicfMzCrlxGNmZpVy4jEzs0o58ZiZWaWceMzMrFJOPGZmViknHjMzq5QTj5mZVcqJx8zMKuXEY2ZmlXLiMTOzSjnxmJlZpZx4zMysUk48ZmZWKSceMzOrlBOPmZlVyonHzHpiZGSyI7CiRu9H1e+VIqLaEaeBgYGBGBsbm+wwzKY1Cfz1MnU0ej+69V5JWhYRA83aVXrEI3GsxHvT8mkSP03Lr5D4msRhEsslrpE4ubDeGomTJZZJ/ERiD4lRiVskDkpt+iUulrgiPfZK5UOp7TkS10ucLaEqt9vMzMZVfaptKbBPWh4ANpHYCNgbuBE4GXg5MA/YXeLg1PbpwGgE84H7gROBVwFvAD6a2twNvCqC3YBDgE8Vxt0VOAbYCdgOeFk5MElHSRqTNLZq1aouba6ZmZVVnXiWAfMlNgUeBn5FloD2AVaTJZdVETwGnA3sm9Z7BPhRWl4O/DyCR9NyfyrfCPiCxHLgO2RJJvfrCFZG8ARwZWGdJ0XE4ogYiIiB2bNnd2t7zcyspK/KwSJ4VGIFcATwS+BqYD9ge+B3wPw6qz4aQX4G8gmypEUET0hPbsP7gbuAF5Ml1L8U1n+4sPw4FW+3mZmNm4y72pYCC9PzxcDRZEchlwKDEltKbAgcBvy8jX5nAneko5p/ADbsatRm1pbh4cmOwIoavR9Vv1eTkXguBrYCfhXBXWRHJhdHcAdwPPAz4Crgigj+u41+PwccLnEpsAPwQHfDNrN2+HbqqcW3U09xvp3azKx9U/J2ajMzMyceMzOrlBOPmZlVyonHzMwq5cRjZmaVcuIxM7NKOfGYmVmlnHjMzKxSTjxmZlYpJx4zM6uUE4+ZmVXKicfMzCrlxGNmZpVy4jEzs0o58ZiZWaWceMzMrFJOPGZmViknHjMzq9SkJR6JoyXelpYXSDy7UPdFiZ0mK7aJ6u8f/xvmIyNrP4aG1l3u7x9fHhoaX87lbfL6Yr95++K4tR79/ePrFdcpK5aX+y2OW25bLM9f1zM0BDNmZP3OmDHedtasrC6PtdhPsbw8D7ViKcdQXKe8Tfn6rW5DrX6L6xbnvRxbPnberritxT7LsZTL67WptQ+U4262jeWYGrWvt4311q8XR611+vqax99srLysVnk55mbv98hIto+2Ml49tfbTfDmvK37+W5nvcln++S7uC8XPWfG7ZrIoIiZv9DwIMQosjGBssmMBGBgYiLGxzkORsueI8eVO5G9NuY9a/bYyVrlNrbdeWnvc4jq1lssxltetpVac9WJrNYZ6cTeKr9brVrahVr+NtqPcT61tKvdfjqXWWK22aTYvtcavFW+jumbvdyfrtBJ/s7GKfbWznbXa1eqnlfmrV1dv/6z3GSvHUm9eGn0PtPIdMBGSlkXEQLN2fd0dtr50dLMQCOBq4GZgDbACGADOlngI2BP4YWr7bOCjqYunAU+JYFuJ+cAngU2Ae4AFEdyREthlwH7ALODtEVws8ULgy8BTyI7y/i6CG3u+0WZmto5KTrWlL/5FwMsjeDHwvrwugnOAMeAtEcyL4KFC3fmpbB5wFXCqxEbAp4E3RjAf+BLwscJwfRHsARwDDKeyo4EzUj8DwMp1Y9RRksYkja1atap7G29mZmup6ojn5cA5EdwDEMG97ZyCkjgWeCiCz0rsDOwM/Dj1sSFwR6H5uel5GdCfln8FLJLYGji31tFORCwGFkN2qq316MzMrB1VJR6RnWJrf0XxCuBNwL6Fvq6NYM86qzycnh8nbV8EX5e4DHg9cIHEkRH8tJN4zMxsYqpKPBcB50mcFsEfJbYo1d8PbFpeSWIu8DngtYVTcDcAsyX2jOBX6dTbDhFcW29wie2AWyL4VFreBXqXeObOhQULsuXh4bXrRkfH7ybJl5csGb+zbXQ0qyvecTI8nLVZsGC8Pu83f10et2zJkvH1iuuUDQ6uu1xve4pty9tZfl0e49JL4VnPgjvvhOOOy8pnzoR582DFinX7GRwcLy9uQ7G+vE4xhvJ2l+dqcHDdu3zqbUOtfsvvV1ExtnzsvF3+vrQ7bj2N9oF681Krj3rj1nufy9tYb/16cdRa58QT4UMfan+9Vspg3ZibzfvwMJx+eufjlcest38W98VW5rtclj+vWDG+L5x00vjnLO+/0fdAr1V2V5vE4cAHyY5EfkN2U8GaCE6V+Dvg47DOzQWvB97D+DWZ2yN4ncQ84FPATLLkeXoEXyjeHSexJTAWQb/E8cBbgUeBO4E3R3BvvVgnelebmdn6qNW72qbE7dRTjROPmVn7Wk08/uUCMzOrlBOPmZlVyonHzMwq5cRjZmaVcuIxM7NKOfGYmVmlnHjMzKxSTjxmZlYpJx4zM6uUE4+ZmVXKicfMzCrlxGNmZpVy4jEzs0o58ZiZWaWceMzMrFJOPGZmViknHjMzq5QTj5mZVartxCMxIrGwF8HY5BgZ6W4/nfQ3MtK9OGzy9fd3//2s19/ICMyYMb4PjYzABhuM1w0NjS8X+8jLux1PtwwN1f5c5HNba9uK7Yvr1VueLIqI9lYQI8CaCE7tSUTNx++L4LFejjEwMBBjY2O9HGJKkaDN3aBhP530J2XP3YjDJl8v3s96+1U+Vlm+L9ZabtTfROPpluJ2Fccpb29x28pltba1l3FLWhYRA83atXTEI7FI4gaJnwDPT2XbS/xIYpnExRI7pvIlEp+X+JnELRKDEl+S+K3EkkKfh0ksl7hG4uRC+WslrpC4SuKiVDYisVjiQuCrEv1pzCvSY6/C+semfq+SOCnFeUWh/nkSy1rZbjMz676+Zg0k5gOHArum9lcAy4DFwNER3CjxEuBzwMvTapun5YOA/wFeBhwJXC4xD7gbOBmYD/wJuFDiYOAS4AvAvhHcKrFFIZT5wN4RPCSxMfCqCP4i8TzgG8CAxP7AwcBLInhQYosI7pW4T2JeBFcCR8B4AhzfTh0FHAUwZ86clibPzMza1zTxAPsA50XwIIDE+cAMYC/gO4VDvKcW1vmfCEJiOXBXBMvTutcC/cBcYDSCVan8bGBf4HFgaQS3AkRwb6HP8yN4KC1vBHwmJbHHgR1S+SuBL+exFtb/InCExAeAQ4A9yhsZEYvJkikDAwM+4WNm1iOtJB6A8hfxBsDqCObVaf9wen6isJy/7oO612hUY6zcA4Xl9wN3AS9OsfylyfrfBYaBnwLLIvhjnTHMzKzHWkk8S4ElEiel9gcCZwG3Srwpgu9ICNglgqtaHPcy4AyJLclOtR0GfBr4FfBZiW3zU22lo57cTGBlBE9IHA5smMovBD4i8fXiqbZ0Su4C4PPA21uMcb0xPNzdfjrpr1sx2NQwdy4sWNDdPuvtI8PDcNJJcNxx42Uf/eh43eho7fUHB3sTT7cMDta+8644t/W2rVxWb3mytHRXm8Qi4G3AbcBK4Dqyo4jPA1uRnfr6ZgQfTTcQfC+CcyT60/LOqZ9i3ZuB48mOUn4QwbGpzf7Ax8mOZO6O4FXlO+nSdZ3vAg8CPwPeE8Emqe64FOsjqd//m8pfmtaZE8HjjbZ3fburzcysG1q9q63t26mnq/R/j2ZG8OFmbZ14zMza12riafUaz7QmcR6wPeN33ZmZ2SRZLxJPBG+Y7BjMzCzj32ozM7NKOfGYmVmlnHjMzKxSTjxmZlYpJx4zM6uUE4+ZmVXKicfMzCrlxGNmZpVy4jEzs0o58ZiZWaWceMzMrFJOPGZmViknHjMzq5QTj5mZVcqJx8zMKuXEY2ZmlepZ4pF4r8RvJc7ucr8j6c9Y/1Xo74eRkfHXIyPrvu6F4jhDQ+uO081x875GRrKxGrUpt283jrz/Xs1bO/L4+/vXLmvUvp3ydtoV576d/auVsYvvab7N5bLcrFn1x6j3vtfrq1lM+aP8On9PZs1ae8y8DLLloSGYMWPtz0cxluJ8luMtb0t5/vP6Ygz52HnbvK64/xTr8+Wy4njFtv390NeXbdOMGdlyrfir+OwoInrTsbge2D+CWwtlfRE8NsF+R4A1EZw6wRDrGhgYiLGxsV51vxYpe87fhlqve/EWFccpj9ntcfO+ao1Ta7xW2rcyVo927bZiybXyftara3VbmvWdx1Frrjvps1abZttcq7/ye9boM9DOXBTjKL4uK9fXa19v7hp9hsv9lNvXel/qjV3etnr7eaPYG5V34ztH0rKIGGjWridHPBJnAtsB50vcJ7FY4kLgqxIbSnxC4nKJqyXeUVjvg4XyEwrliyRukPgJ8PxC+TyJS1P78yQ2T+WjEqdJLE1HXbtLnCtxo8SJvdhmMzNrTU8STwRHA7cD+wGnAfOBv43gzcDbgfsi2B3YHfgniW0lXg08D9gDmAfMl9hXYj5wKLAr8H/SOrmvAv8awS7AcmC4UPdIBPsCZwL/DbwL2BlYIPGMcsySjpI0Jmls1apVXZsLMzNbW19F45wfwUNp+dXALhJvTK9nkiWcV6fHb1L5Jql8U+C8CB4EkDg/Pc8EZkXw89T+K8B3imOm5+XAtRHckda7BdgG+GMxwIhYDCyG7FTbRDfYzMxqqyrxPFBYFvCeCC4oNpB4DfDvEZxVKj8G6CQRPJyenygs56+r2m4zMyuZjC/gC4B3Svw0gkcldgD+kMr/TeLsCNZIPAd4FFgKLJE4KcV7IHBWBPdJ/ElinwguBv4Bnjz6mTbmzoUFC8ZfDw+vXV9+3S3FfgcH173brJvj5n0ND8PoaPN4iu3bNTjY+brdlsewZMm6ZY3at1reTrvi3Nea6076zOVzXmxffJ+LfcycWX+Meu97rf7bian8emgoe09Wr4ZjjhnvNy+D7HPZ3w+XXgovfen456PYT719rfh5KtbVmv+ZM8djyOcmf6+uvDKrK+4/xfpaYxfLRkezOPK2c+fCypXjd7M99ljtPqr47PTyrrYVwADwbgp3oUlsAJxIlkAErAIOTonkfcCRqYs1wFsjuFliEfA24DZgJXBdBKdKzCO7hrMxcAtwRAR/khgFFkYwJjGUlg9I4z9ZVy/2Ku9qMzP7a9HqXW09SzzTmROPmVn7JvV2ajMzs3qceMzMrFJOPGZmViknHjMzq5QTj5mZVcqJx8zMKuXEY2ZmlXLiMTOzSjnxmJlZpZx4zMysUk48ZmZWKSceMzOrlBOPmZlVyonHzMwq5cRjZmaVcuIxM7NKOfGYmVmlnHjMzKxSUy7xSIxILGxQP0/idYXXB0kcV010rRkZaVw/NLRu2/w5rxsaWrcOYNas2v3XatuortxPcfyRkfFHXtfONrWiUZz9/fXXmTVr3bbFmMvtG81LrfJG21mvrtb72UixTb19ofj+19POuMX9qvicz1Gr718r29fpOp303e0Y/lrHh+afBWj/c9wpRUQ1I7VIYgRYE8GpdeoXAAMRvLtXMQwMDMTY2FjH60vQaFqL9flyrWdYuyxvn5fX6rPW2LXqyv2Ux82VY+l0m1tp32gbyjE3irdR+2ZxNNqOVvpoZR7qta+1Te3uR83a1tvPmo1Va8xWtbpOJ313O4ZemezxizF0so+3PoaWRcRAs3ZT4ohHYpHEDRI/AZ6fykYlBtLylhIrJJ4CfBQ4ROJKiUMkFkh8JrWbLfFdicvT42WpfDC1v1LiNxKbTtKmmpmt9/omOwCJ+cChwK5k8VwBLKvVNoJHJD5C4YgnHQHlzgBOi+AXEnOAC4AXAAuBd0VwicQmwF/WjUNHAUcBzJkzp0tbZ2ZmZZOeeIB9gPMieBBA4vwJ9PVKYKfC6YPN0tHNJcAnJc4Gzo1gZXnFiFgMLIbsVNsEYjAzswamQuIBqPVF/xjjpwJntNjPBsCeETxUKj9J4vvA64BLJV4ZwfWdhWpmZhMxFRLPUmCJxElk8RwInAWsAOYDvwbeWGh/P9S9RnMh8G7gE5DdARfBlRLbR7AcWC6xJ7Aj9C7xDA83rh8cXLdt/pzXDQ6O32FS7G/mTDjmmPpj1hq7Vl25n+L45Ttbmm1PMe5WNYpz7tz665x++rpt8zvbGo1RbxvK5Y22tV5drfezkWKbevvC6GjzO4zaGbe4XxWf8/VGRxuv3+o4E1mnk767HcNf6/jFGBrF0u7nuFNT4q42iUXA24DbgJXAdcD3gG8Da4CfAm+NoF9iC7JrNxsB/w48jXTNR2JL4LNk13X6gKURHC3xaWA/4PHU94IIHq4Xz0TvajMzWx+1elfblEg8U40Tj5lZ+6bV7dRmZrb+cOIxM7NKOfGYmVmlnHjMzKxSTjxmZlYpJx4zM6uUE4+ZmVXKicfMzCrlxGNmZpVy4jEzs0o58ZiZWaWceMzMrFJOPGZmViknHjMzq5QTj5mZVcqJx8zMKuXEY2ZmlXLiMTOzSvU08Ui8QSIkduxR/wMSn+pF32Zm1hu9PuI5DPgFcGi3O5boi2Asgvd2u+9uGhqCkZFseWQke12FfMzycjf7ne4mY1umy/yNjEB/f3vrDA1l6wwNje/3eT/563z/L34uivr71y3P+8vjahRzeb0ZM5rPebl+1qy142kUS/F1Xr/BBuPb0deX9ZfPRbFd3kc+Rl5fnL/iPBbnb4MNsn77+rL1y/Oel8+YMR7L0BBI4+1nzVr7vcnHb/d974Qiojcdi02AG4D9gPMj2FFiCDgBuAuYB5wLLAfeBzwNODiCmyVmA2cCc1J3x0RwicQI8GygH7gHWAwsjOCANN6ngQEggBMi+K7E54HdU//nRDDcLPaBgYEYGxvrwixkbzRAxNrLvSaNj1Nc7ma/091kbMt0mb9O9tV8nWYafRZqlZc/Q/ViKtcV42m0HbXWq/fZqfV5LvbfbA7K/dbatlb6aHWu29VsjpuRtCwiBpq16+us+5YcDPwogv+VuFdit1T+YuAFwL3ALcAXI9hD4n3Ae4BjgDOA0yL4hcQc4IK0DsB8YO8IHkqJLPdh4L4IXgQgsXkqXxTBvRIbAhdJ7BLB1T3bajMza6iXiecw4PS0/M30+vvA5RHcASBxM3BharOc7OgI4JXAToWsvpnEpmn5/AgeqjHeKymc0ovgT2nx7yWOItvWrYCdYN3EI+ko4CiAOXPmlKvNzKxLepJ4JJ4BvBzYWSKADclOf/0AeLjQ9InC6ycK8WwA7FlOMCkRPVBv2DRGsf22wEJg9wj+JLEEmFFr5YhYTHbqjoGBgWlwMsTMbHrq1c0FbwS+GsHcCPoj2Aa4Fdi7xfUvBN6dv5CY18E6mwObkSWq+ySeCezf4vhmZtYjvTrVdhhwUqnsu8A7gZtbWP+9wGclriaLcSlwdJN1TkzrXAM8TnZzwbkSvwGuJbuedEnrm9Adg4Pjd6IMD8PoaDXjDg/XXu5mv9PdZGzLdJm/4WFYsqS9dQYHYcWK8bui8v1+yZLxu6fy/b/4uSiaOxcWLFi332JcjWIur3fppXDccY3jLq83c+ba8TSKpfx5HhyEpUthzpxsO048ETbZBI45pnY/xXnO48jr8vkqzlM+ngSbbQZr1sDWW2dlxXk/8cSs/M474VnPymIZHYWf/3x8m1avXvvOtnrb3As9u6ttOuvmXW1mZuuLVu9q8y8XmJlZpZx4zMysUk48ZmZWKSceMzOrlBOPmZlVyonHzMwq5cRjZmaVcuIxM7NKOfGYmVmlnHjMzKxSTjxmZlYpJx4zM6uUE4+ZmVXKicfMzCrlxGNmZpVy4jEzs0o58ZiZWaWceMzMrFJOPGZmViknHjMzq5QTj5mZVcqJx8zMKqWImOwYphxJq4DbOlx9S+CeLobTK46zuxxn90yHGMFx1jI3ImY3a+TE02WSxiJiYLLjaMZxdpfj7J7pECM4zonwqTYzM6uUE4+ZmVXKiaf7Fk8jVRA0AAAISklEQVR2AC1ynN3lOLtnOsQIjrNjvsZjZmaV8hGPmZlVyonHzMwq5cTTRZJeK+kGSTdJOq6C8baR9DNJv5V0raT3pfItJP1Y0o3pefNULkmfSvFdLWm3Ql+Hp/Y3Sjq8UD5f0vK0zqckaQLxbijpN5K+l15vK+myNOa3JD0llT81vb4p1fcX+jg+ld8g6TWF8q7MvaRZks6RdH2a1z2n4nxKen96z6+R9A1JM6bCfEr6kqS7JV1TKOv5/NUbo40YP5He86slnSdpVqdz1Mn70GqchbqFkkLSlpM5lx2LCD+68AA2BG4GtgOeAlwF7NTjMbcCdkvLmwL/C+wEnAIcl8qPA05Oy68DfggIeClwWSrfArglPW+eljdPdb8G9kzr/BDYfwLxfgD4OvC99PrbwKFp+UzgnWn5n4Ez0/KhwLfS8k5pXp8KbJvme8Nuzj3wFeDItPwUYNZUm0/gOcCtwNMK87hgKswnsC+wG3BNoazn81dvjDZifDXQl5ZPLsTY9hy1+z60E2cq3wa4gOw/uW85mXPZ8XdBtztcXx/pDbyg8Pp44PiKY/hv4FXADcBWqWwr4Ia0fBZwWKH9Dan+MOCsQvlZqWwr4PpC+Vrt2oxta+Ai4OXA99LOfk/hw/7k/KUP1Z5puS+1U3lO83bdmntgM7IvdJXKp9R8kiWe36cvk740n6+ZKvMJ9LP2l3rP56/eGK3GWKp7A3B2rW1vNked7NftxgmcA7wYWMF44pm0uezk4VNt3ZN/GeRWprJKpMP2XYHLgGdGxB0A6flvmsTYqHxljfJOnA4cCzyRXj8DWB0Rj9Xo+8l4Uv19qX278bdrO2AV8GVlpwS/KOnpTLH5jIg/AKcCvwPuIJufZUy9+cxVMX/1xujEP5IdAXQSYyf7dcskHQT8ISKuKlVN1bmsyYmne2qdq6/kXnVJmwDfBY6JiD83alqjLDoobze+A4C7I2JZC7E0qutpnGT/Ct0N+HxE7Ao8QHaqoZ7Jms/Ngb8lO/XzbODpwP4N+p6s+WxmysUlaRHwGHB2XtRmLJ3s163GtjGwCPhIreo245m07ytw4ummlWTnXnNbA7f3elBJG5ElnbMj4txUfJekrVL9VsDdTWJsVL51jfJ2vQw4SNIK4Jtkp9tOB2ZJ6qvR95PxpPqZwL0dxN+ulcDKiLgsvT6HLBFNtfl8JXBrRKyKiEeBc4G9mHrzmati/uqN0bJ04f0A4C2RzjN1EOM9tP8+tGp7sn9sXJU+S1sDV0h6Vgdx9nQum+r2ubv19UH2r+VbyHaM/GLjC3s8poCvAqeXyj/B2hcHT0nLr2ftC5C/TuVbkF3b2Dw9bgW2SHWXp7b5BcjXTTDmIcZvLvgOa1+E/ee0/C7Wvgj77bT8Qta+0HsL2UXers09cDHw/LQ8kuZySs0n8BLgWmDj1M9XgPdMlflk3Ws8PZ+/emO0EeNrgeuA2aV2bc9Ru+9DO3GW6lYwfo1n0uayo89Ztztcnx9kd5b8L9ndLosqGG9vssPjq4Er0+N1ZOeNLwJuTM/5jibgsym+5cBAoa9/BG5KjyMK5QPANWmdz9DkYmgLMQ8xnni2I7uz5qb0YX1qKp+RXt+U6rcrrL8oxXIDhTvCujX3wDxgLM3pf6UP65SbT+AE4PrU13+SfTFO+nwC3yC77vQo2b+q317F/NUbo40YbyK7FpJ/js7sdI46eR9ajbNUv4LxxDMpc9npwz+ZY2ZmlfI1HjMzq5QTj5mZVcqJx8zMKuXEY2ZmlXLiMTOzSjnxmHVI0mmSjim8vkDSFwuv/0PSBybQ/4ikhXXqjkq/pny9pF9L2rtQt4+yX66+UtLT0i8vXyvpE22O3y/pzZ3Gb1aPE49Z535J9osBSNoA2JLsPxzm9gIuaaUjSRu2Omj6CaJ3AHtHxI7A0cDX0/9gB3gLcGpEzIuIh1Lb3SLig62OkfQDTjzWdU48Zp27hJR4yBLONcD9kjaX9FTgBcBv0t9K+YSyv52zXNIhAJKGlP09pa+T/ac/JC1Kf+PlJ8Dz64z7r8AHI+IegIi4guzXC94l6Ujg74GPSDpb0vlkv+V2maRDJL0pxXGVpKVpzA1TfJenv+XyjjTOScA+6cjp/d2cOFu/9TVvYma1RMTtkh6TNIcsAf2K7Bd+9yT75eGrI+IRSX9H9osILyY7Kro8/9IH9gB2johbJc0n+ymVXck+m1eQ/ep02QtrlI8Bh0fEh9Npt+9FxDkAktZExLy0vBx4TUT8QeN/7OztwH0RsXtKmJdIupDs51IWRsQBE5sps7U58ZhNTH7UsxfwSbLEsxdZ4vllarM38I2IeJzsBxh/DuwO/JnsN7VuTe32Ac6LiAcB0tFKq0Rrvy58CbBE0rfJflwUsj+CtoukN6bXM4HnAY+0Mb5Zy3yqzWxi8us8LyI71XYp2RFP8fpOoz9v/UDpdSvJ4zpgfqlst1TeUEQcDXyI7BeLr5T0jBTfe9I1oXkRsW1EXNhCHGYdceIxm5hLyH5K/96IeDwi7iX7c9l7kp16A1gKHJKupcwm+5PGv67R11LgDelOtE2BA+uMeQpwckoaSJpH9qevP9csWEnbR8RlEfERsp/wz/+M8jvTn9hA0g7pD+DdT/Yn1c26yqfazCZmOdl1m6+XyjbJL/4D55EloqvIjmiOjYg7Je1Y7CgirpD0LbJfR76N7E80rCMizpf0HOCXkoIsQbw10l+NbOITkp5HdpRzUYrparI72K6QJLK/wnpwKn9M0lXAkog4rYX+zZryr1ObmVmlfKrNzMwq5cRjZmaVcuIxM7NKOfGYmVmlnHjMzKxSTjxmZlYpJx4zM6vU/wcj+b/Sx08YKAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# dispersion plots need numpy and matplotlib\n",
    "# can be installed with pip if you don't have Anaconda\n",
    "text4.dispersion_plot([\"man\", \"woman\", \"citizens\", \"democracy\", \"freedom\", \"duties\", \"America\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZ4AAAEWCAYAAABWn/G6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xe4XGXd7vHvTRJqIJRERSQEEKQJgQwovYgKSD2iAVEJghEbokaEwysEj7wvRQWsGDk0BaQfYyVIC9L3DqkgUgKKRAiGICW05Hf+WM9OVoZdpzyzt9yf65pr1qzyPL+1ZrLvvZ41WVsRgZmZWS4rtLoAMzN7a3HwmJlZVg4eMzPLysFjZmZZOXjMzCwrB4+ZmWXl4LG3LEl/kHRknW2Mk/TnOtuYI2mPetpopEYclxr6nCjplzn7tNZx8NiAIOlxSXs3ss2I2DciLmlkm2WSRkkKSS+mx9OSfivpg1V1bBkRtzarjr5q1nGRdLGk19KxWCDpRkmb1dBOwz8LlpeDx6z51oyIocA2wI3A9ZLGtaoYSYNb1TdwVjoW7wKeAS5uYS3WIg4eG/Ak7S9puqSFku6UtHWav3H6zXq79Pqdkp7tGNaSdKukY0rtfFbSg5JekPRAabsTJT1amn9ILXVGxD8j4jxgInCmpBVS+0t/g5e0g6Q2Sf9OZ0jfT/M7zp7GS3pK0jxJXy/VvkKpzn9JukrS2lXbHi3pb8DNklaW9Mu07kJJ90l6e/VxSe3+l6QnJD0j6VJJw6raPVLS39KxPbmXx+Jl4HJgq86WSzowDUEuTPVsnub/AhgJ/CadOZ3Q1/fBWs/BYwNaCocLgc8B6wA/AyZLWikiHgW+CVwmaVXgIuDizoa1JH2MIhA+DawBHAj8Ky1+FNgVGAacBvxS0rp1lH0d8DbgPZ0sOw84LyLWADYGrqpaviewCfAh4MTSkNNxwMHA7sA7geeAH1dtuzuwOfBh4Mi0P+tTHLdjgUWd1DMuPfYENgKGAj+qWmeXtC8fAE7pCInuSBoKHAHc38myTYErgOOBEcDvKYJmxYj4FPA34ICIGBoRZ/XUl/U/Dh4b6D4L/Cwi7omIxenaxKvA+wEi4ufAw8A9wLpAV7+RH0MxDHRfFB6JiCdSG1dHxFMRsSQirkzt7VBHzU+l57U7WfY68G5JwyPixYi4u2r5aRHxUkTMogjSw9P8zwEnR8STEfEqRYgeWjWsNjFtuyj1sw7w7nTc2iPi353UcwTw/Yh4LCJeBE4CDqtq97SIWBQRM4AZFEOKXZkgaSHwCEWIjetknbHA7yLixoh4HfgusAqwUzft2gDi4LGBbgPg62lIZmH6obY+xW/9HX5OMaTzw/RDuTPrU5zZvImkT5eG8hamtobXUfN66XlBJ8uOBjYF/pKGv/avWv730vQTLNvPDSiuHXXU+CCwGHh7F9v+ArgB+FUaujtL0pBO6nln6qfc5+Cqdv9Zmn6ZIlC68t2IWDMi3hERB6az0m77jIglqfb1OlnXBiAHjw10fwdOTz/MOh6rRsQVsHRI51zg/wITO657dNHOxtUzJW1AEVxfAtaJiDWB2YDqqPkQigvrD1UviIiHI+JwiqG4M4FrJK1WWmX90vRIlp09/R3Yt+o4rBwR/yg3X+rn9Yg4LSK2oDiT2J9imLHaUxShVu7zDeDpXu5rLZbrU5Io9rtjX3xL/QHOwWMDyZB0UbzjMZgiFI6V9D4VVpP0EUmrp23OA9oj4hjgd8D5XbR9AcUw0JjUzrtT6KxG8YNuPoCko+jignhPJL1d0peAU4GT0m/y1et8UtKItGxhmr24tMq3JK0qaUvgKODKNP984PRUM5JGSDqom1r2lPReSYOAf1MMvS3uZNUrgK9K2jCF+H8DV0bEG33Z9z66CviIpA+ks7CvUwyf3pmWP01xvckGKAePDSS/p7gA3vGYGBFtFNd5fkRxQf0R0nWD9IN3H4oL5wBfA7aTdER1wxFxNXA6xTetXgD+H7B2RDwAfA+4i+IH3nuBO/pY90JJLwGzgP2Aj0XEhV2suw8wR9KLFKF5WES8Ulp+W9rHmyiGraak+ecBk4Epkl4A7gbe101N7wCuoQidB1O7nf0HzgsphuWmAnOBV4Avd7+79YmIh4BPAj8EngUOoPgywWtplf8B/isNK05oZi3WHPIfgjPr/ySNovjBP6TJZxtmTeczHjMzy8rBY2ZmWXmozczMsvIZj5mZZdXKmwX2W8OHD49Ro0a1ugwzswGlvb392YgY0dN6Dp5OjBo1ira2tlaXYWY2oEh6oue1PNRmZmaZOXjMzCwrB4+ZmWXl4DEzs6wcPGZmlpWDx8zMsnLwmJlZVg4eMzPLysFjZmZZOXjMzCwrB4+ZmWXl4DEzs6wcPGZmlpWDx8zMsnLwmJlZVg4eMzPLysFjZmZZOXjMzCwrB4+ZmWXl4DEzs6wcPGZmlpWDx8zMsnLwmJlZVg6eJpg4sTHr9badrrbt6/ZdrV9rHbXsX2/ndWfUKNhjj/raqfVY1POeNUOj6qn3M1D93EyN6KM3bXT2GWuG7mqpXpZr3+uliGh+L3WSGAX8Efgz8H5gBnARcBrwNuCItOq5wCrAIuCoCB6SGAccCKwKbAxcH8EJ3fVXqVSira2tnnrpzWHtab3ettPVttC37bvqr9Y6atm/3s7rqV+or51aj0U971kzNKqeej8D1c/N1Ig+etNGrve6u36ql+Xa9663VXtEVHpabyCd8bwbOA/YGtgM+ASwCzAB+N/AX4DdItgWOAX479K2o4GxwHuBsRLrZ6zbzMxKBre6gD6YG8EsAIk5wE0RhMQsYBQwDLhEYhMggCGlbW+K4Pm07QPABsDfy41LGg+MBxg5cmSTd8XM7K1rIJ3xvFqaXlJ6vYQiQP8PcEsEWwEHACt3se1iOgnciJgUEZWIqIwYMaKhhZuZ2TIDKXh6Mgz4R5oe18I6zMysGwNpqK0nZ1EMtX0NuLmVhZx6amPW6207jdq2q21qraOW/evtvO5ssEHxzbZ62qn1WNTznjVDo+qp9zNQ/dxMjeijN23svnv9/fRGd7VUL8u17/UaEN9qy63eb7WZmb0V/Sd+q83MzP4DOHjMzCwrB4+ZmWXl4DEzs6wcPGZmlpWDx8zMsnLwmJlZVg4eMzPLysFjZmZZOXjMzCwrB4+ZmWXl4DEzs6wcPGZmlpWDx8zMsnLwmJlZVg4eMzPLysFjZmZZOXjMzCwrB4+ZmWXl4LEBa+LEVldgjdSo93MgfS7qqXUg7Wc1RUSra+h3KpVKtLW1tboM64EE/vj+52jU+zmQPhf11Nof91NSe0RUelov6xmPxAkSx6XpcyRuTtMfkPilxOESsyRmS5xZ2u5FiTMl2iX+JLGDxK0Sj0kcmNYZJXG7xLT02CnN3yOte43EXyQuk1DO/TYzs2VyD7VNBXZN0xVgqMQQYBfgYeBMYC9gNLC9xMFp3dWAWyMYA7wAfAf4IHAI8O20zjPAByPYDhgL/KDU77bA8cAWwEbAztWFSRovqU1S2/z58xu0u2ZmVi138LQDYyRWB14F7qIIoF2BhRThMj+CN4DLgN3Sdq8Bf0zTs4DbIng9TY9K84cAP5eYBVxNETId7o3gyQiWANNL2ywVEZMiohIRlREjRjRqf83MrMrgnJ1F8LrE48BRwJ3ATGBPYGPgb8CYLjZ9PYKO0cwlFKFFBEukpfvwVeBpYBuKQH2ltP2rpenFZN5vMzNbphXfapsKTEjPtwPHUpyF3A3sLjFcYhBwOHBbH9odBsxLZzWfAgY1tGrrd049tdUVWCM16v0cSJ+LemodSPtZrRXBczuwLnBXBE9TnJncHsE84CTgFmAGMC2CX/eh3Z8AR0rcDWwKvNTYsq2/GchfJ7U389ep823bav46dSf8dWozs77rl1+nNjMzc/CYmVlWDh4zM8vKwWNmZlk5eMzMLCsHj5mZZeXgMTOzrBw8ZmaWlYPHzMyycvCYmVlWDh4zM8vKwWNmZlk5eMzMLCsHj5mZZeXgMTOzrBw8ZmaWlYPHzMyycvCYmVlWLQseiWMlPp2mx0m8s7TsAoktWlWbmZk1z+BWdRzB+aWX44DZwFNp2TGtqMnMzJovW/Cks5sJQAAzgUeBF4HHgQpwmcQiYEfgD2nddwLfTk2sAqwYwYYSY4DvA0OBZ4FxEcyTuBW4B9gTWBM4OoLbJbYELgJWpDjL+2gEDzd9p83M7E2yDLWlH/wnA3tFsA3wlY5lEVwDtAFHRDA6gkWlZZPTvNHADOC7EkOAHwKHRjAGuBA4vdTd4Ah2AI4HTk3zjgXOS+1UgCffXKPGS2qT1DZ//vzG7byZmS0n1xnPXsA1ETwLEMECqfcbS5wALIrgxxJbAVsBN6Y2BgHzSqtfl57bgVFp+i7gZIl3Add1drYTEZOASQCVSiV6X52ZmfVFruARxRBb3zcUHwA+BuxWamtOBDt2scmr6Xkxaf8iuFziHuAjwA0Sx0Rwcy31mJlZfXJ9q+0m4OMS6wBIrF21/AVg9eqNJDYAfgJ8vDQE9xAwQiqCR2JIGsrrksRGwGMR/ACYDGxdz86YmVntspzxRDBH4nTgNonFwP0UXyrocDFwfunLBR3GAesA16dhtaci2E/iUOAHEsMo9uFcYE43JYwFPinxOvBPln1hwczMMlOEL2dUq1Qq0dbW1uoyzMwGFEntEVHpaT3fucDMzLJy8JiZWVYOHjMzy8rBY2ZmWTl4zMwsKwePmZll5eAxM7OsHDxmZpaVg8fMzLJy8JiZWVYOHjMzy8rBY2ZmWTl4zMwsKwePmZll5eAxM7OsHDxmZpaVg8fMzLJy8JiZWVZ9Dh6JiRITmlGMmZn95xtwZzwSg1tdg5mZ1a5XwSNxssRDEn8C3pPmbSzxR4l2idslNkvzL5b4qcQtEo9J7C5xocSDEheX2jxcYpbEbIkzS/P3kZgmMUPipjRvosQkiSnApRKjUp/T0mOn0vYnpHZnSJyR6pxWWr6JRHudx83MzGrU49mDxBjgMGDbtP40oB2YBBwbwcMS7wN+AuyVNlsrTR8I/AbYGTgGuE9iNPAMcCYwBngOmCJxMHAH8HNgtwjmSqxdKmUMsEsEiyRWBT4YwSsSmwBXABWJfYGDgfdF8LLE2hEskHheYnQE04GjYFkALttPjQfGA4wcObJXB8/MzPquN8NWuwLXR/AygMRkYGVgJ+Bqael6K5W2+U0EITELeDqCWWnbOcAoYAPg1gjmp/mXAbsBi4GpEcwFiGBBqc3JESxK00OAH6UQWwxsmubvDVzUUWtp+wuAoyS+BowFdqjeyYiYRBGmVCqV6MVxMTOzGvT2ekn1D+IVgIURjO5i/VfT85LSdMfrwcAbXWynTvrq8FJp+qvA08A2qZZXetj+WuBU4GagPYJ/ddGHmZk1WW+u8UwFDpFYRWJ14ADgZWCuxMcAJCSxTR/6vQfYXWK4xCDgcOA24K40f8PU7tpdbD8MmBfBEuBTwKA0fwrwmTQUt3T7CF4BbgB+ClzUhzrNzKzBegyeCKYBVwLTKc4cbk+LjgCOlpgBzAEO6m2nEcwDTgJuAWYA0yL4dRp6Gw9cl9q9sosmfgIcKXE3xTDbS6ndPwKTgTaJ6bDc174vozgbmtLbOs3MrPEU8da4nJH+79GwCL7V07qVSiXa2toyVGVm9p9DUntEVHpa7y3xf2Ikrgc2Ztm37szMrEXeEsETwSGtrsHMzAoD7s4FZmY2sDl4zMwsKwePmZll5eAxM7OsHDxmZpaVg8fMzLJy8JiZWVYOHjMzy8rBY2ZmWTl4zMwsKwePmZll5eAxM7OsHDxmZpaVg8fMzLJy8JiZWVYOHjMzy6ppwSNxnMSDEpc1uN2J6c9Ym5nZANTMv0D6BWDfCOZ2zJAYHMEbTezTzMz6uaac8UicD2wETJZ4XmKSxBTgUolBEmdL3CcxU+Jzpe2+UZp/Wmn+yRIPSfwJeE9p/miJu9P610uslebfKnGOxNR01rW9xHUSD0t8pxn7bGZmvdOU4IngWOApYE/gHGAMcFAEnwCOBp6PYHtge+CzEhtKfAjYBNgBGA2MkdhNYgxwGLAt8L/SNh0uBb4ZwdbALODU0rLXItgNOB/4NfBFYCtgnMQ61TVLGi+pTVLb/PnzG3YszMxsec0caiubHMGiNP0hYGuJQ9PrYRSB86H0uD/NH5rmrw5cH8HLABKT0/MwYM0IbkvrXwJcXe4zPc8C5kQwL233GLA+8K9ygRExCZgEUKlUot4dNjOzzuUKnpdK0wK+HMEN5RUkPgz8TwQ/q5p/PFBLELyanpeUpjte59pvMzOr0oqvU98AfF5iCIDEphKrpfmfkRia5q8n8TZgKnCIxCoSqwMHAETwPPCcxK6p3U/B0rMfMzPrp1rxm/8FwChgmoSA+cDBEUyR2By4SwLgReCTEUyTuBKYDjwB3F5q60jgfIlVgceAo7LthZmZ1UQRvpxRrVKpRFtbW6vLMDMbUCS1R0Slp/V85wIzM8vKwWNmZlk5eMzMLCsHj5mZZeXgMTOzrBw8ZmaWlYPHzMyycvCYmVlWDh4zM8vKwWNmZlk5eMzMLCsHj5mZZeXgMTOzrBw8ZmaWlYPHzMyycvCYmVlWDh4zM8vKwWNmZln1u+CRmCgxoZvloyX2K70+UOLEPNWZmVm9+l3w9MJoWBY8EUyO4IwW1mNmZn3QL4JH4mSJhyT+BLwnzbtVopKmh0s8LrEi8G1grMR0ibES4yR+lNYbIXGtxH3psXOav3taf7rE/RKrt2hXzcze8ga3ugCJMcBhwLYU9UwD2jtbN4LXJE4BKhF8KW0/rrTKecA5EfxZYiRwA7A5MAH4YgR3SAwFXnlzHRoPjAcYOXJkg/bOzMyqtTx4gF2B6yN4GUBich1t7Q1sIS19vUY6u7kD+L7EZcB1ETxZvWFETAImAVQqlaijBjMz60Z/CB6Azn7Qv8GyocCVe9nOCsCOESyqmn+GxO8org3dLbF3BH+prVQzM6tHf7jGMxU4RGKVdHZyQJr/ODAmTR9aWv8F6PIazRQohuCg+AZcet44glkRnAm0AZs1rnwzM+uLlgdPBNOAK4HpwLXA7WnRd4HPS9wJDC9tcgvFcNp0ibFVzR0HVCRmSjwAHJvmHy8xW2IGsAj4Q5N2x8zMeqAIX86oVqlUoq2trdVlmJkNKJLaI6LS03otP+MxM7O3FgePmZll5eAxM7OsHDxmZpaVg8fMzLJy8JiZWVYOHjMzy8rBY2ZmWTl4zMwsKwePmZll5eAxM7OsHDxmZpaVg8fMzLJy8JiZWVYOHjMzy8rBY2ZmWTl4zMwsKwePmZll1dTgkThEIiQ2a1L7FYkfNKNtMzNrjmaf8RwO/Bk4rNENSwyOoC2C4xrdtpmZNU/TgkdiKLAzcDQpeCT2kLhN4iqJv0qcIXGExL0SsyQ2TuuNkLhW4r702DnNnygxSWIKcGlq77cd/UlclNqZKfHRNP+nEm0ScyROa9b+mplZ7wxuYtsHA3+M4K8SCyS2S/O3ATYHFgCPARdEsIPEV4AvA8cD5wHnRPBniZHADWkbgDHALhEsktij1N+3gOcjeC+AxFpp/skRLJAYBNwksXUEM5u212Zm1q1mBs/hwLlp+lfp9e+A+yKYByDxKDAlrTML2DNN7w1sIS1taw2J1dP05AgWddLf3pSG9CJ4Lk1+XGI8xb6uC2wBbw4eSeOB8QAjR47sy36amVkfNCV4JNYB9gK2kghgEBDA74FXS6suKb1eUqpnBWDH6oBJQfRSV92mPsrrbwhMALaP4DmJi4GVO9s4IiYBkwAqlUp0to6ZmdWvWdd4DgUujWCDCEZFsD4wF9ill9tPAb7U8UJidA3brAWsQRFUz0u8Hdi3l/2bmVmTNCt4Dgeur5p3LfCJXm5/HFBJXxJ4ADi2F9t8B1hLYrbEDGDPCGYA9wNzgAuBO3rZv5mZNYkiPKpUrVKpRFtbW6vLMDMbUCS1R0Slp/V85wIzM8vKwWNmZlk5eMzMLCsHj5mZZeXgMTOzrBw8ZmaWlYPHzMyycvCYmVlWDh4zM8vKwWNmZlk5eMzMLCsHj5mZZeXgMTOzrBw8ZmaWlYPHzMyycvCYmVlWDh4zM8vKwWNmZlk5eMzMLCsHj5mZZeXgMTOzrBw8ZmaWlSKi1TX0O5LmA0/UsOlw4NkGl9MI/bGu/lgTuK6+6I81Qf+sqz/WBI2va4OIGNHTSg6eBpLUFhGVVtdRrT/W1R9rAtfVF/2xJuifdfXHmqB1dXmozczMsnLwmJlZVg6exprU6gK60B/r6o81gevqi/5YE/TPuvpjTdCiunyNx8zMsvIZj5mZZeXgMTOzrBw8DSJpH0kPSXpE0okZ+ntc0ixJ0yW1pXlrS7pR0sPpea00X5J+kGqbKWm7UjtHpvUflnRkDXVcKOkZSbNL8xpWh6QxaT8fSduqxpomSvpHOl7TJe1XWnZSav8hSR8uze/0PZW0oaR7Uq1XSlqxl8dqfUm3SHpQ0hxJX2n18eqmppYeL0krS7pX0oxU12ndtSVppfT6kbR8VK311lDTxZLmlo7V6DQ/y+e9tO0gSfdL+m2rj1WPIsKPOh/AIOBRYCNgRWAGsEWT+3wcGF417yzgxDR9InBmmt4P+AMg4P3APWn+2sBj6XmtNL1WH+vYDdgOmN2MOoB7gR3TNn8A9q2xponAhE7W3SK9XysBG6b3cVB37ylwFXBYmj4f+Hwvj9W6wHZpenXgr6n/lh2vbmpq6fFK9Q9N00OAe9Ix6LQt4AvA+Wn6MODKWuutoaaLgUM7WT/L573U39eAy4Hfdnfccxyrnh4+42mMHYBHIuKxiHgN+BVwUAvqOAi4JE1fAhxcmn9pFO4G1pS0LvBh4MaIWBARzwE3Avv0pcOImAosaEYdadkaEXFXFP8yLi211deaunIQ8KuIeDUi5gKPULyfnb6n6TfQvYBrOtm/nuqaFxHT0vQLwIPAerTweHVTU1eyHK+0zy+ml0PSI7ppq3wMrwE+kPruU7011tSVLJ93AEnvAj4CXJBed3fcm36seuLgaYz1gL+XXj9J9/94GyGAKZLaJY1P894eEfOg+IECvK2H+ppVd6PqWC9NN6q+L6UhjwuVhrNqqGkdYGFEvFFPTWl4Y1uK35r7xfGqqglafLzS0NF04BmKH86PdtPW0v7T8udT3w397FfXFBEdx+r0dKzOkbRSdU297Lue9+9c4ARgSXrd3XHPcqy64+BpjM7GYZv9PfWdI2I7YF/gi5J262bdrurLXXdf62hkfT8FNgZGA/OA77WqJklDgWuB4yPi392tmqu2Tmpq+fGKiMURMRp4F8Vv3Zt301aWuqprkrQVcBKwGbA9xfDZN3PWJGl/4JmIaC/P7qatVv47BBw8jfIksH7p9buAp5rZYUQ8lZ6fAa6n+If5dDpdJz0/00N9zaq7UXU8mabrri8ink4/NJYAP6c4XrXU9CzFkMngWmqSNITiB/xlEXFdmt3S49VZTf3leKVaFgK3Ulwn6aqtpf2n5cMohlub8tkv1bRPGq6MiHgVuIjaj1Wtn/edgQMlPU4xDLYXxRlQvzhWnarnApEfSy/qDaa4QLghyy6+bdnE/lYDVi9N30lxbeZslr9IfVaa/gjLX+S8N81fG5hLcYFzrTS9dg31jGL5C/kNqwO4L63bcbF1vxprWrc0/VWKsWyALVn+gupjFBdTu3xPgatZ/qLtF3pZkyjG7c+tmt+y49VNTS09XsAIYM00vQpwO7B/V20BX2T5C+ZX1VpvDTWtWzqW5wJn5P68l2rcg2VfLmjZseqxzno29mO5N3w/im8EPQqc3OS+Nkpv/gxgTkd/FOO0NwEPp+eOD7OAH6faZgGVUlufobiI+AhwVA21XEExFPM6xW9GRzeyDqACzE7b/Ih0t40aavpF6nMmMJnlf7CenNp/iNK3iLp6T9PxvzfVejWwUi+P1S4UQxQzgenpsV8rj1c3NbX0eAFbA/en/mcDp3TXFrByev1IWr5RrfXWUNPN6VjNBn7Jsm++Zfm8V9W4B8uCp2XHqqeHb5ljZmZZ+RqPmZll5eAxM7OsHDxmZpaVg8fMzLJy8JiZWVYOHrMapdujHF96fYOkC0qvvyfpa3W0P1HShC6WjZf0l/S4V9IupWW7prsnT5e0iqSz0+uz+9j/KEmfqLV+s644eMxqdyewE4CkFYDhFP8Jr8NOwB29aUjSoN52mm6R8jlgl4jYDDgWuFzSO9IqRwDfjYjREbEorbtdRHyjt30kowAHjzWcg8esdneQgocicGYDL0haK90ocnPg/vR3Wc6WNDv9rZWxAJL2UPG3cC6n+A+GSDo5/d2TPwHv6aLfbwLfiIhnAaK4u/QlFPfsOwb4OHCKpMskTaa4u8U9ksZK+liqY4akqanPQam++9KNLj+X+jkD2DWdOX21kQfO3toG97yKmXUmIp6S9IakkRQBdBfFXXt3pLjj78yIeE3SRylutrkNxVnRfR0/9Cnu67VVRMyVNIbiFibbUvzbnAa082ZbdjK/DTgyIr6Vht1+GxHXAEh6MYobWyJpFvDhiPiHpDXTtkcDz0fE9ikw75A0heL2PRMiYv/6jpTZ8hw8ZvXpOOvZCfg+RfDsRBE8d6Z1dgGuiIjFFDcEvY3iTsb/prh/19y03q7A9RHxMkA6W+kt0bs7Bt8BXCzpKqDjJqUfAraWdGh6PQzYBHitD/2b9ZqH2szq03Gd570UQ213U5zxlK/vdPfni1+qet2b8HgAGFM1b7s0v1sRcSzwXxR3G54uaZ1U35fTNaHREbFhREzpRR1mNXHwmNXnDoo7FC+I4s8ILADWpAifu9I6U4Gx6VrKCIo/zX1vJ21NBQ5J30RbHTigiz7PAs5MoYGk0cA44Cc9FStp44i4JyJOofiTBesDNwCfT38eAUmbSloNeIHiz2GbNZSH2szqM4vius3lVfOGdlz8p/h7STtS3E08gBMi4p+SNis3FBHTJF1JcYfoJyhuu/8mETFZ0nrAnZKCIiA+GemvmPbgbEmbUJzl3JRqmknxDbZp6U8gz6f4M8kzgTckzQAujohzetG+WY98d2ozM8vKQ21mZpaVg8fMzLJy8JiZWVaI69gnAAAAHUlEQVQOHjMzy8rBY2ZmWTl4zMwsKwePmZll9f8B5wZF+nXQZNAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Guess which text!\n",
    "text5.dispersion_plot([\"man\", \"woman\", \"citizens\", \"democracy\", \"freedom\", \"duties\", \"America\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAacAAAEWCAYAAADCeVhIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcHFW9/vHPQxLWQNiighAGEFlECKRBQCABAQUB4V4wRBCCYMSfG2rkwo2awZ9eBblsbhC5gAsg+zUimihbkE0mgRC2yC5IhGAECYQt+d4/6jQpmpnM1j19JvO8X69+dfWpU+ecqumZJ3Wq0q2IwMzMLCcrNHsAZmZmtRxOZmaWHYeTmZllx+FkZmbZcTiZmVl2HE5mZpYdh5MNWJJ+J+moXrYxXtKfetnGfZLG9KaNeqrHcelBn62SftmXfVreHE7WL0h6XNJe9WwzIvaNiJ/Vs80ySS2SQtLC9HhG0jWS9q4Zx/si4sZGjaO7GnVcJF0o6bV0LBZI+oOkLXrQTt3fC5Yfh5NZ460ZEUOBbYE/AFdLGt+swUga3Ky+gVPTsdgAeBa4sIljsYw5nKzfk7S/pLslPS/pVknbpPJN07/Qt0+v15f0XHUKTdKNko4ttfNpSQ9IelHS/aXtTpT0SKn84J6MMyL+HhFnAa3AKZJWSO2/eSYgaUdJbZL+lc60Tk/l1bOwCZKeljRP0ldLY1+hNM5/SLpM0to12x4j6a/A9ZJWlvTLVPd5SXdKemftcUntfl3SE5KelfRzScNq2j1K0l/TsZ3UxWPxMnAxsHV76yUdmKY7n0/j2TKV/wIYAfwmnYGd0N2fg/UPDifr11KAnA98BlgHOBeYKmmliHgE+A/gIkmrAhcAF7Y3hSbpUIrQOBJYAzgQ+Eda/QiwGzAMOBn4paT1ejHsq4B3AJu3s+4s4KyIWAPYFLisZv0ewGbAPsCJpemtLwIHAaOB9YF/Aj+q2XY0sCXwYeCotD8bUhy344BF7YxnfHrsAWwCDAV+WFNn17QvHwK+WQ2SZZE0FDgcuKudde8FLgGOB4YD11KE0YoR8Ungr8ABETE0Ik7trC/rnxxO1t99Gjg3Iu6IiMXpWsmrwE4AEfFT4CHgDmA9oKN/2R9LMeV0ZxQejognUhuXR8TTEbEkIi5N7e3YizE/nZ7Xbmfd68B7JK0bEQsj4vaa9SdHxEsRMYcibMel8s8AkyLiqYh4lSJoD6mZwmtN2y5K/awDvCcdt5kR8a92xnM4cHpEPBoRC4GTgMNq2j05IhZFxGxgNsX0ZUcmSnoeeJgi6Ma3U2cs8NuI+ENEvA6cBqwC7LKMdm0543Cy/m4j4Ktp+uf59IdvQ4qzh6qfUkwf/SD94W7PhhRnSG8j6cjStOHzqa11ezHmd6fnBe2sOwZ4L/Bgmmrbv2b9k6XlJ1i6nxtRXMuqjvEBYDHwzg62/QUwDfhVmiY8VdKQdsazfuqn3Ofgmnb/Xlp+mSJ0OnJaRKwZEe+KiAPT2e0y+4yIJWns726nri2nHE7W3z0JfCf9was+Vo2IS+DN6aMzgf8BWqvXYTpoZ9PaQkkbUYTb54F1ImJN4F5AvRjzwRQ3A8ytXRERD0XEOIppv1OAKyStVqqyYWl5BEvPwp4E9q05DitHxN/KzZf6eT0iTo6IrSjOSPanmNKs9TRF8JX7fAN4pov72hNv6VOSKPa7ui/+KoUBwOFk/cmQdCG/+hhMERzHSfqACqtJ+qik1dM2ZwEzI+JY4LfAOR20fR7FlNOo1M57UjCtRvHHcD6ApKPp4CJ+ZyS9U9LngcnASemMoLbOEZKGp3XPp+LFpSrfkLSqpPcBRwOXpvJzgO+kMSNpuKSPLWMse0h6v6RBwL8opvkWt1P1EuDLkjZOQf9fwKUR8UZ39r2bLgM+KulD6WzuqxRTtbem9c9QXP+y5ZjDyfqTayku2lcfrRHRRnHd6YcUNwE8TLqOkf44f4TiYj/AV4DtJR1e23BEXA58h+IOsheB/wXWjoj7gf8GbqP4o/h+4JZujvt5SS8Bc4D9gEMj4vwO6n4EuE/SQopgPSwiXimtvynt43UUU2TTU/lZwFRguqQXgduBDyxjTO8CrqAIpgdSu+39J9jzKaYAZwCPAa8AX1j27vZORMwFjgB+ADwHHEBxA8Rrqcp3ga+nKcyJjRyLNY/8ZYNm+ZPUQhEOQxp81mKWBZ85mZlZdhxOZmaWHU/rmZlZdnzmZGZm2WnmB0Bma911142WlpZmD8PMrF+ZOXPmcxExvB5tOZza0dLSQltbW7OHYWbWr0h6ovNaXeNpPTMzy47DyczMsuNwMjOz7DiczMwsOw4nMzPLjsPJzMyy43AyM7PsOJzMzCw7DiczM8uOw8nMzLLjcDIzs+w4nMzMLDsOJzMzy47DyczMsuNwMjOz7DiczMwsOw4nMzPLjsPJzMyy43AyM7PsOJzMzCw7DiczM8uOw8nMzLLjcDIzs+w4nPqRlpbiubW1eFSXq8rL7b02AxgzZtnrB9r7ZqDtb3+hiGj2GDol0QL8HvgTsBMwG7gAOBl4B3B4qnomsAqwCDg6grkS44EDgVWBTYGrIzhhWf1VKpVoa2ur/470kgQRxTMsXa7+CMvL7b02g87fFwPtfTPQ9reRJM2MiEo92upPZ07vAc4CtgG2AD4B7ApMBP4TeBDYPYLtgG8C/1XadiQwFng/MFZiwz4ct5mZddPgZg+gGx6LYA6AxH3AdRGExBygBRgG/ExiMyCAIaVtr4vghbTt/cBGwJPlxiVNACYAjBgxosG7YmZmy9KfzpxeLS0vKb1eQhGy/x+4IYKtgQOAlTvYdjHthHJETImISkRUhg8fXteBm5lZ9/SncOrMMOBvaXl8E8dhZma91J+m9TpzKsW03leA65s9mEbYaKPiefLkpWUdLbf32gxg9Ohlrx9o75uBtr/9Rb+4W6+v5Xq3nplZzgbq3XpmZjZAOJzMzCw7DiczM8uOw8nMzLLjcDIzs+w4nMzMLDsOJzMzy47DyczMsuNwMjOz7DiczMwsOw4nMzPLjsPJzMyy43AyM7PsOJzMzCw7DiczM8uOw8nMzLLjcDIzs+w4nMzMLDsOJzMzy47DqR8bMwZaW5e+ri6Xy+qhO+3Vu++cjRnT7BGYLb8UEc0eQ3YqlUq0tbU1exidkorn6o9QKparz/Xsp6vt1bvvnA2kfTXrCkkzI6JSj7b69MxJ4gSJL6blMySuT8sfkvilxDiJORL3SpxS2m6hxCkSMyX+KLGjxI0Sj0ocmOq0SNwsMSs9dknlY1LdKyQelLhIQn2532Zm1j19Pa03A9gtLVeAoRJDgF2Bh4BTgD2BkcAOEgeluqsBN0YwCngR+DawN3Aw8K1U51lg7wi2B8YCZ5f63Q44HtgK2AT4YO3AJE2Q1Capbf78+XXaXTMz64m+DqeZwCiJ1YFXgdsoQmo34HmKAJofwRvARcDuabvXgN+n5TnATRG8npZbUvkQ4KcSc4DLKYKo6s8RPBXBEuDu0jZviogpEVGJiMrw4cPrtb9mZtYDg/uyswhel3gcOBq4FbgH2APYFPgrMKqDTV+PoDq7v4Qi2IhgifTmPnwZeAbYliJ0Xylt/2ppeTF9vN9mZtY9zbhbbwYwMT3fDBxHcTZzOzBaYl2JQcA44KZutDsMmJfOjj4JDKrrqDM0ejRMnrz0dXW5XFYP3Wmv3n3nbPToZo/AbPnVjDOIm4FJwG0RvCTxCnBzBPMkTgJuAARcG8Gvu9Huj4ErJQ5NbbxU74Hn5sYb3/rat5L3rdrjb2b141vJ29FfbiU3M8tJv72V3MzMrCscTmZmlh2Hk5mZZcfhZGZm2XE4mZlZdhxOZmaWHYeTmZllx+FkZmbZcTiZmVl2HE5mZpYdh5OZmWXH4WRmZtlxOJmZWXYcTmZmlh2Hk5mZZcfhZGZm2XE4mZlZdhxOZmaWnaaFk8RxEkem5fES65fWnSexVbPG1lutrc0egZlZ/6aIaPYYkLgRmBhBW7PHAlCpVKKtredDkSCDw2pm1qckzYyISj3aGlyPRroinSVNBAK4B3gEWAg8DlSAiyQWATsDv0t11we+lZpYBVgxgo0lRgGnA0OB54DxEcxLIXcHsAewJnBMBDdLvA+4AFiR4mzx3yN4qOE7bWZmPdIn03opHCYBe0awLfCl6roIrgDagMMjGBnBotK6qalsJDAbOE1iCPAD4JAIRgHnA98pdTc4gh2B44HJqew44KzUTgV46u1j1ARJbZLa5s+fX7+dNzOzbuurM6c9gSsieA4gggVS1zeWOAFYFMGPJLYGtgb+kNoYBMwrVb8qPc8EWtLybcAkiQ2Aq9o7a4qIKcAUKKb1uj46MzOrt74KJ1FM53V/Q/Eh4FBg91Jb90WwcwebvJqeF5P2L4KLJe4APgpMkzg2gut7Mh4zM2u8vrpb7zrg4xLrAEisXbP+RWD12o0kNgJ+DHy8NN03FxguFeEkMSRNG3ZIYhPg0QjOBqYC2/RmZzozeXLndczMrGN9cuYUwX0S3wFuklgM3EVxI0TVhcA5pRsiqsYD6wBXpym8pyPYT+IQ4GyJYRT7cCZw3zKGMBY4QuJ14O8svcmiIXwruZlZ72RxK3luensruZnZQFTPW8n9CRFmZpYdh5OZmWXH4WRmZtlxOJmZWXYcTmZmlh2Hk5mZZcfhZGZm2XE4mZlZdhxOZmaWHYeTmZllx+FkZmbZcTiZmVl2HE5mZpYdh5OZmWXH4WRmZtlxOJmZWXYcTmZmlh2Hk5mZZafb4STRKjGxEYMxs/pqbS0etWVmuVNEdG8D0QosjOC0hoyo8/4HR/BGI/uoVCrR1tbWyC7M+oRUPJd/zaW3vjarF0kzI6JSj7a6dOYkMUlirsQfgc1T2aYSv5eYKXGzxBap/EKJn0jcIPGoxGiJ8yUekLiw1OY4iTkS90qcUir/iMQsidkS16WyVokpEtOBn0u0pD5npccupe1PSO3OlvheGues0vrNJGb28riZmVkDDe6sgsQo4DBgu1R/FjATmAIcF8FDEh8AfgzsmTZbKy0fCPwG+CBwLHCnxEjgWeAUYBTwT2C6xEHALcBPgd0jeExi7dJQRgG7RrBIYlVg7whekdgMuASoSOwLHAR8IIKXJdaOYIHECxIjI7gbOBqWhuTS/dQEYALAiBEjunTwzMysMToNJ2A34OoIXgaQmAqsDOwCXF6dNgBWKm3zmwhCYg7wTARz0rb3AS3ARsCNEcxP5RcBuwOLgRkRPAYQwYJSm1MjWJSWhwA/TEG3GHhvKt8LuKA61tL25wFHS3wFGAvsWLuTETGFInCpVCqe9DAza6KuhBNA7R/rFYDnIxjZQf1X0/OS0nL19WDo8JqR2umr6qXS8peBZ4Bt01he6WT7K4HJwPXAzAj+0UEfZmaWga5cc5oBHCyxisTqwAHAy8BjEocCSEhi2270ewcwWmJdiUHAOOAm4LZUvnFqd+0Oth8GzItgCfBJYFAqnw58Kk37vbl9BK8A04CfABd0Y5xm/drkycWjtswsd52eOUUwS+JS4G7gCeDmtOpw4CcSX6eYZvsVMLsrnUYwT+Ik4AaKs51rI/g1gMQE4CqJFSiuTe3dThM/Bq5M4XgD6awqgt+nqb42ideAa4H/TNtcBPwbRYCZDQjt3TbuW8mtP+j2reT9Vfq/WcMi+EZndX0ruZlZ99XzVvKuXnPq1ySuBjZl6d2EZmaWsQERThEc3OwxmJlZ1/mz9czMLDsOJzMzy47DyczMsuNwMjOz7DiczMwsOw4nMzPLjsPJzMyy43AyM7PsOJzMzCw7DiczM8uOw8nMzLLjcDIzs+w4nMzMLDsOJzMzy47DyczMsuNwMjOz7DQsnCS+KPGAxEV1brc1feW6taO1dWD3Xy/Ly36Y9VeKiMY0LB4E9o3gsVLZ4Aje6GW7rcDCCE7r5RA7VKlUoq2trVHNN5QEDfqR9ov+62V52Q+zviRpZkRU6tFWQ86cJM4BNgGmSrwgMUViOvBziUES35e4U+Ieic+UtvtaqfzkUvkkibkSfwQ2L5WPlLg91b9aYq1UfqPEGRIz0tnbDhJXSTwk8e1G7LOZmdVPQ8IpguOAp4E9gDOAUcDHIvgEcAzwQgQ7ADsAn5bYWGIfYDNgR2AkMEpid4lRwGHAdsC/pW2qfg78RwTbAHOAyaV1r0WwO3AO8Gvgc8DWwHiJdWrHLGmCpDZJbfPnz6/bsTAzs+4b3Ef9TI1gUVreB9hG4pD0ehhFKO2THnel8qGpfHXg6gheBpCYmp6HAWtGcFOq/zPg8nKf6XkOcF8E89J2jwIbAv8oDzAipgBToJjW6+0Om5lZz/VVOL1UWhbwhQimlStIfBj4bgTn1pQfD/QkLF5Nz0tKy9XXfbXfZmbWA824lXwa8FmJIQAS75VYLZV/SmJoKn+3xDuAGcDBEqtIrA4cABDBC8A/JXZL7X4S3jyLGrAmT+68zvLcf70sL/th1l814wziPKAFmCUhYD5wUATTJbYEbpMAWAgcEcEsiUuBu4EngJtLbR0FnCOxKvAocHSf7UWmmn0LdLP7r5flZT/M+quG3Uren/XnW8nNzJol+1vJzczMesPhZGZm2XE4mZlZdhxOZmaWHYeTmZllx+FkZmbZcTiZmVl2HE5mZpYdh5OZmWXH4WRmZtlxOJmZWXYcTmZmlh2Hk5mZZcfhZGZm2XE4mZlZdhxOZmaWHYeTmZllx+FkZmbZyS6cJFolJi5j/UiJ/UqvD5Q4sW9GZ8ub1tZmjyA/tcdkzBgfp0Zrbe27Y9xffpaKiGaP4S0kWoGFEZzWwfrxQCWCzzdqDJVKJdra2hrVvGVEgsx+BZqu9phIxbOPU+P05TFu5Hte0syIqNSjrSzOnCQmScyV+COweSq7UaKSlteVeFxiReBbwFiJuyXGSoyX+GGqN1ziSok70+ODqXx0qn+3xF0SqzdpV83MrAsGN3sAEqOAw4DtKMYzC5jZXt0IXpP4JqUzp3QmVXUWcEYEf5IYAUwDtgQmAp+L4BaJocArbx+HJgATAEaMGFGnvTMzs55oejgBuwFXR/AygMTUXrS1F7BV9RQZWCOdJd0CnC5xEXBVBE/VbhgRU4ApUEzr9WIMZmbWSzmEE0B7YfAGS6cdV+5iOysAO0ewqKb8exK/BfYDbpfYK4IHezZUMzNrtByuOc0ADpZYJZ3lHJDKHwdGpeVDSvVfhA6vGU2HpTdKSIxMz5tGMCeCU4A2YIv6Dd/6s8mTmz2C/NQek9GjfZwabfLkvjvG/eVnmcXdehKTgCOBJ4CngPuBa4DLgIXA9cAREbRIrE1xLWkI8F1gFdI1KIl1gR9RXGcaDMyI4DiJHwB7AItT2+MjeLWj8fhuPTOz7qvn3XpZhFNuHE5mZt233N1KbmZmVuZwMjOz7DiczMwsOw4nMzPLjsPJzMyy43AyM7PsOJzMzCw7DiczM8uOw8nMzLLjcDIzs+w4nMzMLDsOJzMzy47DyczMsuNwMjOz7DiczMwsOw4nMzPLjsPJzMyy43AyM7PsNDScJA6WCIktGtR+ReLsRrRtZmbN0+gzp3HAn4DD6t2wxOAI2iL4Yr3btu5rbe1Zva5u11vVfsr99VXfXdXZeHoy3tbWt+57bvtsPbdC6a93PX+uubxHFBGNaVgMBeYCewBTI9hCYgxwMvAMMBK4CpgDfAlYBTgogkckhgPnACNSc8dHcItEK7A+0AI8B0wBJkawf+rvB0AFCODkCK6U+AmwQ2r/iggmdzb2SqUSbW1tdTgKA4cEXXkr1dbr6na9Ve2n3F9f9d1VnY2nJ+OViufqvleXrf9r1Hu5N21JmhkRlXqMY3A9GunAQcDvI/iLxAKJ7VP5tsCWwALgUeC8CHaU+BLwBeB44CzgjAj+JDECmJa2ARgF7BrBohR2Vd8AXojg/QASa6XySREskBgEXCexTQT3NGyvzcys1xoZTuOAM9Pyr9Lr3wJ3RjAPQOIRYHqqM4fiLAtgL2Cr6r/0gDUkVk/LUyNY1E5/e1GaPozgn2nx4xITKPZ1PWAreHs4SZoATAAYMWJE7WozM+tDDQkniXWAPYGtJQIYRDHVdi3waqnqktLrJaXxrADsXBtCKaxe6qjb1Ee5/sbARGCHCP4pcSGwcnsbR8QUimlCKpWKJz7MzJqoUTdEHAL8PIKNImiJYEPgMWDXLm4/Hfh89YXEyB5ssxawBkWYvSDxTmDfLvZvZmZN1KhpvXHA92rKrgQ+CzzShe2/CPxI4h6KMc4Ajutkm2+nbe4FFlPcEHGVxF3AfRTXt27p+i5Yd0zu9DaT9ut1dbveqvZT7q+v+u6qzsbTk/HmvL/WO6XLHnX92ebyPmnY3Xr9me/WMzPrvnreredPiDAzs+w4nMzMLDsOJzMzy47DyczMsuNwMjOz7DiczMwsOw4nMzPLjsPJzMyy43AyM7PsOJzMzCw7DiczM8uOw8nMzLLjcDIzs+w4nMzMLDsOJzMzy47DyczMsuNwMjOz7DiczMwsOw4nMzPLjsPJzMyy43AyM7PsOJzMzCw7iohmjyE7kuYDTzR7HE20LvBcsweRAR+Hgo9DwcehsKzjsFFEDK9HJw4nextJbRFRafY4ms3HoeDjUPBxKPTVcfC0npmZZcfhZGZm2XE4WXumNHsAmfBxKPg4FHwcCn1yHHzNyczMsuMzJzMzy47DyczMsuNwWk5J2lDSDZIekHSfpC+l8rUl/UHSQ+l5rVQuSWdLeljSPZK2L7V1VKr/kKSjSuWjJM1J25wtSX2/p10jaZCkuyRdk15vLOmOtE+XSloxla+UXj+c1reU2jgplc+V9OFS+UdS2cOSTuzrfesqSWtKukLSg+l9sfNAfD9I+nL6nbhX0iWSVh4I7wdJ50t6VtK9pbKG//w76qNTEeHHcvgA1gO2T8urA38BtgJOBU5M5ScCp6Tl/YDfAQJ2Au5I5WsDj6bntdLyWmndn4Gd0za/A/Zt9n4v43h8BbgYuCa9vgw4LC2fA3w2Lf8/4Jy0fBhwaVreCpgNrARsDDwCDEqPR4BNgBVTna2avb8dHIOfAcem5RWBNQfa+wF4N/AYsErpfTB+ILwfgN2B7YF7S2UN//l31Een4232AfOjbx7Ar4G9gbnAeqlsPWBuWj4XGFeqPzetHwecWyo/N5WtBzxYKn9LvZwewAbAdcCewDXpl+c5YHBavzMwLS1PA3ZOy4NTPQEnASeV2pyWtntz21T+lnq5PIA10h9l1ZQPqPcDRTg9mf64Dk7vhw8PlPcD0MJbw6nhP/+O+ujs4Wm9ASBNRWwH3AG8MyLmAaTnd6Rq1V/aqqdS2bLKn2qnPEdnAicAS9LrdYDnI+KN9Lo89jf3N61/IdXv7vHJzSbAfOCCNL15nqTVGGDvh4j4G3Aa8FdgHsXPdyYD7/1Q1Rc//476WCaH03JO0lDgSuD4iPjXsqq2UxY9KM+KpP2BZyNiZrm4narRybp+fRwo/tW/PfCTiNgOeIliiqUjy+VxSNc7PkYxFbc+sBqwbztVl/f3Q2eavt8Op+WYpCEUwXRRRFyVip+RtF5avx7wbCp/CtiwtPkGwNOdlG/QTnluPggcKOlx4FcUU3tnAmtKGpzqlMf+5v6m9cOABXT/+OTmKeCpiLgjvb6CIqwG2vthL+CxiJgfEa8DVwG7MPDeD1V98fPvqI9lcjgtp9KdMv8DPBARp5dWTQWqd9gcRXEtqlp+ZLpLZyfghXQKPg3YR9Ja6V+d+1DMqc8DXpS0U+rryFJb2YiIkyJig4hoobigfX1EHA7cABySqtUeh+rxOSTVj1R+WLp7a2NgM4oLwHcCm6W7vVZMfUztg13rloj4O/CkpM1T0YeA+xlg7weK6bydJK2axlk9DgPq/VDSFz//jvpYtmZfoPOjMQ9gV4rT6nuAu9NjP4r58uuAh9Lz2qm+gB9R3Gk0B6iU2voU8HB6HF0qrwD3pm1+SM3F9twewBiW3q23CcUfk4eBy4GVUvnK6fXDaf0mpe0npX2dS+lOtHRc/5LWTWr2fi5j/0cCbek98b8Ud1sNuPcDcDLwYBrrLyjuuFvu3w/AJRTX2V6nONM5pi9+/h310dnDH19kZmbZ8bSemZllx+FkZmbZcTiZmVl2HE5mZpYdh5OZmWXH4WTWQ5LOkHR86fU0SeeVXv+3pK/0ov1WSRM7WDdBxaeLPyjpz5J2La3bTcWnbt8taRVJ30+vv9/N/lskfaKn4zfrDYeTWc/dSvHpAkhaAVgXeF9p/S7ALV1pSNKgrnaaPpLpM8CuEbEFcBxwsaR3pSqHA6dFxMiIWJTqbh8RX+tqH0kL4HCypnA4mfXcLaRwogileyn+l/xaklYCtgTuSv/L/vsqvj9ojqSxAJLGqPjOrYsp/qMjkiap+C6gPwKbv71LAP4D+FpEPAcQEbMovg7jc5KOBT4OfFPSRZKmUnx+3B2Sxko6NI1jtqQZqc9BaXx3qvjuns+kfr4H7JbOwL5czwNn1pnBnVcxs/ZExNOS3pA0giKkbqP4JOadKT69+p6IeE3Sv1N8OsO2FGdXd1aDAdgR2DoiHpM0iuLjbraj+N2cRfGJ2bXe1055G3BURHwjTfFdExFXAEhaGBEj0/Ic4MMR8TdJa6Ztj6H4eJodUqjeImk6xQfDToyI/Xt3pMy6z+Fk1jvVs6ddgNMpwmkXinC6NdXZFbgkIhZTfAjmTcAOwL+AP0fEY6nebsDVEfEyQDrr6SrRtU+BvgW4UNJlFB96CsXno20jqfrZcsMoPivutW70b1ZXntYz653qdaf3U0zr3U5x5lS+3rSsryt/qeZ1VwLmfmBUTdn2qXyZIuI44OsUnyx9t6R10vi+kK5RjYyIjSNiehfGYdYwDiez3rkF2B9YEBGLI2IBxdef70wxzQcwAxibru0Mp/i67D+309YM4OB0h93qwAEd9HkqcEoKFiSNpPiq8R93NlhJm0bEHRHxTYpvdd2Q4pOmP6viK1aQ9F4VX0T4IrB6p0fArAE8rWfWO3MoriNdXFM2tHrDAnA1RVjNpjgzOiEi/i5pi3JDETFL0qUUnyD/BHBzex1GxFRJ7wZulRQUIXJEpG8b7cT3JW1GcbZ0XRrTPRTjW6yOAAAATklEQVR35s1KX3cwHzgolb8haTZwYUSc0YX2zerCn0puZmbZ8bSemZllx+FkZmbZcTiZmVl2HE5mZpYdh5OZmWXH4WRmZtlxOJmZWXb+D34K3anwoOL6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Wall Street Journal\n",
    "text7.dispersion_plot([\"man\", \"woman\", \"citizens\", \"democracy\", \"freedom\", \"duties\", \"America\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "true contemptible christian abundant few part mean careful puzzled\n",
      "mystifying passing curious loving wise doleful gamesome singular\n",
      "delightfully perilous fearless\n"
     ]
    }
   ],
   "source": [
    "text1.similar(\"monstrous\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "board chairman form percentage deaths unit journal risk institute\n",
      "university type billion number total rate support class kind areas\n",
      "process\n"
     ]
    }
   ],
   "source": [
    "text7.similar(\"dilemma\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['austen-emma.txt',\n",
       " 'austen-persuasion.txt',\n",
       " 'austen-sense.txt',\n",
       " 'bible-kjv.txt',\n",
       " 'blake-poems.txt',\n",
       " 'bryant-stories.txt',\n",
       " 'burgess-busterbrown.txt',\n",
       " 'carroll-alice.txt',\n",
       " 'chesterton-ball.txt',\n",
       " 'chesterton-brown.txt',\n",
       " 'chesterton-thursday.txt',\n",
       " 'edgeworth-parents.txt',\n",
       " 'melville-moby_dick.txt',\n",
       " 'milton-paradise.txt',\n",
       " 'shakespeare-caesar.txt',\n",
       " 'shakespeare-hamlet.txt',\n",
       " 'shakespeare-macbeth.txt',\n",
       " 'whitman-leaves.txt']"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Accessing a text archive by loading an NLTK package then asking to see\n",
    "# the file identifiers.\n",
    "import nltk\n",
    "nltk.corpus.gutenberg.fileids()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "192427"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Give the text a name and find how many words.\n",
    "# This defines emma, invokes the words() function of the gutenberg object\n",
    "# in NLTK's corpus package.\n",
    "emma = nltk.corpus.gutenberg.words('austen-emma.txt')\n",
    "len(emma)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['austen-emma.txt',\n",
       " 'austen-persuasion.txt',\n",
       " 'austen-sense.txt',\n",
       " 'bible-kjv.txt',\n",
       " 'blake-poems.txt',\n",
       " 'bryant-stories.txt',\n",
       " 'burgess-busterbrown.txt',\n",
       " 'carroll-alice.txt',\n",
       " 'chesterton-ball.txt',\n",
       " 'chesterton-brown.txt',\n",
       " 'chesterton-thursday.txt',\n",
       " 'edgeworth-parents.txt',\n",
       " 'melville-moby_dick.txt',\n",
       " 'milton-paradise.txt',\n",
       " 'shakespeare-caesar.txt',\n",
       " 'shakespeare-hamlet.txt',\n",
       " 'shakespeare-macbeth.txt',\n",
       " 'whitman-leaves.txt']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# It is cumbersome to write such long names \n",
    "# so Python offers another version of the import statement \n",
    "from nltk.corpus import gutenberg\n",
    "gutenberg.fileids()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "192427"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# shorter request \n",
    "emma = gutenberg.words('austen-emma.txt')\n",
    "len(emma)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8354"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# quicker to type\n",
    "blake = gutenberg.words('blake-poems.txt')\n",
    "len(blake)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "34110"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# better for examining more texts\n",
    "alice = gutenberg.words('carroll-alice.txt')\n",
    "len(alice)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5 25 26 austen-emma.txt\n",
      "5 26 17 austen-persuasion.txt\n",
      "5 28 22 austen-sense.txt\n",
      "4 34 79 bible-kjv.txt\n",
      "5 19 5 blake-poems.txt\n",
      "4 19 14 bryant-stories.txt\n",
      "4 18 12 burgess-busterbrown.txt\n",
      "4 20 13 carroll-alice.txt\n",
      "5 20 12 chesterton-ball.txt\n",
      "5 23 11 chesterton-brown.txt\n",
      "5 18 11 chesterton-thursday.txt\n",
      "4 21 25 edgeworth-parents.txt\n",
      "5 26 15 melville-moby_dick.txt\n",
      "5 52 11 milton-paradise.txt\n",
      "4 12 9 shakespeare-caesar.txt\n",
      "4 12 8 shakespeare-hamlet.txt\n",
      "4 12 7 shakespeare-macbeth.txt\n",
      "5 36 12 whitman-leaves.txt\n"
     ]
    }
   ],
   "source": [
    "# A short program to compute statistics\n",
    "# average word length\n",
    "# average sentence length\n",
    "# average times each word appears in text = lexical diversity \n",
    "\n",
    "for fileid in gutenberg.fileids():\n",
    "    num_chars = len(gutenberg.raw(fileid))\n",
    "    num_words = len(gutenberg.words(fileid))\n",
    "    num_sents = len(gutenberg.sents(fileid))\n",
    "    num_vocab = len(set(w.lower() for w in gutenberg.words(fileid))) \n",
    "    print(round(num_chars/num_words), round(num_words/num_sents), round(num_words/num_vocab), fileid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "145735"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(text4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['!',\n",
       " '\"',\n",
       " '\";',\n",
       " '\"?',\n",
       " '$',\n",
       " \"'\",\n",
       " '(',\n",
       " ')',\n",
       " '),',\n",
       " ',',\n",
       " ',\"',\n",
       " '-',\n",
       " '--',\n",
       " '.',\n",
       " '.\"',\n",
       " '.)',\n",
       " '...',\n",
       " '....',\n",
       " '.¡¨',\n",
       " '000',\n",
       " '1',\n",
       " '100',\n",
       " '120',\n",
       " '125',\n",
       " '13',\n",
       " '14th',\n",
       " '15th',\n",
       " '16',\n",
       " '1774',\n",
       " '1776',\n",
       " '1778',\n",
       " '1780',\n",
       " '1787',\n",
       " '1789',\n",
       " '1790',\n",
       " '1800',\n",
       " '1801',\n",
       " '1812',\n",
       " '1815',\n",
       " '1816',\n",
       " '1817',\n",
       " '1818',\n",
       " '1826',\n",
       " '1850',\n",
       " '1861',\n",
       " '1868',\n",
       " '1873',\n",
       " '1880',\n",
       " '1886',\n",
       " '1890',\n",
       " '1893',\n",
       " '1896',\n",
       " '1897',\n",
       " '1898',\n",
       " '1899',\n",
       " '18th',\n",
       " '1907',\n",
       " '1917',\n",
       " '1933',\n",
       " '1941',\n",
       " '1945',\n",
       " '1963',\n",
       " '1972',\n",
       " '1980',\n",
       " '1984',\n",
       " '19th',\n",
       " '2',\n",
       " '20',\n",
       " '200',\n",
       " '200th',\n",
       " '20th',\n",
       " '21st',\n",
       " '225',\n",
       " '25',\n",
       " '3',\n",
       " '30',\n",
       " '30th',\n",
       " '3d',\n",
       " '4',\n",
       " '41',\n",
       " '48',\n",
       " '4th',\n",
       " '5',\n",
       " '50',\n",
       " '50th',\n",
       " '6',\n",
       " '60',\n",
       " '67',\n",
       " '6th',\n",
       " '7',\n",
       " '8',\n",
       " '9',\n",
       " ':',\n",
       " ';',\n",
       " '?',\n",
       " '?\"',\n",
       " 'A',\n",
       " 'AIDS',\n",
       " 'ALL',\n",
       " 'AMENDMENT',\n",
       " 'AS',\n",
       " 'Abandonment',\n",
       " 'Abhorring',\n",
       " 'About',\n",
       " 'Above',\n",
       " 'Abraham',\n",
       " 'Abroad',\n",
       " 'Accept',\n",
       " 'Across',\n",
       " 'Act',\n",
       " 'Acting',\n",
       " 'Action',\n",
       " 'Actual',\n",
       " 'Adams',\n",
       " 'Additional',\n",
       " 'Address',\n",
       " 'Administered',\n",
       " 'Administration',\n",
       " 'Administrations',\n",
       " 'Advance',\n",
       " 'Advancing',\n",
       " 'Afghanistan',\n",
       " 'Africa',\n",
       " 'African',\n",
       " 'After',\n",
       " 'Again',\n",
       " 'Age',\n",
       " 'Agriculture',\n",
       " 'Alamo',\n",
       " 'All',\n",
       " 'Alleghanies',\n",
       " 'Almighty',\n",
       " 'Almost',\n",
       " 'Along',\n",
       " 'Already',\n",
       " 'Although',\n",
       " 'Always',\n",
       " 'Amelia',\n",
       " 'Amen',\n",
       " 'America',\n",
       " 'American',\n",
       " 'Americanism',\n",
       " 'Americans',\n",
       " 'Americas',\n",
       " 'Amid',\n",
       " 'Amidst',\n",
       " 'Among',\n",
       " 'Amongst',\n",
       " 'An',\n",
       " 'And',\n",
       " 'Andes',\n",
       " 'Annihilate',\n",
       " 'Another',\n",
       " 'Anthony',\n",
       " 'Anxiety',\n",
       " 'Anxious',\n",
       " 'Any',\n",
       " 'Anyone',\n",
       " 'Apollo',\n",
       " 'Appreciating',\n",
       " 'Apprehension',\n",
       " 'April',\n",
       " 'Arbitration',\n",
       " 'Archibald',\n",
       " 'Are',\n",
       " 'Argonne',\n",
       " 'Arlington',\n",
       " 'Armies',\n",
       " 'Army',\n",
       " 'Articles',\n",
       " 'As',\n",
       " 'Asia',\n",
       " 'Asiatic',\n",
       " 'Assessing',\n",
       " 'Assistance',\n",
       " 'Association',\n",
       " 'Assuring',\n",
       " 'At',\n",
       " 'Athens',\n",
       " 'Atlantic',\n",
       " 'Attention',\n",
       " 'Author',\n",
       " 'BUSINESS',\n",
       " 'BY',\n",
       " 'Bad',\n",
       " 'Baker',\n",
       " 'Baltimore',\n",
       " 'Barbary',\n",
       " 'Batavian',\n",
       " 'Beach',\n",
       " 'Because',\n",
       " 'Before',\n",
       " 'Being',\n",
       " 'Belief',\n",
       " 'Believing',\n",
       " 'Bell',\n",
       " 'Belleau',\n",
       " 'Beneath',\n",
       " 'Beneficent',\n",
       " 'Bernardin',\n",
       " 'Besides',\n",
       " 'Between',\n",
       " 'Beyond',\n",
       " 'Bible',\n",
       " 'Bicentennial',\n",
       " 'Bill',\n",
       " 'Blessed',\n",
       " 'Board',\n",
       " 'Bolivar',\n",
       " 'Boston',\n",
       " 'Both',\n",
       " 'Britain',\n",
       " 'Britannic',\n",
       " 'British',\n",
       " 'Brutus',\n",
       " 'Budapest',\n",
       " 'Bureau',\n",
       " 'Burger',\n",
       " 'Burma',\n",
       " 'Bush',\n",
       " 'Business',\n",
       " 'But',\n",
       " 'By',\n",
       " 'CONCLUSION',\n",
       " 'CONGRESS',\n",
       " 'COOPERATION',\n",
       " 'CRIMINAL',\n",
       " 'Cabinet',\n",
       " 'Caesar',\n",
       " 'California',\n",
       " 'Called',\n",
       " 'Calmness',\n",
       " 'Calvin',\n",
       " 'Camillus',\n",
       " 'Can',\n",
       " 'Canal',\n",
       " 'Cantons',\n",
       " 'Cape',\n",
       " 'Capital',\n",
       " 'Capitol',\n",
       " 'Cardinal',\n",
       " 'Care',\n",
       " 'Carolina',\n",
       " 'Carter',\n",
       " 'Cemetery',\n",
       " 'Central',\n",
       " 'Century',\n",
       " 'Certain',\n",
       " 'Chairman',\n",
       " 'Chamber',\n",
       " 'Change',\n",
       " 'Charta',\n",
       " 'Charter',\n",
       " 'Cheney',\n",
       " 'Chief',\n",
       " 'China',\n",
       " 'Chop',\n",
       " 'Chosen',\n",
       " 'Chosin',\n",
       " 'Christian',\n",
       " 'Christianity',\n",
       " 'Christians',\n",
       " 'Christmas',\n",
       " 'Christs',\n",
       " 'Church',\n",
       " 'Churchill',\n",
       " 'Cincinnati',\n",
       " 'Citizens',\n",
       " 'Civil',\n",
       " 'Civility',\n",
       " 'Claiming',\n",
       " 'Clay',\n",
       " 'Clearly',\n",
       " 'Clergy',\n",
       " 'Clinton',\n",
       " 'Closely',\n",
       " 'Coast',\n",
       " 'Cold',\n",
       " 'Coleman',\n",
       " 'Colonel',\n",
       " 'Columbia',\n",
       " 'Comfort',\n",
       " 'Commander',\n",
       " 'Commendable',\n",
       " 'Commerce',\n",
       " 'Commission',\n",
       " 'Commissioners',\n",
       " 'Common',\n",
       " 'Commons',\n",
       " 'Commonwealth',\n",
       " 'Communications',\n",
       " 'Communism',\n",
       " 'Communist',\n",
       " 'Communists',\n",
       " 'Compact',\n",
       " 'Compared',\n",
       " 'Conceived',\n",
       " 'Conceiving',\n",
       " 'Concord',\n",
       " 'Conditions',\n",
       " 'Confederacy',\n",
       " 'Confederation',\n",
       " 'Confidence',\n",
       " 'Confident',\n",
       " 'Confidently',\n",
       " 'Confusion',\n",
       " 'Congress',\n",
       " 'Congressional',\n",
       " 'Congressman',\n",
       " 'Connected',\n",
       " 'Conscious',\n",
       " 'Considering',\n",
       " 'Constantly',\n",
       " 'Constitution',\n",
       " 'Constitutional',\n",
       " 'Contemplating',\n",
       " 'Content',\n",
       " 'Continue',\n",
       " 'Convention',\n",
       " 'Convinced',\n",
       " 'Cook',\n",
       " 'Coolidge',\n",
       " 'Corporations',\n",
       " 'Cortes',\n",
       " 'Country',\n",
       " 'Countrymen',\n",
       " 'Courage',\n",
       " 'Court',\n",
       " 'Covenant',\n",
       " 'Creator',\n",
       " 'Crime',\n",
       " 'Croix',\n",
       " 'Cromwell',\n",
       " 'Cuba',\n",
       " 'Cuban',\n",
       " 'Cubans',\n",
       " 'Cures',\n",
       " 'Curtii',\n",
       " 'Dakota',\n",
       " 'Dakotas',\n",
       " 'Dangers',\n",
       " 'Dark',\n",
       " 'David',\n",
       " 'Day',\n",
       " 'December',\n",
       " 'Decii',\n",
       " 'Declaration',\n",
       " 'Deeply',\n",
       " 'Delano',\n",
       " 'Democracy',\n",
       " 'Democratic',\n",
       " 'Democrats',\n",
       " 'Department',\n",
       " 'Departments',\n",
       " 'Depression',\n",
       " 'Descending',\n",
       " 'Destructive',\n",
       " 'Did',\n",
       " 'Difficult',\n",
       " 'Dingley',\n",
       " 'Directly',\n",
       " 'Dirksen',\n",
       " 'Discord',\n",
       " 'Discouraging',\n",
       " 'Disease',\n",
       " 'Distinct',\n",
       " 'District',\n",
       " 'Distrust',\n",
       " 'Divided',\n",
       " 'Divine',\n",
       " 'Division',\n",
       " 'Do',\n",
       " 'Does',\n",
       " 'Doing',\n",
       " 'Dole',\n",
       " 'Domestic',\n",
       " 'Domingo',\n",
       " 'Dr',\n",
       " 'Dream',\n",
       " 'Due',\n",
       " 'Dulled',\n",
       " 'During',\n",
       " 'EDUCATION',\n",
       " 'EIGHTEENTH',\n",
       " 'ELECTION',\n",
       " 'ENFORCEMENT',\n",
       " 'Each',\n",
       " 'Earth',\n",
       " 'East',\n",
       " 'Economic',\n",
       " 'Economy',\n",
       " 'Egypt',\n",
       " 'Eight',\n",
       " 'Eisenhower',\n",
       " 'Elected',\n",
       " 'Emerson',\n",
       " 'Emperor',\n",
       " 'Employed',\n",
       " 'Encountering',\n",
       " 'Encouraging',\n",
       " 'Ending',\n",
       " 'Endowed',\n",
       " 'Engineers',\n",
       " 'England',\n",
       " 'English',\n",
       " 'Enough',\n",
       " 'Entering',\n",
       " 'Enterprises',\n",
       " 'Envy',\n",
       " 'Equal',\n",
       " 'Equality',\n",
       " 'Equally',\n",
       " 'Especially',\n",
       " 'Establishment',\n",
       " 'Europe',\n",
       " 'European',\n",
       " 'Eve',\n",
       " 'Even',\n",
       " 'Events',\n",
       " 'Eventually',\n",
       " 'Ever',\n",
       " 'Every',\n",
       " 'Everyone',\n",
       " 'Everywhere',\n",
       " 'Executive',\n",
       " 'Existing',\n",
       " 'Expenditure',\n",
       " 'Experience',\n",
       " 'Experiencing',\n",
       " 'FAILURE',\n",
       " 'FROM',\n",
       " 'Faced',\n",
       " 'Far',\n",
       " 'Farewell',\n",
       " 'Father',\n",
       " 'Fathers',\n",
       " 'February',\n",
       " 'Federal',\n",
       " 'Federalists',\n",
       " 'Fellow',\n",
       " 'Fifty',\n",
       " 'Finally',\n",
       " 'First',\n",
       " 'Florida',\n",
       " 'Floridas',\n",
       " 'Following',\n",
       " 'Fondly',\n",
       " 'For',\n",
       " 'Force',\n",
       " 'Forces',\n",
       " 'Foreign',\n",
       " 'Forge',\n",
       " 'Fort',\n",
       " 'Fortunate',\n",
       " 'Fortunately',\n",
       " 'Forty',\n",
       " 'Founders',\n",
       " 'Founding',\n",
       " 'Fountain',\n",
       " 'Four',\n",
       " 'Fourth',\n",
       " 'France',\n",
       " 'Franklin',\n",
       " 'Freed',\n",
       " 'Freedom',\n",
       " 'French',\n",
       " 'Friends',\n",
       " 'From',\n",
       " 'Front',\n",
       " 'Fruitful',\n",
       " 'Fully',\n",
       " 'G',\n",
       " 'GOVERNMENT',\n",
       " 'Gatun',\n",
       " 'Gaul',\n",
       " 'General',\n",
       " 'Genius',\n",
       " 'George',\n",
       " 'Germany',\n",
       " 'Gettysburg',\n",
       " 'Ghent',\n",
       " 'Gillis',\n",
       " 'Give',\n",
       " 'Giver',\n",
       " 'God',\n",
       " 'Godsend',\n",
       " 'Goethals',\n",
       " 'Golden',\n",
       " 'Good',\n",
       " 'Gore',\n",
       " 'Government',\n",
       " 'Governments',\n",
       " 'Grant',\n",
       " 'Grave',\n",
       " 'Great',\n",
       " 'Greater',\n",
       " 'Greatness',\n",
       " 'Greece',\n",
       " 'Gross',\n",
       " 'Growing',\n",
       " 'Guadalcanal',\n",
       " 'Guarantees',\n",
       " 'Guided',\n",
       " 'Gulf',\n",
       " 'HEALTH',\n",
       " 'Had',\n",
       " 'Hague',\n",
       " 'Half',\n",
       " 'Hall',\n",
       " 'Hand',\n",
       " 'Happily',\n",
       " 'Happiness',\n",
       " 'Happy',\n",
       " 'Hard',\n",
       " 'Has',\n",
       " 'Hatfield',\n",
       " 'Have',\n",
       " 'Having',\n",
       " 'He',\n",
       " 'Heads',\n",
       " 'Heaven',\n",
       " 'Heavenly',\n",
       " 'Help',\n",
       " 'Helvetic',\n",
       " 'Hemisphere',\n",
       " 'Hence',\n",
       " 'Here',\n",
       " 'Herein',\n",
       " 'High',\n",
       " 'Higher',\n",
       " 'Hill',\n",
       " 'Him',\n",
       " 'Hindus',\n",
       " 'His',\n",
       " 'History',\n",
       " 'Hitherto',\n",
       " 'Holy',\n",
       " 'Homes',\n",
       " 'Homestead',\n",
       " 'Honesty',\n",
       " 'Honorable',\n",
       " 'Honored',\n",
       " 'Honoring',\n",
       " 'Hope',\n",
       " 'Horn',\n",
       " 'House',\n",
       " 'Houses',\n",
       " 'How',\n",
       " 'However',\n",
       " 'Human',\n",
       " 'Humanity',\n",
       " 'Humphrey',\n",
       " 'Hundreds',\n",
       " 'I',\n",
       " 'II',\n",
       " 'INVESTIGATION',\n",
       " 'Idle',\n",
       " 'If',\n",
       " 'Ill',\n",
       " 'Illiteracy',\n",
       " 'Immunity',\n",
       " 'In',\n",
       " 'Inauguration',\n",
       " 'Inconsiderable',\n",
       " 'Indeed',\n",
       " 'Independence',\n",
       " 'Independently',\n",
       " 'Indian',\n",
       " 'Indians',\n",
       " 'Indies',\n",
       " 'Indo',\n",
       " 'Indulging',\n",
       " 'Industrial',\n",
       " 'Industry',\n",
       " 'Infinite',\n",
       " 'Information',\n",
       " 'Instead',\n",
       " 'Instinctively',\n",
       " 'Intelligence',\n",
       " 'Intentions',\n",
       " 'Internal',\n",
       " 'International',\n",
       " 'Internet',\n",
       " 'Interstate',\n",
       " 'Intrusted',\n",
       " 'Invasions',\n",
       " 'Investment',\n",
       " 'Invisible',\n",
       " 'Iowa',\n",
       " 'Iraq',\n",
       " 'Irritated',\n",
       " 'Is',\n",
       " 'Isaiah',\n",
       " 'Island',\n",
       " 'Islands',\n",
       " 'Isles',\n",
       " 'Israel',\n",
       " 'Isthmus',\n",
       " 'It',\n",
       " 'Italy',\n",
       " 'Its',\n",
       " 'Iwo',\n",
       " 'JUSTICE',\n",
       " 'Janeiro',\n",
       " 'January',\n",
       " 'Jealous',\n",
       " 'Jefferson',\n",
       " 'Jericho',\n",
       " 'Jews',\n",
       " 'Jima',\n",
       " 'John',\n",
       " 'Johnson',\n",
       " 'Joseph',\n",
       " 'Julia',\n",
       " 'July',\n",
       " 'Just',\n",
       " 'Justice',\n",
       " 'K',\n",
       " 'Kansas',\n",
       " 'Khe',\n",
       " 'Kindly',\n",
       " 'King',\n",
       " 'Knowing',\n",
       " 'Koran',\n",
       " 'Korea',\n",
       " 'Labor',\n",
       " 'Laboring',\n",
       " 'Lakes',\n",
       " 'Latent',\n",
       " 'Laws',\n",
       " 'Leader',\n",
       " 'League',\n",
       " 'Legislation',\n",
       " 'Legislature',\n",
       " 'Legitimate',\n",
       " 'Less',\n",
       " 'Let',\n",
       " 'Liberty',\n",
       " 'Like',\n",
       " 'Likewise',\n",
       " 'Limited',\n",
       " 'Lincoln',\n",
       " 'Lives',\n",
       " 'Loans',\n",
       " 'Local',\n",
       " 'Long',\n",
       " 'Look',\n",
       " 'Looking',\n",
       " 'Lord',\n",
       " 'Louisiana',\n",
       " 'Love',\n",
       " 'Loyalty',\n",
       " 'Luther',\n",
       " 'Lynchings',\n",
       " 'MANDATES',\n",
       " 'MY',\n",
       " 'MacLeish',\n",
       " 'Madison',\n",
       " 'Magistrate',\n",
       " 'Magna',\n",
       " 'Magnifying',\n",
       " 'Majesty',\n",
       " 'Majority',\n",
       " 'Make',\n",
       " 'Maker',\n",
       " 'Malaya',\n",
       " 'Mall',\n",
       " 'Manifestly',\n",
       " 'Mankind',\n",
       " 'Many',\n",
       " 'March',\n",
       " 'Mars',\n",
       " 'Martin',\n",
       " 'Massachusetts',\n",
       " 'Masses',\n",
       " 'Mathias',\n",
       " 'May',\n",
       " 'Mayflower',\n",
       " 'Meantime',\n",
       " 'Measures',\n",
       " 'Medicare',\n",
       " 'Mediterranean',\n",
       " 'Melancholy',\n",
       " 'Memorial',\n",
       " 'Men',\n",
       " 'Mental',\n",
       " 'Mexico',\n",
       " 'Micah',\n",
       " 'Michael',\n",
       " 'Middle',\n",
       " 'Mill',\n",
       " 'Mind',\n",
       " 'Mindful',\n",
       " 'Minorities',\n",
       " 'Mischievous',\n",
       " 'Misled',\n",
       " 'Miss',\n",
       " 'Mississippi',\n",
       " 'Missouri',\n",
       " 'Mitchell',\n",
       " 'Modern',\n",
       " 'Moment',\n",
       " 'Mondale',\n",
       " 'Monday',\n",
       " 'Monroe',\n",
       " 'Montana',\n",
       " 'Monticello',\n",
       " 'Moomaw',\n",
       " 'Moon',\n",
       " 'Moral',\n",
       " 'More',\n",
       " 'Moreover',\n",
       " 'Mormon',\n",
       " 'Moscow',\n",
       " 'Most',\n",
       " 'Mount',\n",
       " 'Mountains',\n",
       " 'Mr',\n",
       " 'Mrs',\n",
       " 'Much',\n",
       " 'Muslim',\n",
       " 'Muslims',\n",
       " 'Must',\n",
       " 'My',\n",
       " 'NATIONAL',\n",
       " 'Nation',\n",
       " 'National',\n",
       " 'Nations',\n",
       " 'Nature',\n",
       " 'Naval',\n",
       " 'Navy',\n",
       " 'Nearly',\n",
       " 'Nebraska',\n",
       " 'Negligence',\n",
       " 'Negro',\n",
       " 'Negroes',\n",
       " 'Neill',\n",
       " 'Neither',\n",
       " 'Never',\n",
       " 'New',\n",
       " 'Next',\n",
       " 'Nixon',\n",
       " 'No',\n",
       " 'None',\n",
       " 'Nor',\n",
       " 'Normandy',\n",
       " 'North',\n",
       " 'Northerners',\n",
       " 'Not',\n",
       " 'Nothing',\n",
       " 'Notwithstanding',\n",
       " 'November',\n",
       " 'Now',\n",
       " 'Nowhere',\n",
       " 'O',\n",
       " 'OF',\n",
       " 'OTHER',\n",
       " 'OUR',\n",
       " 'Occasionally',\n",
       " 'Ocean',\n",
       " 'Oceans',\n",
       " 'Octavius',\n",
       " 'October',\n",
       " 'Of',\n",
       " 'Off',\n",
       " 'Offices',\n",
       " 'Ohio',\n",
       " 'Old',\n",
       " 'Omaha',\n",
       " 'Omnipotence',\n",
       " 'On',\n",
       " 'Once',\n",
       " 'One',\n",
       " 'Only',\n",
       " 'Opportunism',\n",
       " 'Opportunity',\n",
       " 'Or',\n",
       " 'Order',\n",
       " 'Oregon',\n",
       " 'Orient',\n",
       " 'Orleans',\n",
       " 'Other',\n",
       " 'Others',\n",
       " 'Our',\n",
       " 'Ours',\n",
       " 'Out',\n",
       " 'Over',\n",
       " 'Overlooking',\n",
       " 'Overwhelmingly',\n",
       " 'PARTY',\n",
       " 'PEACE',\n",
       " 'PROGRESS',\n",
       " 'PUBLIC',\n",
       " 'Pacific',\n",
       " 'Page',\n",
       " 'Panama',\n",
       " 'Parchment',\n",
       " 'Parent',\n",
       " 'Parents',\n",
       " 'Paris',\n",
       " 'Parliament',\n",
       " 'Partial',\n",
       " 'Party',\n",
       " 'Passing',\n",
       " 'Passion',\n",
       " 'Patriotism',\n",
       " 'Patron',\n",
       " 'Peabody',\n",
       " 'Peace',\n",
       " 'Peking',\n",
       " 'Penetrating',\n",
       " 'Pennsylvania',\n",
       " 'People',\n",
       " 'Peoples',\n",
       " 'Perceiving',\n",
       " 'Perfection',\n",
       " 'Perhaps',\n",
       " 'Permanent',\n",
       " 'Perpetual',\n",
       " 'Perpetuity',\n",
       " 'Persian',\n",
       " 'Persistent',\n",
       " 'Personally',\n",
       " 'Petitions',\n",
       " 'Philippine',\n",
       " 'Philippines',\n",
       " 'Physically',\n",
       " 'Plainly',\n",
       " 'Pledge',\n",
       " 'Plenty',\n",
       " 'Political',\n",
       " 'Pool',\n",
       " 'Pork',\n",
       " 'Porto',\n",
       " 'Ports',\n",
       " 'Position',\n",
       " 'Possessing',\n",
       " 'Posterity',\n",
       " 'Potomac',\n",
       " 'Power',\n",
       " 'Powerful',\n",
       " 'Powers',\n",
       " 'Practices',\n",
       " 'Preeminently',\n",
       " 'Prejudice',\n",
       " 'Present',\n",
       " 'Presidency',\n",
       " 'President',\n",
       " 'Presidential',\n",
       " 'Presidents',\n",
       " 'Previous',\n",
       " 'Price',\n",
       " 'Prices',\n",
       " 'Primarily',\n",
       " 'Problems',\n",
       " 'Proceeding',\n",
       " 'Product',\n",
       " 'Profound',\n",
       " 'Progress',\n",
       " 'Proofs',\n",
       " 'Prophets',\n",
       " 'Prosperity',\n",
       " 'Protector',\n",
       " 'Providence',\n",
       " 'Province',\n",
       " 'Public',\n",
       " 'Purposes',\n",
       " 'Putting',\n",
       " 'Quayle',\n",
       " 'RELATION',\n",
       " 'RESPONSIBILITIES',\n",
       " 'Race',\n",
       " 'Rainbow',\n",
       " 'Raised',\n",
       " 'Rather',\n",
       " 'Reagan',\n",
       " 'Realizing',\n",
       " 'Reasoned',\n",
       " 'Recall',\n",
       " 'Recent',\n",
       " 'Recognition',\n",
       " 'Recognizing',\n",
       " 'Reconstruction',\n",
       " 'Red',\n",
       " 'Reflecting',\n",
       " 'Reform',\n",
       " 'Reforms',\n",
       " 'Relief',\n",
       " 'Relying',\n",
       " 'Removing',\n",
       " 'Renewed',\n",
       " 'Repeated',\n",
       " 'Representative',\n",
       " 'Representatives',\n",
       " 'Republic',\n",
       " 'Republican',\n",
       " 'Republicans',\n",
       " 'Republics',\n",
       " 'Reservoir',\n",
       " 'Resolved',\n",
       " 'Respecting',\n",
       " 'Resting',\n",
       " 'Restoration',\n",
       " 'Retrospect',\n",
       " 'Returning',\n",
       " 'Reverend',\n",
       " 'Revolution',\n",
       " 'Revolutionary',\n",
       " 'Rico',\n",
       " 'Rights',\n",
       " 'Rigid',\n",
       " 'Rio',\n",
       " 'River',\n",
       " 'Rocky',\n",
       " 'Roman',\n",
       " 'Rome',\n",
       " 'Roosevelt',\n",
       " 'Root',\n",
       " 'Rule',\n",
       " 'Ruler',\n",
       " 'Russia',\n",
       " 'SESSION',\n",
       " 'SPECIAL',\n",
       " 'SYSTEM',\n",
       " 'Sabine',\n",
       " 'Sacredly',\n",
       " 'Sahn',\n",
       " 'Salerno',\n",
       " 'Sanitary',\n",
       " 'Santo',\n",
       " 'Savior',\n",
       " 'Scandinavia',\n",
       " 'Science',\n",
       " 'Scientists',\n",
       " 'Scipios',\n",
       " 'Scripture',\n",
       " 'Scythia',\n",
       " 'Second',\n",
       " 'Secretary',\n",
       " 'Sectionalism',\n",
       " 'Security',\n",
       " 'Seeking',\n",
       " 'Self',\n",
       " 'Senate',\n",
       " 'Senator',\n",
       " 'Senators',\n",
       " 'September',\n",
       " 'Sermon',\n",
       " 'Service',\n",
       " 'Shall',\n",
       " 'She',\n",
       " 'Shortly',\n",
       " 'Should',\n",
       " 'Sinai',\n",
       " 'Since',\n",
       " 'Situated',\n",
       " 'Slowly',\n",
       " 'Small',\n",
       " 'So',\n",
       " 'Social',\n",
       " 'Societies',\n",
       " 'Society',\n",
       " 'Solemnly',\n",
       " 'Somalia',\n",
       " 'Some',\n",
       " 'Something',\n",
       " 'Sometimes',\n",
       " 'South',\n",
       " 'Southern',\n",
       " 'Sovereignty',\n",
       " 'Soviet',\n",
       " 'Soviets',\n",
       " 'Spain',\n",
       " 'Spanish',\n",
       " 'Speaker',\n",
       " 'Splendid',\n",
       " 'St',\n",
       " 'Stages',\n",
       " 'Standing',\n",
       " 'Stars',\n",
       " 'Start',\n",
       " 'Starting',\n",
       " 'State',\n",
       " 'States',\n",
       " 'Steadfast',\n",
       " 'Stennis',\n",
       " 'Steps',\n",
       " 'Still',\n",
       " 'Stripped',\n",
       " 'Strong',\n",
       " 'Subordinate',\n",
       " 'Such',\n",
       " 'Suicide',\n",
       " 'Suitably',\n",
       " 'Sumter',\n",
       " 'Superficial',\n",
       " 'Suppose',\n",
       " 'Supreme',\n",
       " 'Surely',\n",
       " 'THE',\n",
       " ...]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# sorted list of vocabulary items\n",
    "sorted(set(text4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9754"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# lexical richness\n",
    "len(set(text4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14.941049825712529"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# each word used 15 times on average - or each 15th word is unique?\n",
    "# Or each set of 15 words is unique? \n",
    "# Need to clarify how I interpret this.\n",
    "len(text4) / len(set(text4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.06692970116993173"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 6 percent? (I'm not sure how to read this as a percentage.)\n",
    "len(set(text4)) / len(text4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How often a word occurs in a text\n",
    "text4.count('Atlantic')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.004803238755274985"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what percentage of the text is taken up by a specific word\n",
    "100 * text4.count('Atlantic') / len(text4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "704"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How many times does lol occur in the Chat Corpus?\n",
    "text5.count('lol')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.5640968673628082"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Percentage of lol in total number of words\n",
    "100 * text5.count('lol') / len(text5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to repeat such calculations on several texts, it is tedious to keep retyping the formula. Instead, you can come up with your own name for a task, like \"lexical_diversity\" or \"percentage\", and associate it with a block of code. Now you only have to type a short name instead of one or more complete lines of Python code, and you can re-use it as often as you like. The block of code that does a task for us is called a **function**, and we define a short name for our function with the keyword `def`. The next example shows how to define two new functions, `lexical_diversity()` and `percentage()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# How to define two new functions\n",
    "# !Important - indentation\n",
    "# finish the indented block with a blank line\n",
    "\n",
    "def lexical_diversity(text):\n",
    "    return len(set(text)) / len(text)\n",
    "\n",
    "def percentage(count, total):\n",
    "    return 100 * count / total\n",
    "\n",
    "\n",
    "# In the definition of lexical_diversity() [1], \n",
    "# we specify a parameter named text . \n",
    "# This parameter is a \"placeholder\" for the actual text whose \n",
    "# lexical diversity we want to compute, and reoccurs in \n",
    "# the block of code that will run when the function is used [2]. \n",
    "# Similarly, percentage() is defined to take two parameters, \n",
    "# named count and total [3]."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once Python knows that lexical_diversity() and percentage() are the names for specific blocks of code, we can go ahead and use these functions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.06230453042623537"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lexical_diversity(text3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.13477005109975562"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lexical_diversity(text5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "80.0"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "percentage(4, 5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.4643016433938312"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "percentage(text4.count('a'), len(text4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Recap\n",
    "\n",
    "We use or call a function such as `lexical_diversity()` by typing its name, followed by an open parenthesis, the name of the text, and then a close parenthesis. These parentheses will show up often; their role is to separate the name of a task — such as `lexical_diversity()` — from the data that the task is to be performed on — such as `text3`. The data value that we place in the parentheses when we call a function is an **argument** to the function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}