02b_widgets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f961b01b-e8c5-4ecf-909b-982fe2a494d9",
   "metadata": {},
   "source": [
    "# Reviewing Jupyter Widgets: Building Widgets for our Data Dashboard"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "969eb6af",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import exception handling\n",
    "import dashboard"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29398c72",
   "metadata": {},
   "source": [
    "### Goal of this Notebook\n",
    "\n",
    "In this notebook we will be developing a `widgets.py` mofile which will allow use to build widgets for our data dashboard that build GUI controls for our [previous data workflow](./01b_workflow.ipynb). \n",
    "We will learn how to...\n",
    "\n",
    "### Steps You Will Take In This Notebook\n",
    "In this notebook you will:\n",
    "1. Briefly review ipywidgets.\n",
    "2. Load data from a file.\n",
    "3. Review the role of traits and how they can be used to create interactivity with widgets.\n",
    "4. Create an interactive slider widget to select and display selected range of years.  To do this we must:\n",
    "    - Use the slider value to update the `selected` data.\n",
    "    - Update the `selected` data *every time* the range is modified.\n",
    "    - Create a pandas dataframe representing the new time range.\n",
    "5. Create some widgets to control the smoothing/filtering of the temperature data.  To do this we must:\n",
    "    - Create a slider widget to select the window size (aka the number of adjacent data points) to use for the smoothed/filtered data.\n",
    "    - Create an integer input widget to select the degree of the polynomial fit for the smoothed/filtered data.\n",
    "    - Update the smoothed/filtered data in the pandas dataframe.\n",
    "6. Use output widgets to display both the raw data and a plot.  Both should be updated when year range or smoothing/filtering parameters are changed."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bbdd068",
   "metadata": {},
   "source": [
    "## Building the Widgets Module of our Package\n",
    "\n",
    "We will be developing the `widgets.py` module in this notebook, so we start by declaring that module to be our intended export target in `nbdev`. Notice that as we work through this notebook, we will export only those cells necessary to create the `widgets.py` module, leaving other cells which provide examples our testing or development process intact."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3026a02-fb27-4620-a6b0-3378e2d37f27",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| default_exp widgets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41ed38cf-fc0b-4ef5-8c9b-2619f4ab3058",
   "metadata": {},
   "source": [
    "## Preview the Final Product\n",
    "\n",
    "We will be converting our previous data workflow into a fully interactive web-based dashboard using Jupyter widgets.  The final product will look something like this:\n",
    "\n",
    "![Preview Image of Data Dashboard](static/dashboard_screenshot.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ad7eb7b-1a62-494d-82bb-bb457ebbf6b2",
   "metadata": {},
   "source": [
    "## Reviewing Jupyter Widgets\n",
    "\n",
    "Let's add an import statement for Jupyter widgets (aka `ipywidgets`). We will use a lot of widgets, so it's good to `import ipywidgets as widgets`, or something similarly short. \n",
    "\n",
    "**EXERCISE**: Add an import statement below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5c6ecde-5850-4d35-903e-6ed258669fe6",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# %answer key/dashboard/widgets.py 7\n",
    "\n",
    "import pandas as pd\n",
    "import os\n",
    "from matplotlib import pyplot as plt\n",
    "from scipy.signal import savgol_filter\n",
    "# add import statement for Jupyter widgets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "004d9171",
   "metadata": {},
   "source": [
    "The package `ipywidgets` provides two things:\n",
    "\n",
    "+ A communication framework between the front end (your browser) and back end (Python or other kernel).\n",
    "+ A set of fundamental user interface elements like buttons and checkboxes.\n",
    "\n",
    "Execute the next cell to create a browser of the available elements. To see some example code for implementing any of the elements click on its title. It will be easier to view both this overview and [the code examples](reference/complete-ipywidgets-widget-list.ipynb) if you have them open in separate tabs.\n",
    "\n",
    "Additional Resources:\n",
    "- [nicole-brewer/awesome-jupyter-widgets](/~https://github.com/nicole-brewer/awesome-jupyter-widgets) lists many packages taking advantage of Jupyter widgets\n",
    "- [Jupyter widgets documentation site](https://ipywidgets.readthedocs.io/en/latest/examples/Widget%20List.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6e7791e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from static.widget_org import organized_widgets, list_overview_widget\n",
    "\n",
    "groups = organized_widgets(organize_by='ui')\n",
    "help_url_base='static/complete-ipywidgets-widget-list.ipynb'\n",
    "list_overview_widget(groups, columns=2, min_width_single_widget=200, help_url_base=help_url_base)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12edac50-71aa-43c8-b94b-4bafc9d5c943",
   "metadata": {},
   "source": [
    "## Load Data from File\n",
    "\n",
    "Now we will create a cell to load the data into a pandas dataframe, just [like last time](./01b_workflow.ipynb).  We will export this since we will need this for our dashboard."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2754b1af-94db-4775-83fc-f1d178180fd0",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# Load data into memory from file\n",
    "DATA_DIR = 'data'\n",
    "DATA_FILE = 'land-ocean-temp-index.csv'\n",
    "\n",
    "original_df = pd.read_csv(os.path.join(DATA_DIR, DATA_FILE), escapechar='#')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7942663",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the original data\n",
    "original_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dfa4039-7f5d-4e8e-87e0-4a9b86f86c9e",
   "metadata": {},
   "source": [
    "## Our First Widget: `IntRangeSlider`\n",
    "\n",
    "We want to select the range of years of temperature data to display interactively via the dashboard.\n",
    "\n",
    "To do this, we will implement our first widget. Because we are selecting a range of years, we will use the `IntRangeSlider` widget, which has two click and drag nobs.\n",
    "\n",
    "Let's see what how to create the `IntRangeSlider` widget:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee19e21f-6159-4e5f-b44c-916c7a036361",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "year_range = widgets.IntRangeSlider(description = 'Range of Years',\n",
    "                                    style={'description_width': 'initial'})\n",
    "# The style argument is used to set the width of the description wide enough\n",
    "# to display the entire text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6a92889",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the widget\n",
    "year_range"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce69851a-0cc5-489c-967f-c46663e37350",
   "metadata": {},
   "source": [
    "Okay great! But 0 - 100 isn't the range we want. Instead, we would like the minimum and maximum to be the earliest and latest years in our dataset. So we need to obtain those first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1fa20d9-7431-4117-a9a9-5b66302605d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "(min(original_df['Year']), max(original_df['Year']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc9e4f88-ee00-4665-9ddf-3c16e71b9de3",
   "metadata": {},
   "source": [
    "### Setting Slider Limits"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0464be4a-4d35-4089-8047-d97b87eb7c77",
   "metadata": {},
   "source": [
    "Widgets, like other Python objects, have attributes. Recall that to set the value of an attribute of an object, we use 'dot notation' (e.g. `box.width `sets the `width` attribute of object `box`).\n",
    "\n",
    "**EXERCISE**: Set the `min` and `max` attributes of the slider widget.  Then look back at the `year_range` widget above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60755adb-0aef-488a-8b6c-fdbe21b6af5e",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# %answer key/dashboard/widgets.py 20\n",
    "\n",
    "# set the 'max' attribute of the slider to the maximum year of the our data\n",
    "# and let's do the same for 'min'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a838e4c9-0ab3-4bd5-abe7-02215b6a4488",
   "metadata": {},
   "source": [
    "Now go up and look at the slider widget...\n",
    "\n",
    "Okay that looks a lot better, but how to we actually get the value of the range from the widget? Generally, most widgets have a `value` attribute, but since there are actually two values in a range, I'm not sure what to expect. Let's see what happens when we try and poll the value attribute. What data type is it?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76e46257-deee-4736-857b-46e10dbe0946",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range.value"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74c336fe-ece1-424c-a712-458aff545fe0",
   "metadata": {},
   "source": [
    "That looks like a tuple. Let's try setting the value programmatically as well and look back at the slider widget."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50aa5155-63a7-4b4a-9e2c-25011324aba3",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range.value = (1900, 2000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "899b89cd-ebc3-4d1d-afd3-104a006b7318",
   "metadata": {},
   "source": [
    "Okay, now that we can get the range from the slider, how do we relect those selections back to the dataframe? \n",
    "\n",
    "## Use the slider value to update the `selected_df` data\n",
    "Well, we probably want to keep two copies, because if we change the original to have a shorter range, and then we want to expand that range, we're going to have lost data. So let's maintain a dataframe called `selected_df` that reflects the range specified by the Slider. Let's look at how we did that in workflow.ipynb. \n",
    "\n",
    "```python\n",
    "selected_df = original_df[(original_df['Year'] >= from_year) & (original_df['Year'] <= to_year)]\n",
    "```\n",
    "The only difference is this time, we need to replace \"from_year\" and \"to_year\" with the value of our slider.\n",
    "\n",
    "**EXERCISE**:  Update the `selected_df` dataframe to reflect the range specified by the slider.  You can test if your solution works by executing the next cell and then running the cell below it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2af0625b-1774-4895-a335-7522c58bf960",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# %answer key/dashboard/widgets.py 26\n",
    "\n",
    "# selected_df = original_df[(original_df['Year'] >= from_year)\n",
    "#                           & (original_df['Year'] <= to_year)]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9384c1d5",
   "metadata": {},
   "source": [
    "Display the `selected_df` dataframe to make sure it worked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7542d02c-ed8b-409b-a5f7-fddac5b42e06",
   "metadata": {},
   "outputs": [],
   "source": [
    "selected_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0e99fba-395f-4dc8-80b2-de3d120578da",
   "metadata": {},
   "source": [
    "Okay, that's a start. But now we want to do this dynamically. That is, we want the line above to be called every time the range is modified. \n",
    "\n",
    "## Update the `selected_df` data *every time* the range is modified\n",
    "\n",
    "How do we track changes to the value of the slider?\n",
    "\n",
    "### Observing trait changes\n",
    "\n",
    "Widgets are built on the `traitlets` library, which we will learn more about later on. Attributes of objects that inherit the `HasTraits` class, such as widgets do, are called **traits**.  Jupyter widgets have an `observe` method that lets you register a handler function that is called when a trait changes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e4cc6a5-ac6a-488c-8dce-43387e99e032",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a function that will print the change event\n",
    "def on_range_change(change):\n",
    "    print(change)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec226266",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Attach the above 'handler' function to the widget so that it gets called whenever a\n",
    "# trait of the widget changes.\n",
    "year_range.observe(on_range_change) # Warning: This is actually a poor practice, see below..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82011a05-8e12-4150-a0f0-fa90fe1fd7bd",
   "metadata": {},
   "source": [
    "So if we change the value of the slider, our handler function (`on_range_change`) will print the variable `change`. Apparently the `observe` method of a widget passes some variable to the handler function (the variable we are assigning the name `change`).\n",
    "\n",
    "But what is contained in `change`? Let's find out by changing the value of the slider programmatically and thus triggering the handler function.."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "661de20a-3160-43d9-8b25-5835e8f2e90e",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range.value = (1880, 2002)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4ee768e-6d90-4444-a325-1f4276fda768",
   "metadata": {},
   "source": [
    "It's clear that `change` is a dictionary. There are a lot of key/value pairs here, but the most important ones for us right now are the `old` and `new` keys. We can use their associated values to see what the widget's traits used to be, and what they were updated to."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8263e67d-d239-4477-a1a9-4e5c68a5dd2f",
   "metadata": {},
   "source": [
    "## Update `selected_df` dataframe with new range\n",
    "Now that we know how to observe changes in the slider value, we want to use that `observe` method to update the `selected_df` dataframe. Let's rewrite the `on_range_change` function so that it updates the `selected_df` dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "979cf0b4-3164-49db-bc7d-f736ff1ea522",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "\n",
    "# Create a function that will change the selected_df based on the range of\n",
    "# years selected by the user using the year_range widget\n",
    "def on_range_change(change):\n",
    "    global selected_df\n",
    "    selected_df = original_df[(original_df['Year'] >= change['new'][0])\n",
    "                              & (original_df['Year'] <= change['new'][1])]\n",
    "    # WARNING: You could have thought to use:\n",
    "    #  selected_df = original_df[(original_df['Year'] >= year_range.value[0])\n",
    "    #                             & (original_df['Year'] <= year_range.value[1])]\n",
    "    # but for many manual interactions, such as a click and drag slider, a high\n",
    "    # overturn of quickly changing widget values could silently cause\n",
    "    # unexpected behavior in changing values, so that syntax should be avoided"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "355286a4",
   "metadata": {},
   "source": [
    "And now let's test it by directing the `observe` method to call this new `on_range_change` every time the value of the slider changes. \n",
    "\n",
    "While we do this, let's address a poor Jupyter widget programming practice we showed earlier.  There is an optional second argument to the `observe` method which is the name of the trait to watch for changes.  We didn't include it... this is not a good choice.\n",
    "\n",
    "While the name of the trait to watch is technically optional, you really should always include it.  Why?  It has a default value of 'All' which watches all traits. This includes traits that start with an underscore (i.e. traits meant by the programmer to be private).  Furthermore, if too many changes occur at once, there is a chance your function will be called on a change to one of these internal traits - such as '_property_lock' - and cause your callback to fail or even silently cause unexpected behavior.\n",
    "\n",
    "So a good Jupyter widgets programmer should always specify the trait we are observing for changes, which is normally 'value'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dba5dddf-8b7e-4663-a500-34fdf1ec4d76",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Connect the new version of the handler function to the widget via observe\n",
    "# method, this time providing the name of the trait to watch.\n",
    "year_range.observe(on_range_change, 'value')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f6be451",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Programmatically set the value of the widget\n",
    "year_range.value = (1990, 1993)\n",
    "\n",
    "# Confirm the selected dataframe changed\n",
    "selected_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dffd6b33-466c-4759-ac30-c8abd7a5df24",
   "metadata": {},
   "source": [
    "That worked really well! But we are still printing the value of `change`. I thought we redefined the function!\n",
    "\n",
    "As it turns out, both versions of the function have copies that are called when there is a change in `year_range`.  This is an illustration that we can attach MULTIPLE handler functions to a single widget.  That said, it is a bit confusing to have two functions with the same name attached to this widget.\n",
    "\n",
    "**Note**: This happened because functions are immutable objects in Python, so when we redefined the function, we were actually creating a new function object and assigning it to the same name.  The old function object was still attached to the widget."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83c0238e-b1f1-42af-be97-64f10f3734d0",
   "metadata": {},
   "source": [
    "### Unobserve trait changes\n",
    "\n",
    "To get rid of all those observe functions, we use the `unobserve_all` method. Go ahead and try that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62cfcbd0-9c9c-446f-810a-e32031f463c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range.unobserve_all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f47a2ee8-d763-41ea-a6f7-bec2e6a9ef80",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%exception\n",
    "\n",
    "year_range.observe()\n",
    "year_range.value = (1987, 1991)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0be7ffd7-09d9-4569-91e3-859ca13eee93",
   "metadata": {},
   "source": [
    "**EXERCISE**: Notice the previous cell returned an exception. Sort of a weird one. We are missing something on our `observe` call. Can you think of what it is? Go ahead and make the changes needed below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "928b3283-f1bf-4ce0-9697-107cc0ea064a",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range.unobserve_all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75819dae-4645-4273-8e8c-23c6a3855794",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# %answer key/dashboard/widgets.py 46\n",
    "\n",
    "# year_range.observe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3dee888-fa47-4965-b8c6-5bd40874c247",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Adjust the range of years and confirm the selected dataframe changed\n",
    "year_range.value = (1900, 2000)\n",
    "\n",
    "selected_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f82900ec-70fa-4d56-91f0-f8f100ceffe1",
   "metadata": {},
   "source": [
    "Does it run like it's supposed to now? Great! Now that we have that down, the next step is to update the dataframe display every time the range is changed. But how do we achieve that? The dataframe isn't actually a widget yet. It's just being displayed with ipython. But we can turn anything that can be displayed with ipython into a widget with Output."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95a21862-c3e2-47fa-a435-1cc096728815",
   "metadata": {},
   "source": [
    "## Output widgets\n",
    "\n",
    "First, let's see what output looks like on it's own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "100c959a-101c-4d89-9545-3f877d74d0c2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create an Output widget to display the selected data\n",
    "selected_dataframe_view = widgets.Output()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "baa328f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the Output widget\n",
    "selected_dataframe_view"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "590c3e5f-2b5e-4897-bebd-9a98eb2cf328",
   "metadata": {},
   "source": [
    "Pretty boring. **Let's right click on the Output widget and \"Create new view for output\"** for now. How do we get it to display our dataframe? We use the Output widget as a context manager. Anything that we can display with ipython.display can be displayed by the Output widget using the following syntax."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdd92318-fc39-4a42-8ebb-4255612613fc",
   "metadata": {},
   "outputs": [],
   "source": [
    "selected_dataframe_view.clear_output()\n",
    "with selected_dataframe_view:\n",
    "    display(original_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a21b7a1-ad8e-41c3-9201-b2fe30103f15",
   "metadata": {},
   "source": [
    "Voila! The `selected_dataframe_view` widget now displays the original dataframe (notice it contains years 1880 to 2023). \n",
    "\n",
    "Now we need to be able to update that display when someone changes the year range (i.e. displaying the `selected_df` dataframe which it changes in response to a `year_range` change). Let's write a handler function for this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e2660a0-2cad-4033-8e65-16483f6ae543",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a handler function that will display the selected dataframe\n",
    "def output_selected_dataframe(change):\n",
    "    selected_dataframe_view.clear_output(wait=True)\n",
    "    with selected_dataframe_view:\n",
    "        display(selected_df)\n",
    "\n",
    "# Call the function (since called outside observe method, use None as argument)\n",
    "output_selected_dataframe(None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f2cc082-3d7f-4c2f-9efd-2537d39a851a",
   "metadata": {},
   "source": [
    "Cool! Now the only thing that is missing is that we want to call this function every time the range is updated. How might we go about that?\n",
    "\n",
    "Let's connect the `output_selected_dataframe` function to the `year_range` widget with the `observe` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f83030d-648e-4ad3-99bb-8835fa07c2fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range.observe(output_selected_dataframe, 'value')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c47a2de8-6193-40bb-8f26-f33107b5099c",
   "metadata": {},
   "source": [
    "Okay let's create another copy of the slider so we can change the value. Then go ahead and move the range around and see if you notice changes to the displayed dataframe above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7647b5d-b4d4-46a9-b1a3-8bd626a1a4d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7510f842",
   "metadata": {},
   "source": [
    "Way cool. Now our dataframe reflects the range of values."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eab31b81-150d-4065-add3-e2460bfc63b1",
   "metadata": {},
   "source": [
    "### A Better Way to Display Dataframes\n",
    "\n",
    "Notice that the size of the displayed pandas dataframe changes because default display of a dataframe is to only show up to 30 rows. This means our Output widget changes size when we change the range of years in the selected data.\n",
    "\n",
    "Another approach is rather than using the generic output widget to instead use one that is specifically designed for pandas dataframes. It's called [ipydatagrid](/~https://github.com/bloomberg/ipydatagrid). Let's import it and see how it works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae5f9cca",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "from ipydatagrid import DataGrid\n",
    "\n",
    "# Define a DataGrid widget containing our Pandas dataframe setting column and row sizes\n",
    "selected_data_grid = DataGrid(selected_df, header_visibility=\"column\", auto_fit_columns=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45723cad",
   "metadata": {},
   "source": [
    "And let's view our selected dataframe in this widget"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8387bea9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the DataGrid widget here\n",
    "selected_data_grid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fd291e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a handler function that will display the selected dataframe\n",
    "def output_selected_dataframe(change):\n",
    "    selected_dataframe_view.clear_output(wait=True)\n",
    "    with selected_dataframe_view:\n",
    "        display(selected_data_grid)\n",
    "\n",
    "# Call the function (since called outside observe method, use None as argument)\n",
    "output_selected_dataframe(None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10353fdb",
   "metadata": {},
   "source": [
    "Notice that the `DataGrid` widget includes some nice features such as auto-fitting the column and row size and choosing to only show the column headers (which hides the 'index' at the beginning of each row of a `pandas` dataframe).  We can also resort data in the visual display of the table via drop downs. But its biggest advantage is that it doesn't change size when the number of rows changes, instead a scrollbar appears or disappears as needed.\n",
    "\n",
    "It is a widget which means we can define a function to update the data displayed in the widget.  Let's try that below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b49f22c",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# Create a handler function that will display the selected dataframe\n",
    "def update_selected_datagrid(change):\n",
    "    selected_data_grid.data = selected_df\n",
    "\n",
    "# Attach the handler function to the year_range widget\n",
    "year_range.observe(update_selected_datagrid, 'value')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5114cd0",
   "metadata": {},
   "source": [
    "Let's create yet another copy of the `year_range` slider and see how changing it affects the datagrid widget we created above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e59a7857",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c7e7630-f949-4003-81f6-70b94e5108c2",
   "metadata": {},
   "source": [
    "## Add column for Smoothed Data\n",
    "\n",
    "Now we have data selection down, so the next step is to also add a column to our data for the smoothed data curve. Rather than plotting two different smoothed curves as we did in [our original data workflow](./01b_workflow.ipynb), we will only plot one, but we will allow the user to change the degree of the polynomial fit and the window size of the filter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8da0f8a4-dfb2-4f0e-b674-bcb1d9a70f0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the window size and polynomial order for the Savitzky-Golay filter\n",
    "window_size_int = 20\n",
    "poly_order_int = 3\n",
    "\n",
    "# Apply Savitzky-Golay smoothing to the Temperature column\n",
    "original_df['Smoothed Data'] = savgol_filter(original_df['Temperature'],\n",
    "                                             window_size_int, \n",
    "                                             poly_order_int).round(decimals=3)\n",
    "\n",
    "# Display the pandas DataFrame with the new column\n",
    "original_df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2437578e-7166-4ce9-8f51-a5d6769b048b",
   "metadata": {},
   "source": [
    "Okay. So we have these two variables that we want to control via widgets. We want to use a slider for the window size and an integer input for the degree of the polynomial fit. Let's start with the slider."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f65341b-948f-4c60-8bcd-cd45083610b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "window_size = widgets.IntSlider(description = 'Window Size', value=20, min=2, max=100,\n",
    "                                style={'description_width': 'initial'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1789aa10-fdbf-4ed1-8f05-f826ba30ef01",
   "metadata": {},
   "outputs": [],
   "source": [
    "window_size"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88fcb6f8",
   "metadata": {},
   "source": [
    "For the Polynomial order, we will use an integer text field input that is bounded between 1 and 10."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2a89387-5334-4b89-a227-e32ad25290b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "poly_order = widgets.BoundedIntText(description = 'Poly Order', min=1, value=3, max=10,\n",
    "                                    style={'description_width': 'initial'})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbf95d58-0045-4272-9f9f-56430779ff41",
   "metadata": {},
   "outputs": [],
   "source": [
    "poly_order"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7f1fe98-93c2-4ad2-b69c-6672ff86b4cf",
   "metadata": {},
   "source": [
    "Now when we update window size, we want a few things to happen.\n",
    "\n",
    "1. Update `original_df` with the new smoothed data column.  Remember to use windowsize and polynomial degree from the widgets.\n",
    "2. Use `original_df` to update selected\n",
    "3. Update Datagrid display of `selected_df`\n",
    "\n",
    "**EXERCISE**: Modify the function in the next cell so that it takes care of the first two items listed above.  Replace the comments in the `on_window_size_change` function in the cell below with the correct lines to accomplish those functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b7c924b-7e99-488c-b85d-d531e23702be",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %answer key/02b/01.py\n",
    "\n",
    "def on_window_size_change(change):\n",
    "    global original_df, selected_df # Remember to access the global variables\n",
    "    # Update the following call to update the original_df['Smoothed Data'] column\n",
    "    # values to properly capture the change in window_size value and the\n",
    "    # current poly_order value.\n",
    "    original_df['Smoothed Data'] = savgol_filter(original_df['Temperature'],\n",
    "                                                 window_size,\n",
    "                                                 poly_order).round(decimals=3)\n",
    "    selected_df = original_df[(original_df['Year'] >= year_range.value[0])\n",
    "                              & (original_df['Year'] <= year_range.value[1])]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b09b2f2-4e82-46b1-b302-ef6e64acd595",
   "metadata": {},
   "source": [
    "Then do you recall what function we used to display the dataframe? It was `update_selected_datagrid`. \n",
    "\n",
    "Let's update add both of these handler functions (`on_window_size_change` and `update_selected_datagrid`) as a handler function for the `observe` method of `window_size`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c27b144f-733d-4b59-9a9a-f8a883663e9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "window_size.observe(on_window_size_change, 'value')\n",
    "window_size.observe(update_selected_datagrid, 'value')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dbd2b136",
   "metadata": {},
   "outputs": [],
   "source": [
    "selected_data_grid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bdb93578-bced-43ca-8e27-35f52603e5a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "window_size"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "677f5e26-fe20-4be6-9f66-80954604049c",
   "metadata": {},
   "source": [
    "Sweet! If you increase the value of `window_size` you will see that there is now a \"Smoothed Data\"\" column.\n",
    "\n",
    "**EXERCISE**: Now create a Slider widget that is linked to the `poly_order`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5621bc33-53da-446b-b90c-ee1cc426e38f",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# %answer key/dashboard/widgets.py 85\n",
    "\n",
    "def on_poly_order_change(change):\n",
    "    global original_df, selected_df\n",
    "    # Change the following line that sets the 'Smoothed Data' column of original_df\n",
    "    # to catch the change in poly_order widget value and update the original_df for it.\n",
    "    # Remember to also catch the current window_size value.\n",
    "    original_df['Smoothed Data'] = savgol_filter(original_df['Temperature'],\n",
    "                                                 window_size,\n",
    "                                                 poly_order).round(decimals=3)\n",
    "    selected_df = original_df[(original_df['Year'] >= year_range.value[0])\n",
    "                              & (original_df['Year'] <= year_range.value[1])]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9367c9d-4235-446d-814a-73c405bda4ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "poly_order.observe(on_poly_order_change, 'value')\n",
    "poly_order.observe(update_selected_datagrid, 'value')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d715861b-8f24-40e5-b699-33404dbadfa5",
   "metadata": {},
   "outputs": [],
   "source": [
    "poly_order"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01271499-69da-4b2c-8fda-3fdb3f7d10cc",
   "metadata": {},
   "source": [
    "That looks pretty good! Let's try setting both values programmatically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "00ec288a-d269-4a48-aef6-34c943d03988",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%exception\n",
    "\n",
    "window_size.value = 3\n",
    "poly_order.value = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "675567a7",
   "metadata": {},
   "source": [
    "Oh oh! Why did this happen? Fortunately the error message is pretty clear. We need to ensure that our `poly_order` Slider doesn't allow a user to select a value greater than `window_size - 1`. But we also don't want to let `poly_order` be greater than 10 to avoid poor conditioning. Where should we put this validation? \n",
    "\n",
    "**EXERCISE**: Update the \"max\" trait of `poly_order` every time `window_size` is updated. So we need to add a line to add a line in the `on_window_size_change` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "685730e5-9984-4205-bc77-5995548de196",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# %answer key/dashboard/widgets.py 91\n",
    "\n",
    "def on_window_size_change(change):\n",
    "    global original_data, selected_df, poly_order\n",
    "    # change the maximum of the poly_order widget\n",
    "    # catch the change in the window_size widget value and update the original_df\n",
    "    original_df['Smoothed Data'] = savgol_filter(original_df['Temperature'],\n",
    "                                                 change['new'],\n",
    "                                                 poly_order.value).round(decimals=3)\n",
    "    selected_df = original_df[(original_df['Year'] >= year_range.value[0])\n",
    "                              & (original_df['Year'] <= year_range.value[1])]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "714f9f17-96af-4540-a8ea-b7a18f03dd8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "window_size.unobserve_all()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c9fc4c77-d75e-453d-8b4c-ef87acf78e33",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "window_size.observe(on_window_size_change, 'value')\n",
    "window_size.observe(update_selected_datagrid, 'value')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f2bffeb5-28d7-499c-88e2-ae3edf973536",
   "metadata": {},
   "outputs": [],
   "source": [
    "window_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a307d16e-db12-4560-aff8-6bb854f6b2e9",
   "metadata": {},
   "outputs": [],
   "source": [
    "poly_order"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c029a825",
   "metadata": {},
   "source": [
    "Export default values that will be used to perform a running average for the smoothing initially."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f4c27ce-e686-4dc2-8241-533293bcf83b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "window_size.value = 10\n",
    "poly_order.value = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "879447cf-3876-4bda-9b3d-325d732af697",
   "metadata": {},
   "source": [
    "Amazing! We are done visualizing our selected data. Now we can move on to visualizing our data with a plot.\n",
    "\n",
    "## Visualizing Data\n",
    "\n",
    "We will use [matplotlib](https://matplotlib.org), because the syntax is so common, but there are more interactive widgets in the ipywidget ecosystem that would work even better, such as [bqplot](https://bqplot.github.io/bqplot/).\n",
    "\n",
    "Before we get started, let's take a look at what we had in the original workflow.\n",
    "\n",
    "```python\n",
    "plt.xlabel('Year')\n",
    "plt.ylabel('Temperature Anomalies over Land w.r.t. 1951-80 (˚C)')\n",
    "plt.title('Global Annual Mean Surface Air Temperature Change')\n",
    "plt.plot(subset_df['Year'], subset_df['Temperature'], label='Raw Data')\n",
    "plt.plot(subset_df['Year'], subset_df[SG_col], label='Smoothed Data')\n",
    "plt.legend()\n",
    "plt.show()\n",
    "```\n",
    "\n",
    "**EXERCISE** We are going to want to turn set of commands into a handler function that accepts a \"change\" parameter and draws the plot. Is there anything we need to fix here to make this work? It's very minor. Replace the commented code in the cell below with functional code so that it works in our current notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd93c412-8834-46f9-9986-6c3a78edc0b2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %answer key/02b/02.py\n",
    "\n",
    "plt.xlabel('Year')\n",
    "plt.ylabel('Temperature Anomalies over Land w.r.t. 1951-80 (˚C)')\n",
    "plt.title('Global Annual Mean Surface Air Temperature Change')\n",
    "# plt.plot(subset_df['Year'], subset_df['Temperature'], label='Raw Data')\n",
    "# plt.plot(subset_df['Year'], subset_df[SG_col], label='Smoothed Data')\n",
    "plt.legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ced1b75-94e3-462a-b4fd-8697652f63aa",
   "metadata": {},
   "source": [
    "Okay that works, but the plot is showing up inline in our notebook. Once created, it can't be modified except by changing the data and re-executing the cell.  It sure would be nice if we could have the plot automatically update when the data year range or smoothing parameters are changed.  How do we do that?\n",
    "\n",
    "Yeah, we can use an Output widget again, just like we did with the dataframe.\n",
    "\n",
    "**EXERCISE** Create a `plot_view` output widget (you don't need to display it as it will be initially empty, although you certainly can if you want to test it)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fa36f751-4904-427e-9ead-1a2471486d0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# %answer key/dashboard/widgets.py 101\n",
    "\n",
    "# create an output widget called plot_view"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "53609542-9daf-473e-99c9-381eb9d5a864",
   "metadata": {},
   "source": [
    "If we call the `plot_view` Output widget, it would currently be blank.  Let's fix this by creating a function that will update the `plot_view` Output widget display. Recall that to update a Output widget (it was `selected_dataframe_view`) we created a function that would update it by calling `display` with the content to show in that Output widget:\n",
    "```python\n",
    "def output_selected_dataframe(change):\n",
    "    selected_dataframe_view.clear_output(wait=True)\n",
    "    with selected_dataframe_view:\n",
    "        display(selected_df)\n",
    "```\n",
    "\n",
    "**EXERCISE** Go ahead and adapt this approach in the cell below so that instead of displaying the selected dataframe, the function displays an updated plot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "868bcb6f-9fe3-4370-9a49-6754298a6e4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "# %answer key/dashboard/widgets.py 103\n",
    "\n",
    "def output_plot(change):\n",
    "    plot_view.clear_output(wait=True)\n",
    "    # Add with content handler for `plot_view`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bed98f1f-0220-417e-ab19-992e567b49f8",
   "metadata": {},
   "source": [
    "Let's make sure our new callback works when we call it explicitly. If we display `plot_view` again, we shouldn't see any output initially."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da7f6b7b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display plot_view again\n",
    "plot_view"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5f9a436-72d1-4bb6-8334-86cb0eb8f0dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Call the output_plot function to test output_plot function works.\n",
    "# Use None as argument (since this function is expecting a change variable as argument).\n",
    "output_plot(None)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22829fff-2eec-4a1f-b973-ab5d869dc341",
   "metadata": {},
   "source": [
    "Now let's make sure this function gets called any time there is a change to our selected data or to the Savitzky-Golay parameters. How do we make sure this function runs every time there is a change?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06bbdd3b-8478-424c-89bd-9da323b5f50d",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "year_range.observe(output_plot, 'value')\n",
    "window_size.observe(output_plot, 'value')\n",
    "poly_order.observe(output_plot, 'value')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4903751a",
   "metadata": {},
   "source": [
    "Let's set the initial year range to use for selected data to be 1900-2000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cfa742bc-0ee9-46f2-a833-26a66c6d498a",
   "metadata": {},
   "outputs": [],
   "source": [
    "#| export\n",
    "year_range.value = (1900, 2000)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0006b7bf",
   "metadata": {},
   "source": [
    "Now let's create copies of the sliders and see if the above plot responds to changes in the sliders."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d89ae5e5-3a9d-4dcc-80c3-27442b88ed3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "year_range"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad3f37b5-fb34-4edf-8964-a5fc48cd0ce1",
   "metadata": {},
   "outputs": [],
   "source": [
    "window_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc2dd5c3-02c4-4d5c-89d9-ab7752083600",
   "metadata": {},
   "outputs": [],
   "source": [
    "poly_order"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe02ff42-8ade-45ba-bf3f-b22dba762138",
   "metadata": {},
   "source": [
    "Wow that looks amazing! We have a total of five widgets. Three of them control various parameters and they have dependencies on each other. In the next notebook, we will see how this can all come together into one very beautiful interface."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e91e724-9497-4571-ab90-9b1652cbd868",
   "metadata": {},
   "outputs": [],
   "source": [
    "from nbdev.export import nb_export\n",
    "nb_export('02b_widgets.ipynb', 'dashboard')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}