05 - Train Models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train Models\n",
    "\n",
    "The central goal of machine learning is to train predictive models that can be used by applications. In Azure Machine Learning,  you can use scripts to train models leveraging common machine learning frameworks like Scikit-Learn, Tensorflow, PyTorch, SparkML, and others. You can run these training scripts as experiments in order to track metrics and outputs, which include the trained models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to your workspace\n",
    "\n",
    "To get started, connect to your workspace.\n",
    "\n",
    "> **Note**: If you haven't already established an authenticated session with your Azure subscription, you'll be prompted to authenticate by clicking a link, entering an authentication code, and signing into Azure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace\n",
    "\n",
    "# Load the workspace from the saved config file\n",
    "ws = Workspace.from_config()\n",
    "print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a training script\n",
    "\n",
    "You're going to use a Python script to train a machine learning model based on the diabates data, so let's start by creating a folder for the script and data files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, shutil\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "training_folder = 'diabetes-training'\n",
    "os.makedirs(training_folder, exist_ok=True)\n",
    "\n",
    "# Copy the data file into the experiment folder\n",
    "shutil.copy('data/diabetes.csv', os.path.join(training_folder, \"diabetes.csv\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you're ready to create the training script and save it in the folder.\n",
    "\n",
    "> **Note**: This code *creates* the script - it doesn't run it!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $training_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "import os\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# load the diabetes dataset\n",
    "print(\"Loading Data...\")\n",
    "diabetes = pd.read_csv('diabetes.csv')\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Set regularization hyperparameter\n",
    "reg = 0.01\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "# Save the trained model in the outputs folder\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the training script as an experiment\n",
    "\n",
    "Now you're ready to run the script as an experiment. Note that the default environment does not include the **scikit-learn** package, so you need to explicitly add that to the configuration. The conda environment is built on-demand the first time the experiment is run, and cached for future runs that use the same configuration; so the first run will take a little longer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Experiment, ScriptRunConfig, Environment\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "# Create a Python environment for the experiment (from a .yml file)\n",
    "env = Environment.from_conda_specification(\"experiment_env\", \"environment.yml\")\n",
    "\n",
    "# Create a script config\n",
    "script_config = ScriptRunConfig(source_directory=training_folder,\n",
    "                                script='diabetes_training.py',\n",
    "                                environment=env) \n",
    "\n",
    "# submit the experiment run\n",
    "experiment_name = 'mslearn-train-diabetes'\n",
    "experiment = Experiment(workspace=ws, name=experiment_name)\n",
    "run = experiment.submit(config=script_config)\n",
    "\n",
    "# Show the running experiment run in the notebook widget\n",
    "RunDetails(run).show()\n",
    "\n",
    "# Block until the experiment run has completed\n",
    "run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can retrieve the metrics and outputs from the **Run** object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics and files\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register the trained model\n",
    "\n",
    "Note that the outputs of the experiment include the trained model file (**diabetes_model.pkl**). You can register this model in your Azure Machine Learning workspace, making it possible to track model versions and retrieve them later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "# Register the model\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'Script'},\n",
    "                   properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "# List registered models\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Note**: The code above uses *properties* and *tags* to note additional information about the model. The key difference between properties and tags is that properties cannot be changed after the model is registered, while tags for a registered model can be modified.\n",
    "\n",
    "## Create a parameterized training script\n",
    "\n",
    "You can increase the flexibility of your training experiment by adding parameters to your script, enabling you to repeat the same training experiment with different settings. In this case, you'll add a parameter for the regularization rate used by the logistic regression algorithm when training the model.\n",
    "\n",
    "Again, lets start by creating a folder for the parameterized script and the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, shutil\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "training_folder = 'diabetes-training-params'\n",
    "os.makedirs(training_folder, exist_ok=True)\n",
    "\n",
    "# Copy the data file into the experiment folder\n",
    "shutil.copy('data/diabetes.csv', os.path.join(training_folder, \"diabetes.csv\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create a script with an argument for the regularization rate hyperparameter. The argument is read using a Python **argparse.ArgumentParser** object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $training_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "import os\n",
    "import argparse\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# Set regularization hyperparameter\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--reg_rate', type=float, dest='reg', default=0.01)\n",
    "args = parser.parse_args()\n",
    "reg = args.reg\n",
    "\n",
    "# load the diabetes dataset\n",
    "print(\"Loading Data...\")\n",
    "# load the diabetes dataset\n",
    "diabetes = pd.read_csv('diabetes.csv')\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the script with arguments\n",
    "\n",
    "You run the script as an experiment like you did previously, reusing the environment you created; but this time you must provide the **--reg_rate** parameter that the script expects as an argument."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a script config\n",
    "script_config = ScriptRunConfig(source_directory=training_folder,\n",
    "                                script='diabetes_training.py',\n",
    "                                arguments = ['--reg_rate', 0.1],\n",
    "                                environment=env) \n",
    "\n",
    "# submit the experiment\n",
    "experiment_name = 'mslearn-train-diabetes'\n",
    "experiment = Experiment(workspace=ws, name=experiment_name)\n",
    "run = experiment.submit(config=script_config)\n",
    "RunDetails(run).show()\n",
    "run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, we can get the metrics and outputs from the completed run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register a new version of the model\n",
    "\n",
    "Now that you've trained a new model, you can register it as a new version in the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "# Register the model\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'Parameterized script'},\n",
    "                   properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "# List registered models\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also view registered models in your workspace on the **Models** page in [Azure Machine Learning studio](https://ml.azure.com).\n",
    "\n",
    "If you've finished exploring, you can close this notebook and shut down your compute instance."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}