07 - Work with Compute.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Work with Compute\n",
    "\n",
    "When you run a script as an Azure Machine Learning experiment, you need to define the execution context for the experiment run. The execution context is made up of:\n",
    "\n",
    "* The Python environment for the script, which must include all Python packages used in the script.\n",
    "* The compute target on which the script will be run. This could be the local workstation from which the experiment run is initiated, or a remote compute target such as a training cluster that is provisioned on-demand.\n",
    "\n",
    "In this notebook, you'll explore *environments* and *compute targets* for experiments."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to your workspace\n",
    "\n",
    "To get started, connect to your workspace.\n",
    "\n",
    "> **Note**: If you haven't already established an authenticated session with your Azure subscription, you'll be prompted to authenticate by clicking a link, entering an authentication code, and signing into Azure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace\n",
    "\n",
    "# Load the workspace from the saved config file\n",
    "ws = Workspace.from_config()\n",
    "print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare data for an experiment\n",
    "\n",
    "In this notebook, you'll use a dataset containing details of diabetes patients. Run the cell below to create this dataset (if it already exists, the code will find the existing version)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Dataset\n",
    "\n",
    "default_ds = ws.get_default_datastore()\n",
    "\n",
    "if 'diabetes dataset' not in ws.datasets:\n",
    "    default_ds.upload_files(files=['./data/diabetes.csv', './data/diabetes2.csv'], # Upload the diabetes csv files in /data\n",
    "                        target_path='diabetes-data/', # Put it in a folder path in the datastore\n",
    "                        overwrite=True, # Replace existing files of the same name\n",
    "                        show_progress=True)\n",
    "\n",
    "    #Create a tabular dataset from the path on the datastore (this may take a short while)\n",
    "    tab_data_set = Dataset.Tabular.from_delimited_files(path=(default_ds, 'diabetes-data/*.csv'))\n",
    "\n",
    "    # Register the tabular dataset\n",
    "    try:\n",
    "        tab_data_set = tab_data_set.register(workspace=ws, \n",
    "                                name='diabetes dataset',\n",
    "                                description='diabetes data',\n",
    "                                tags = {'format':'CSV'},\n",
    "                                create_new_version=True)\n",
    "        print('Dataset registered.')\n",
    "    except Exception as ex:\n",
    "        print(ex)\n",
    "else:\n",
    "    print('Dataset already registered.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a training script\n",
    "\n",
    "Run the following two cells to create:\n",
    "\n",
    "1. A folder for a new experiment\n",
    "2. An training script file that uses **scikit-learn** to train a model and **matplotlib** to plot a ROC curve."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "experiment_folder = 'diabetes_training_logistic'\n",
    "os.makedirs(experiment_folder, exist_ok=True)\n",
    "print(experiment_folder, 'folder created')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $experiment_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "import argparse\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "import os\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Get script arguments\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--regularization', type=float, dest='reg_rate', default=0.01, help='regularization rate')\n",
    "parser.add_argument(\"--input-data\", type=str, dest='training_dataset_id', help='training dataset')\n",
    "args = parser.parse_args()\n",
    "\n",
    "# Set regularization hyperparameter\n",
    "reg = args.reg_rate\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# load the diabetes data (passed as an input dataset)\n",
    "print(\"Loading Data...\")\n",
    "diabetes = run.input_datasets['training_data'].to_pandas_dataframe()\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "# plot ROC curve\n",
    "fpr, tpr, thresholds = roc_curve(y_test, y_scores[:,1])\n",
    "fig = plt.figure(figsize=(6, 4))\n",
    "# Plot the diagonal 50% line\n",
    "plt.plot([0, 1], [0, 1], 'k--')\n",
    "# Plot the FPR and TPR achieved by our model\n",
    "plt.plot(fpr, tpr)\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('ROC Curve')\n",
    "run.log_image(name = \"ROC\", plot = fig)\n",
    "plt.show()\n",
    "\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "# note file saved in the outputs folder is automatically uploaded into experiment record\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define an environment\n",
    "\n",
    "When you run a Python script as an experiment in Azure Machine Learning, a Conda environment is created to define the execution context for the script. Azure Machine Learning provides a default environment that includes many common packages; including the **azureml-defaults** package that contains the libraries necessary for working with an experiment run, as well as popular packages like **pandas** and **numpy**.\n",
    "\n",
    "You can also define your own environment in a Conda specification file, adding packages by using **conda** or **pip** to ensure your experiment has access to all the libraries it requires.\n",
    "\n",
    "> **Note**: The conda dependencies are installed first, followed by the pip dependencies. Since the **pip** package is required to install the pip dependencies, it's good practice to include it in the conda dependencies.\n",
    "\n",
    "Run the following cell to create a Conda specification file named *experiment_env.yml* in the same folder as this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $experiment_folder/experiment_env.yml\n",
    "name: experiment_env\n",
    "dependencies:\n",
    "  # The python interpreter version.\n",
    "  # Currently Azure ML only supports 3.5.2 and later.\n",
    "- python=3.6.2\n",
    "- scikit-learn\n",
    "- ipykernel\n",
    "- matplotlib\n",
    "- pandas\n",
    "- pip\n",
    "- pip:\n",
    "  - azureml-defaults\n",
    "  - pyarrow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can use your custom conda specification file to create an environment for your experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Environment\n",
    "\n",
    "# Create a Python environment for the experiment (from a .yml file)\n",
    "experiment_env = Environment.from_conda_specification(\"experiment_env\", experiment_folder + \"/experiment_env.yml\")\n",
    "\n",
    "# Let Azure ML manage dependencies\n",
    "experiment_env.python.user_managed_dependencies = False \n",
    "\n",
    "# Print the environment details\n",
    "print(experiment_env.name, 'defined.')\n",
    "print(experiment_env.python.conda_dependencies.serialize_to_string())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can use the environment to run a script as an experiment.\n",
    "\n",
    "The following code assigns the environment you created to a ScriptRunConfig, and submits an experiment. As the experiment runs, observe the run details in the widget and in the **azureml_logs/60_control_log.txt** output log, you'll see the conda environment being built.\n",
    "\n",
    "> **Note**: The code below creates a **DockerConfiguration** for the script run, and setting its **use_docker** attribute to **True** in order to host the script's environment in a Docker container. This is the default behavior, so you can omit this; but we're including it here to be explicit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Experiment, ScriptRunConfig\n",
    "from azureml.core.runconfig import DockerConfiguration\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "# Get the training dataset\n",
    "diabetes_ds = ws.datasets.get(\"diabetes dataset\")\n",
    "\n",
    "# Create a script config\n",
    "script_config = ScriptRunConfig(source_directory=experiment_folder,\n",
    "                                script='diabetes_training.py',\n",
    "                                arguments = ['--regularization', 0.1, # Regularizaton rate parameter\n",
    "                                             '--input-data', diabetes_ds.as_named_input('training_data')], # Reference to dataset\n",
    "                                environment=experiment_env,\n",
    "                                docker_runtime_config=DockerConfiguration(use_docker=True)) # Use docker to host environment\n",
    "\n",
    "# submit the experiment\n",
    "experiment_name = 'mslearn-train-diabetes'\n",
    "experiment = Experiment(workspace=ws, name=experiment_name)\n",
    "run = experiment.submit(config=script_config)\n",
    "RunDetails(run).show()\n",
    "run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The experiment successfully used the environment, which included all of the packages it required - you can view the metrics and outputs from the experiment run in Azure Machine Learning Studio, or by running the code below - including the model trained using **scikit-learn** and the ROC chart image generated using **matplotlib**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register the environment\n",
    "\n",
    "Having gone to the trouble of defining an environment with the packages you need, you can register it in the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Register the environment\n",
    "experiment_env.register(workspace=ws)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the environment is registered with the name you assigned when you first created it (in this case, *diabetes-experiment-env*).\n",
    "\n",
    "With the environment registered, you can reuse it for any scripts that have the same requirements. For example, let's create a folder and script to train a diabetes model using a different algorithm:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "experiment_folder = 'diabetes_training_tree'\n",
    "os.makedirs(experiment_folder, exist_ok=True)\n",
    "print(experiment_folder, 'folder created')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $experiment_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "import argparse\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "import os\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Get script arguments\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument(\"--input-data\", type=str, dest='training_dataset_id', help='training dataset')\n",
    "args = parser.parse_args()\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# load the diabetes data (passed as an input dataset)\n",
    "print(\"Loading Data...\")\n",
    "diabetes = run.input_datasets['training_data'].to_pandas_dataframe()\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Train a decision tree model\n",
    "print('Training a decision tree model')\n",
    "model = DecisionTreeClassifier().fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "# plot ROC curve\n",
    "fpr, tpr, thresholds = roc_curve(y_test, y_scores[:,1])\n",
    "fig = plt.figure(figsize=(6, 4))\n",
    "# Plot the diagonal 50% line\n",
    "plt.plot([0, 1], [0, 1], 'k--')\n",
    "# Plot the FPR and TPR achieved by our model\n",
    "plt.plot(fpr, tpr)\n",
    "plt.xlabel('False Positive Rate')\n",
    "plt.ylabel('True Positive Rate')\n",
    "plt.title('ROC Curve')\n",
    "run.log_image(name = \"ROC\", plot = fig)\n",
    "plt.show()\n",
    "\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "# note file saved in the outputs folder is automatically uploaded into experiment record\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can retrieve the registered environment and use it in a new experiment that runs the alternative training script (there is no regularization parameter this time because a Decision Tree classifier doesn't require it)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the registered environment\n",
    "registered_env = Environment.get(ws, 'experiment_env')\n",
    "\n",
    "# Get the training dataset\n",
    "diabetes_ds = ws.datasets.get(\"diabetes dataset\")\n",
    "\n",
    "# Create a script config\n",
    "script_config = ScriptRunConfig(source_directory=experiment_folder,\n",
    "                              script='diabetes_training.py',\n",
    "                              arguments = ['--input-data', diabetes_ds.as_named_input('training_data')], # Reference to dataset\n",
    "                              environment=registered_env,\n",
    "                              docker_runtime_config=DockerConfiguration(use_docker=True)) # Use docker to host environment \n",
    "\n",
    "# submit the experiment\n",
    "experiment_name = 'mslearn-train-diabetes'\n",
    "experiment = Experiment(workspace=ws, name=experiment_name)\n",
    "run = experiment.submit(config=script_config)\n",
    "RunDetails(run).show()\n",
    "run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time the experiment runs more quickly because a matching environment has been cached from the previous run, so it doesn't need to be recreated on the local compute. However, even on a different compute target, the same environment would be created and used - ensuring consistency for your experiment script execution context.\n",
    "\n",
    "Let's look at the metrics and outputs from the experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View registered environments\n",
    "\n",
    "In addition to registering your own environments, you can leverage pre-built \"curated\" environments for common experiment types. The following code lists all registered environments:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from azureml.core import Environment\n",
    "\n",
    "envs = Environment.list(workspace=ws)\n",
    "for env in envs:\n",
    "    print(\"Name\",env)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All curated environments have names that begin ***AzureML-*** (you can't use this prefix for your own environments)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a compute cluster\n",
    "\n",
    "In many cases, your local compute resources may not be sufficient to process a complex or long-running experiment that needs to process a large volume of data; and you may want to take advantage of the ability to dynamically create and use compute resources in the cloud. Azure Machine Learning supports a range of compute targets, which you can define in your workpace and use to run experiments; paying for the resources only when using them.\n",
    "\n",
    "You can create a compute cluster in [Azure Machine Learning studio](https://ml.azure.com), or by using the Azure Machine Learning SDK. The following code cell checks your workspace for the existance of a compute cluster with a specified name, and if it doesn't exist, creates it.\n",
    "\n",
    "> **Important**: Change *your-compute-cluster* to a suitable name for your compute cluster in the code below before running it - you can specify the name of an existing cluster if you have one. Cluster names must be globally unique names between 2 to 16 characters in length. Valid characters are letters, digits, and the - character."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core.compute import ComputeTarget, AmlCompute\n",
    "from azureml.core.compute_target import ComputeTargetException\n",
    "\n",
    "cluster_name = \"your-compute-cluster\"\n",
    "\n",
    "try:\n",
    "    # Check for existing compute target\n",
    "    training_cluster = ComputeTarget(workspace=ws, name=cluster_name)\n",
    "    print('Found existing cluster, use it.')\n",
    "except ComputeTargetException:\n",
    "    # If it doesn't already exist, create it\n",
    "    try:\n",
    "        compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_DS11_V2', max_nodes=2)\n",
    "        training_cluster = ComputeTarget.create(ws, cluster_name, compute_config)\n",
    "        training_cluster.wait_for_completion(show_output=True)\n",
    "    except Exception as ex:\n",
    "        print(ex)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **Note**: Compute instances and clusters are based on standard Azure virtual machine images. For this exercise, the *Standard_DS11_v2* image is recommended to achieve the optimal balance of cost and performance. If your subscription has a quota that does not include this image, choose an alternative image; but bear in mind that a larger image may incur higher cost and a smaller image may not be sufficient to complete the tasks. Alternatively, ask your Azure administrator to extend your quota.\n",
    "\n",
    "## Run an experiment on remote compute\n",
    "\n",
    "Now you're ready to re-run the experiment you ran previously, but this time on the compute cluster you created. \n",
    "\n",
    "> **Note**: The experiment will take quite a lot longer because a container image must be built with the conda environment, and then the cluster nodes must be started and the image deployed before the script can be run. For a simple experiment like the diabetes training script, this may seem inefficient; but imagine you needed to run a more complex experiment that takes several hours - dynamically creating more scalable compute may reduce the overall time significantly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a script config\n",
    "script_config = ScriptRunConfig(source_directory=experiment_folder,\n",
    "                                script='diabetes_training.py',\n",
    "                                arguments = ['--input-data', diabetes_ds.as_named_input('training_data')],\n",
    "                                environment=registered_env,\n",
    "                                compute_target=cluster_name) \n",
    "\n",
    "# submit the experiment\n",
    "experiment_name = 'mslearn-train-diabetes'\n",
    "experiment = Experiment(workspace=ws, name=experiment_name)\n",
    "run = experiment.submit(config=script_config)\n",
    "RunDetails(run).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While you're waiting for the experiment to run, you can check on the status of the compute in the widget above or in [Azure Machine Learning studio](https://ml.azure.com). You can also check the status of the compute using the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cluster_state = training_cluster.get_status()\n",
    "print(cluster_state.allocation_state, cluster_state.current_node_count)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that it will take a while before the status changes from *steady* to *resizing* (now might be a good time to take a coffee break!). To block the kernel until the run completes, run the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.wait_for_completion()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keep an eye on the kernel indicator at the top right of the page, when it turns from **&#9899;** to **&#9711;**, the code has finished running.\n",
    "\n",
    "After the experiment has finished, you can get the metrics and files generated by the experiment run. This time, the files will include logs for building the image and managing the compute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can register the model that was trained by the experiment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "# Register the model\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'Compute cluster'}, properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "# List registered models\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **More Information**:\n",
    ">\n",
    "> - For more information about environments in Azure Machine Learning, see [Create & use software environments in Azure Machine Learning](https://docs.microsoft.com/azure/machine-learning/how-to-use-environments)\n",
    "> - For more information about compute targets in Azure Machine Learning, see the [What are compute targets in Azure Machine Learning?](https://docs.microsoft.com/azure/machine-learning/concept-compute-target)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}