Measures_of_Association.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yDtE-yrbPCd_"
   },
   "source": [
    "# Measures of Association"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AYh1WrnWO9a6"
   },
   "source": [
    "## ANTICIPATED TIME\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1-dReOxVO_CF"
   },
   "source": [
    "2 hour\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2PZMiNjZPV2F"
   },
   "source": [
    "## BEFORE YOU BEGIN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3y9WT3ewPYc6"
   },
   "source": [
    "[Descriptive Statistics](Descriptive_Statistics.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rutjr1-0PdOO"
   },
   "source": [
    "## WHAT YOU WILL LEARN\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yKDhWTxnPeua"
   },
   "source": [
    "- How should you determine the relationship between two variables?\n",
    "- What are Pearson’s correlation coefficient and Spearman’s rank correlation coefficient, and how are they used?\n",
    "- How is correlation different than causation?\n",
    "- How do you calculate correlations between variables and create heatmaps for visualization?\n",
    "- How do you interpret correlation coefficients using data science?\n",
    "- How do you create a contingency table to make categorical associations?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CUNwwXYrPmPO"
   },
   "source": [
    "## DEFINITIONS YOU’LL NEED TO KNOW"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jj-RpA1MPnqS"
   },
   "source": [
    "\n",
    "- Correlations - seeing how two different variables are related.\n",
    "- Correlation coefficient - measures the strength of a linear relationship.\n",
    "- Ratio data - numerical data with a meaningful zero point.\n",
    "- Pearson’s correlation coefficient - measures linear relationships between two variables.\n",
    "- Heatmap - a visual representation that uses colors to show the magnitude of values in a dataset.\n",
    "- Spearman’s rank correlation coefficient - computes a measure of association using ranks.\n",
    "- Ordinal data - categorical data with a natural order or ranking.\n",
    "- Contingency table - a type of table used to show the frequency distribution of variables.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mmj-iJYpPuzr"
   },
   "source": [
    "## SCENARIO:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false,
    "html": "<iframe class='metadata-html' width='560' height='315' src='https://video.olney.ai/4-measures.mp4' title='YouTube video player' frameborder='0' allow='accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share' referrerpolicy='strict-origin-when-cross-origin' allowfullscreen></iframe>",
    "id": "npqY6ZiSbavX",
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "source": [
    "\n",
    "Kiana is a new member of the afterschool program that Ethan and Diego are involved in. They all shared their ideas for solving the traffic and pollution problems. Diego has been collecting different types of data to help Angelina and the group understand the variables causing pollution in the city. Angelina and Diego have both expressed that they wanted to understand the data they’ve collected better by looking at the middle points of the samples. Since Kiana joined the group, she’s started to brainstorm to see if two variables are related and follow similar patterns. For example, Diego suggested to Kiana to look at the number of cars and their emissions levels. However, Kiana suggests looking at the relationship between the number of sidewalks or bike lanes and whether there is a relationship with pollution levels. Kiana wonders if there are other variables to consider that could cause the pollution levels to be low, in addition to the sidewalks and bike lanes.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_X-NqMnxP_dT"
   },
   "source": [
    "## WHAT DO I NEED TO KNOW?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fWQknlPpQCbJ"
   },
   "source": [
    "\n",
    "We looked at the type of variables in Data Science and Nature of Data. So let’s assume that you have a hunch that two variables are related. Maybe they both go up in kinda the same direction? Or maybe something goes up, but the other variable seems to go down? Can we go beyond a ‘hunch’ and actually measure if they are related?\n",
    "\n",
    "You’re in luck because that’s what we are going to look at!\n",
    "\n",
    "A couple of things.\n",
    "\n",
    "1. Just because our correlation finds things are related, that doesn’t necessarily mean that it causes it….only that we see the strength of the relationship and the direction it’s moving.\n",
    "\n",
    "2. As a general rule, anything below 0.10 is considered a weak association between the variables, values between 0.10 and 0.30 are considered a medium association, and values above 0.30 are evidence of a strong association.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6poMKy-_QNs8"
   },
   "source": [
    "## YOUR TURN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BhMFszExQO6g"
   },
   "source": [
    "\n",
    "Now that you understand measures of association and the different ways these methods can show us relationships between data, it’s time to practice applying this concept! As you go through this, pay close attention to the different relationships that come from the data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RQ0IpuEKSSU-"
   },
   "source": [
    "## Correlations for Ratio Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1d7Hunz2SSwl"
   },
   "source": [
    "\n",
    "So let’s say we want to look at two ratio variables. How do we know if something is actually related?\n",
    "\n",
    "\n",
    "Welp, the **Pearson correlation coefficient (r)**  is a number that lets us know how two things are related. To make it even easier, we can just think of this number going from either -1 to 1. If the correlation is above 0 and up to 1, it means the two things have a positive correlation and moving. An example might be the number of minutes spent exercising and the amount of water you drink. In other words, as something goes up, the other variable will also go up.\n",
    "\n",
    "\n",
    "If the correlation is below 0 and goes to -1, it means the two things are still related but have a negative correlation because they move in opposite directions. Thinking through our exercise example, you might think about the number of minutes spent biking might be related to lower weight.\n",
    "\n",
    "\n",
    "And how about 0? That is simple - it just means the two things are not related at all and we could say there is no correlation. Maybe think of the minutes spent exercising and the number of times you stub your toe.\n",
    "\n",
    "\n",
    "Huh?\n",
    "\n",
    "\n",
    "Since we probably wouldn’t expect to see any relationship there, we would expect it to be r = 0.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DoHgMQveSdDX"
   },
   "source": [
    "### Goal 1: Importing the Pandas Library\n",
    "\n",
    "Need extra tools to help solve this problem? Well, we can bring in extra ‘libraries’ to help us do extra data science stuff. You can think of it as an ‘add-on’. In this case, we bring in pandas, which is a popular library for doing data science stuff."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lrs7plwJSgRd"
   },
   "source": [
    "#### Blockly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ic4uVHFWSqZg"
   },
   "source": [
    "\n",
    "**Step 1 - Starting the import**\n",
    "\n",
    "First, we need to set up a “command” to tell the computer what to do. In this case, “command” is to “import” to bring the add-on package in.\n",
    "\n",
    "Bring in the IMPORT menu, which can be helpful to bring in other data tools. In this case, we're bringing in the **import** block.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OWpn7vNbSrlQ"
   },
   "source": [
    "\n",
    "**Step 2 - Telling what library to import**\n",
    "\n",
    "In the text area, we type the name of the library we want to import. A library is like an extra thing we bring in to give us more coding abilities. In our case, we will type out **pandas**, which will bring in some cool data manipulation features.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-ycwWW77SscR"
   },
   "source": [
    "\n",
    "**Step 3 - Renaming the library so it’s easy to remember**\n",
    "\n",
    "Once you are done, put the **import** and **package** together in a single variable. This handy feature helps cut down on all the typing later on. You can call it whatever is easiest for you to remember. In the example below, we’ve put everything into **pd**, and we type it in the open area.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PK6sFegCStMm"
   },
   "source": [
    "\n",
    "**Step 4 - Connect the blocks to run the code**\n",
    "\n",
    "Connect the blocks and run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GcxNkjYXkAAlpKD?format=png&name=240x240)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#blocks code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HmtQdAZhTCq_"
   },
   "source": [
    "#### Freehand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nhgMdXLTTEhQ"
   },
   "source": [
    "**Step 1 - Starting the import**\n",
    "\n",
    "First, we need to set up a “command” to tell the computer what to do. In this case, “command” is to “import” to bring the add-on package in.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "w6sZGLwETNih"
   },
   "source": [
    "\n",
    "**Step 2 - Telling what library to import**\n",
    "\n",
    "In the text area, we type the name of the library we want to import. A library is like an extra thing we bring in to give us more coding abilities. In our case, we will type out **pandas**, which will bring in some cool data manipulation features.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cySUaR4wTRdE"
   },
   "source": [
    "\n",
    "**Step 3 - Renaming the library so it’s easy to remember**\n",
    "\n",
    "Once you are done, put the ‘import’ and ‘package’ together in a single variable. This handy feature helps cut down on all the typing later on. Feel free to use whatever name you want that will help you remember it later on. In the example below, we’ve put everything into **‘pd’**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "542iC2S9TVl6"
   },
   "source": [
    "\n",
    "\n",
    "**Step 4 - Run the code**\n",
    "\n",
    "Hit ‘control’ and ‘enter’ at the same time to run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GZmkVCYWEA4oGso?format=jpg&name=small) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yvJG2NQ3UbHH"
   },
   "source": [
    "**Your Turn**: Now it’s your turn! We’re going to dive into the pandas package, which helps us with some really cool data science things. First, let’s import the package and assign it to the variable “pd” to make it easier to use throughout our notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "SW--84fFUd0H"
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2sc0G9gCUgTA"
   },
   "source": [
    "**Explanation**: *Congrats!  Your attempts finally made it!  Now you have successfully imported the \"pandas\" package as the variable \"pd\"*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RE7dbfyjUlJX"
   },
   "source": [
    "### Goal 2: Bringing in the Dataframe\n",
    "\n",
    "Let’s bring in the data that we want to look at.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ebRr2iPKUsBP"
   },
   "source": [
    "#### Blockly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cOeKvGoyUuGT"
   },
   "source": [
    "\n",
    "**Step 1 - Write out the variable name you want to use**\n",
    "\n",
    "Now that we’re all set with our new package to help us to do cool things, let’s bring the data into a variable and call it **dataframe**. Think of it as a digital spreadsheet with much more power to analyze and manipulate the data!\n",
    "\n",
    "In Blockly, bring in the VARIABLES menu.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wxDrYWKmUwgR"
   },
   "source": [
    "\n",
    "**Step 2 - Assign the dataframe to the variable you created**\n",
    "\n",
    "Just like we did before, let’s type out a variable name. Rather than type out the full file name for our data, this easy to remember name will hold the data we bring in.\n",
    "\n",
    "In Blockly, go to the Variables and drag the Set block for the **dataframe** variable. This will allow us to assign the result of a function call to the variable.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WhcQHHefUxew"
   },
   "source": [
    "\n",
    "**Step 3 - Brining in the data**\n",
    "\n",
    "Now we need to look at the file that has all our data.\n",
    "To load our dataframe, we’ll use a simple command to bring in the file we need (CSV….Comma Separated Values). Let’s say we have a file called ‘datasets/AirQuality.csv' in the folder **‘datasets’**. We’re telling Python to read the CSV file and store it in a variable called **dataframe**.\n",
    "\n",
    "From the Variable menu, drag a DO block using the **pd** variable, go ahead with the do operation **read_csv**. The read_csv function reads a CSV file and returns a DataFrame object.\n",
    "\n",
    "In our case, let’s bring in the “datasets/AirQuality.csv\" (use the Quotes from the TEXT menu) because that is what Angelina is working with.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "haLRHtB5UzcN"
   },
   "source": [
    "\n",
    "**Step 4 - Display the variable**\n",
    "\n",
    "Let’s see it now by ‘displaying’ and showing our work.\n",
    "\n",
    "Drag the **dataframe** variable to the workspace, making it available for further use in our program. This step is more of a visualization step, as it allows us to see the variable in the Blockly workspace.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rUg_jnVEU0LM"
   },
   "source": [
    "\n",
    "**Step 5 - Connect the blocks to run the code**\n",
    "\n",
    "Connect the blocks and run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GaNM5olXIAAzaAB?format=png&name=small)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#blocks code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2ViSJ8WuWaV8"
   },
   "source": [
    "#### Freehand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xeaHzNEPWbcf"
   },
   "source": [
    "\n",
    "**Step 1 - Write out the variable name you want to use**\n",
    "\n",
    "Now that we’re all set with our new package to help us to do cool things, let’s bring the data into a variable called **dataframe**. Think of it as a digital spreadsheet with much more power to analyze and manipulate the data!\n",
    "\n",
    "Just like we did before, let’s type out a variable name. Rather than type out the full file name for our data, this easy to remember name will hold the data we bring in.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CyHOfuOKWegA"
   },
   "source": [
    "\n",
    "**Step 2 - Assign the dataframe to the variable you created**\n",
    "\n",
    "Just like we did before, let’s type out a variable name. Rather than type out the full file name for our data, this easy to remember name will hold the data we bring in."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NXqYRCtiWgSm"
   },
   "source": [
    "\n",
    "\n",
    "**Step 3 - Bring in the data**\n",
    "\n",
    "Now we need to look at the file that has all our data.\n",
    "\n",
    "To load our dataframe, we’ll use a simple command to bring in the file we need (CSV….Comma Separated Values). Let’s say we have a file called ‘datasets/AirQuality.csv' in the folder **‘datasets’**. We’re telling Python to read the CSV file and store it in a variable called **dataframe**. For this function, we need to specify the code as “pd.read_csv”, which makes the code read the csv file. This variable is now our dataframe!\n",
    "\n",
    "In our case, let’s bring in the “datasets/AirQuality.csv\" (user the Quotes from the TEXT menu) because that is what Kiana is working with.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xKHaZiCUWhgG"
   },
   "source": [
    "\n",
    "**Step 4 - Print the variable**\n",
    "\n",
    "Let’s see it now by ‘printing’ and showing our work. Retype the variable name underneath the code and it will print the code. In this case, we will type out the variable name **dataframe**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5GpiJSdnWblk"
   },
   "source": [
    "\n",
    "**Step 5 - Run the code**\n",
    "\n",
    "Hit ‘control’ and ‘enter’ at the same time to run the code!\n",
    "\n",
    "\n",
    "![](https://pbs.twimg.com/media/GaNNf6QXIAAhIEA?format=png&name=small)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IiL8_-BTWbnx"
   },
   "source": [
    "**Your Turn**: Now it’s your turn! Let’s dive in and start working with the data! We’ll begin by loading it into a dataframe, which will allow us to easily interact with and analyze the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423
    },
    "id": "FBEntrGMXZbU",
    "outputId": "095f031d-4e25-4f02-cdb4-02874390d0b0"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.google.colaboratory.intrinsic+json": {
       "summary": "{\n  \"name\": \"dataframe\",\n  \"rows\": 150,\n  \"fields\": [\n    {\n      \"column\": \"TrafficVolume\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 2300.1836888404623,\n        \"min\": 0.0,\n        \"max\": 10000.0,\n        \"num_unique_values\": 35,\n        \"samples\": [\n          5277.777778,\n          555.5555556,\n          3611.111111\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"AverageSpeed\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 21,\n        \"min\": 0,\n        \"max\": 120,\n        \"num_unique_values\": 23,\n        \"samples\": [\n          15,\n          100,\n          75\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"CO2Emissions\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 149.52715423255,\n        \"min\": 0.0,\n        \"max\": 500.0,\n        \"num_unique_values\": 43,\n        \"samples\": [\n          483.0508475,\n          237.2881356,\n          228.8135593\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"NoiseLevel\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 28.61852781378158,\n        \"min\": 30.0,\n        \"max\": 120.0,\n        \"num_unique_values\": 22,\n        \"samples\": [\n          33.75,\n          71.25,\n          75.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"TrafficCondition\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"Free Flow\",\n          \"Moderate\",\n          \"Heavy\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Road Type\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3,\n        \"samples\": [\n          \"Rural\",\n          \"Highway\",\n          \"City\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
       "type": "dataframe",
       "variable_name": "dataframe"
      },
      "text/html": [
       "\n",
       "  <div id=\"df-d43cfd6f-5228-4df0-8419-fc7587b0f0cd\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>TrafficVolume</th>\n",
       "      <th>AverageSpeed</th>\n",
       "      <th>CO2Emissions</th>\n",
       "      <th>NoiseLevel</th>\n",
       "      <th>TrafficCondition</th>\n",
       "      <th>Road Type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2222.222222</td>\n",
       "      <td>75</td>\n",
       "      <td>33.898305</td>\n",
       "      <td>33.75</td>\n",
       "      <td>Free Flow</td>\n",
       "      <td>Rural</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1666.666667</td>\n",
       "      <td>50</td>\n",
       "      <td>33.898305</td>\n",
       "      <td>33.75</td>\n",
       "      <td>Free Flow</td>\n",
       "      <td>Rural</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1111.111111</td>\n",
       "      <td>60</td>\n",
       "      <td>25.423729</td>\n",
       "      <td>33.75</td>\n",
       "      <td>Free Flow</td>\n",
       "      <td>Rural</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>833.333333</td>\n",
       "      <td>55</td>\n",
       "      <td>42.372881</td>\n",
       "      <td>33.75</td>\n",
       "      <td>Free Flow</td>\n",
       "      <td>Rural</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1944.444444</td>\n",
       "      <td>80</td>\n",
       "      <td>33.898305</td>\n",
       "      <td>33.75</td>\n",
       "      <td>Free Flow</td>\n",
       "      <td>Rural</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>145</th>\n",
       "      <td>6666.666667</td>\n",
       "      <td>50</td>\n",
       "      <td>355.932203</td>\n",
       "      <td>112.50</td>\n",
       "      <td>Heavy</td>\n",
       "      <td>City</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>146</th>\n",
       "      <td>5555.555556</td>\n",
       "      <td>25</td>\n",
       "      <td>338.983051</td>\n",
       "      <td>97.50</td>\n",
       "      <td>Heavy</td>\n",
       "      <td>City</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>147</th>\n",
       "      <td>6111.111111</td>\n",
       "      <td>50</td>\n",
       "      <td>355.932203</td>\n",
       "      <td>101.25</td>\n",
       "      <td>Heavy</td>\n",
       "      <td>City</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>5277.777778</td>\n",
       "      <td>70</td>\n",
       "      <td>372.881356</td>\n",
       "      <td>112.50</td>\n",
       "      <td>Heavy</td>\n",
       "      <td>City</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>4444.444444</td>\n",
       "      <td>50</td>\n",
       "      <td>347.457627</td>\n",
       "      <td>93.75</td>\n",
       "      <td>Heavy</td>\n",
       "      <td>City</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>150 rows × 6 columns</p>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d43cfd6f-5228-4df0-8419-fc7587b0f0cd')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-d43cfd6f-5228-4df0-8419-fc7587b0f0cd button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-d43cfd6f-5228-4df0-8419-fc7587b0f0cd');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-3b0bda9f-263f-4b39-adfe-7ade207083bb\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-3b0bda9f-263f-4b39-adfe-7ade207083bb')\"\n",
       "            title=\"Suggest charts\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-3b0bda9f-263f-4b39-adfe-7ade207083bb button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "\n",
       "  <div id=\"id_c5193976-e2e6-4088-bf72-155cb7873d0b\">\n",
       "    <style>\n",
       "      .colab-df-generate {\n",
       "        background-color: #E8F0FE;\n",
       "        border: none;\n",
       "        border-radius: 50%;\n",
       "        cursor: pointer;\n",
       "        display: none;\n",
       "        fill: #1967D2;\n",
       "        height: 32px;\n",
       "        padding: 0 0 0 0;\n",
       "        width: 32px;\n",
       "      }\n",
       "\n",
       "      .colab-df-generate:hover {\n",
       "        background-color: #E2EBFA;\n",
       "        box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "        fill: #174EA6;\n",
       "      }\n",
       "\n",
       "      [theme=dark] .colab-df-generate {\n",
       "        background-color: #3B4455;\n",
       "        fill: #D2E3FC;\n",
       "      }\n",
       "\n",
       "      [theme=dark] .colab-df-generate:hover {\n",
       "        background-color: #434B5C;\n",
       "        box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "        filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "        fill: #FFFFFF;\n",
       "      }\n",
       "    </style>\n",
       "    <button class=\"colab-df-generate\" onclick=\"generateWithVariable('dataframe')\"\n",
       "            title=\"Generate code using this dataframe.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M7,19H8.4L18.45,9,17,7.55,7,17.6ZM5,21V16.75L18.45,3.32a2,2,0,0,1,2.83,0l1.4,1.43a1.91,1.91,0,0,1,.58,1.4,1.91,1.91,0,0,1-.58,1.4L9.25,21ZM18.45,9,17,7.55Zm-12,3A5.31,5.31,0,0,0,4.9,8.1,5.31,5.31,0,0,0,1,6.5,5.31,5.31,0,0,0,4.9,4.9,5.31,5.31,0,0,0,6.5,1,5.31,5.31,0,0,0,8.1,4.9,5.31,5.31,0,0,0,12,6.5,5.46,5.46,0,0,0,6.5,12Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "    <script>\n",
       "      (() => {\n",
       "      const buttonEl =\n",
       "        document.querySelector('#id_c5193976-e2e6-4088-bf72-155cb7873d0b button.colab-df-generate');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      buttonEl.onclick = () => {\n",
       "        google.colab.notebook.generateWithVariable('dataframe');\n",
       "      }\n",
       "      })();\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "     TrafficVolume  AverageSpeed  CO2Emissions  NoiseLevel TrafficCondition  \\\n",
       "0      2222.222222            75     33.898305       33.75        Free Flow   \n",
       "1      1666.666667            50     33.898305       33.75        Free Flow   \n",
       "2      1111.111111            60     25.423729       33.75        Free Flow   \n",
       "3       833.333333            55     42.372881       33.75        Free Flow   \n",
       "4      1944.444444            80     33.898305       33.75        Free Flow   \n",
       "..             ...           ...           ...         ...              ...   \n",
       "145    6666.666667            50    355.932203      112.50            Heavy   \n",
       "146    5555.555556            25    338.983051       97.50            Heavy   \n",
       "147    6111.111111            50    355.932203      101.25            Heavy   \n",
       "148    5277.777778            70    372.881356      112.50            Heavy   \n",
       "149    4444.444444            50    347.457627       93.75            Heavy   \n",
       "\n",
       "    Road Type  \n",
       "0       Rural  \n",
       "1       Rural  \n",
       "2       Rural  \n",
       "3       Rural  \n",
       "4       Rural  \n",
       "..        ...  \n",
       "145      City  \n",
       "146      City  \n",
       "147      City  \n",
       "148      City  \n",
       "149      City  \n",
       "\n",
       "[150 rows x 6 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe = pd.read_csv('datasets/AirQuality.csv')\n",
    "dataframe\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_NevbqmsWbp7"
   },
   "source": [
    "**Explanation**: *Easy-peasy! Just like before, you’ve brought in the dataset*.\n",
    "\n",
    "*The AirQuality dataset contains hourly air quality measurements from Italy, collected from March 2004 to February 2005. It has 19 columns, including date, time, and concentrations of various pollutants like CO, NOx, and O3, as well as temperature, humidity, and absolute humidity. The dataset has 9357 rows, representing hourly measurements over a year. It can be used for analyzing air quality patterns, predicting pollution levels, and investigating the impact on human health*.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aDXMKSazWbsT"
   },
   "source": [
    "### Goal 3: Calculate Correlations\n",
    "\n",
    "Now that we’ve brought in Pandas and have our dataset loaded, let’s see if we can find any relationships between the variables we can calculate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2GlXe-atWbuo"
   },
   "source": [
    "\n",
    "**Step 1 - Write out the variable name you want to use for the correlation matrix**\n",
    "\n",
    "Create a new variable named **corrMatrix** and drag it into the VARIABLES menu. This variable will hold the correlation matrix calculated from the dataframe.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g95JS6n0X2ZC"
   },
   "source": [
    "\n",
    "**Step 2 - Call the corr() method of the dataframe**\n",
    "\n",
    "To get the correlation of variables, we first need to call the corr() method to help calculate the correlations. We’ll apply it to the data we have in our **dataframe** variable.\n",
    "\n",
    "From the VARIABLES menu, set the **corrMatrix** variable to the result of WITH the dataframe variable DO the **corr**() function.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lAci5AToX4Ak"
   },
   "source": [
    "\n",
    "**Step 3 - Run correlations for only numeric values**\n",
    "\n",
    "We tell our code **numeric_only=True** (FREESTYLE menu), which means we only want to consider numeric columns in the calculation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9ZZj-FlLX4wp"
   },
   "source": [
    "\n",
    "**Step 4 - Assign the correlation matrix to the variable you created**\n",
    "\n",
    "we need to store the correlation into a variable for later use. Here, we store the correlation into the variable **corrMatrix**.\n",
    "\n",
    "We will do this through Set **corrMatrix** variable to the dataframe DO corr block.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1XsOmzaLX5zu"
   },
   "source": [
    "\n",
    "**Step 5 - Display the correlation matrix**\n",
    "\n",
    "Lastly, we display the correlation variable. To do this, we drag the **corrMatrix** variable to the workspace, making it available for further use in our program. This step is more of a visualization step, allowing us to see the variable in the Blockly workspace.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Z23Kzwu6X7N0"
   },
   "source": [
    "\n",
    "**Step 6 - Connect the blocks to run the code**\n",
    "\n",
    "Connect the blocks and run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GaNORqEXEAAJDF8?format=png&name=small)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#blocks code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "--7VOG9UYuo7"
   },
   "source": [
    "#### Freehand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aG-6A9IvZJbL"
   },
   "source": [
    "\n",
    "**Step 1 - Call the corr() method of the dataframe**\n",
    "\n",
    "To get the correlation of variables, we first need to call the corr() method to help calculate the correlations. We’ll apply it to the data we have in our **dataframe** variable.\n",
    "\n",
    "`dataframe.corr()`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gQ78v10NZMwt"
   },
   "source": [
    "\n",
    "**Step 2 - Run correlations for only numeric values**\n",
    "\n",
    "We tell our code numeric_only=True (FREESTYLE menu), which means we only want to consider numeric columns in the calculation.\n",
    "\n",
    "`dataframe.corr(numeric_only=True)`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6ZXBO5G0ZPVL"
   },
   "source": [
    "\n",
    "**Step 3 - Assign the correlation matrix to the variable you created**\n",
    "\n",
    "Finally, we need to store the correlation into a variable for later use. Here, we store the correlation into the variable **corrMatrix**\n",
    "\n",
    "`corrMatrix= dataframe.corr(numeric_only=True)`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2fwEIYZnZUtc"
   },
   "source": [
    "\n",
    "**Step 4 - Print the correlation matrix**\n",
    "\n",
    "Lastly, we print the correlation variable.\n",
    "\n",
    "`print(corrMatrix)`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gt0fyKoOZcTk"
   },
   "source": [
    "\n",
    "**Step 5 - Run the code**\n",
    "\n",
    "Hit ‘control’ and ‘enter’ at the same time to run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GZpDrBdX0AUY5IP?format=png&name=small)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kGsV78GGZe7F"
   },
   "source": [
    "**Your Turn**: Now it’s your turn! Let’s start calculating! We’ll begin by calculating the correlation matrix and assigning it a variable, this will allow us to print the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "snWLzZfpZhiD",
    "outputId": "56cd53a0-e5e3-4934-8031-c19cf8a74b30"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "               TrafficVolume  AverageSpeed  CO2Emissions  NoiseLevel\n",
      "TrafficVolume       1.000000     -0.109369      0.871754    0.817954\n",
      "AverageSpeed       -0.109369      1.000000     -0.420516   -0.356544\n",
      "CO2Emissions        0.871754     -0.420516      1.000000    0.962757\n",
      "NoiseLevel          0.817954     -0.356544      0.962757    1.000000\n"
     ]
    }
   ],
   "source": [
    "corrMatrix = dataframe.corr(numeric_only=True)\n",
    "print(corrMatrix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Bpje-SnlZjSS"
   },
   "source": [
    "**Explanation**: *The correlation matrix reveals several interesting relationships between the variables. Traffic volume is strongly positively correlated with CO2 emissions and noise Levels. This means that as traffic volume increases, so do emissions and noise pollution*.\n",
    "\n",
    "*On the other hand, Average Speed is negatively correlated with CO2 Emissions and Noise Level, suggesting that higher speeds are associated with lower emissions and noise levels*.\n",
    "\n",
    "*Also, CO2 Emissions and Noise Level are very strongly correlated, implying that they may be driven by similar factors. Overall, the correlations suggest that traffic volume is a key driver of environmental pollution, while average speed may have a mitigating effect*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LqgLazgLaHJV"
   },
   "source": [
    "### Goal 4: Import the Plotly.Express Library\n",
    "\n",
    "We’ve already brought pandas to help with data science. Let’s bring in Plotly Express to help with some fancy-pants visualizations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "48kwQWmnaKEZ"
   },
   "source": [
    "#### Blockly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "S4Tf3De2aOBf"
   },
   "source": [
    "**Step 1 - Starting the import**\n",
    "\n",
    "First, we need to set up a “command” to tell the computer what to do. In this case, “command” is to “import” to bring the add-on package in.\n",
    "\n",
    "Bring in the IMPORT menu, which can be helpful to bring in other data tools. In this case, we're bringing in the **import** block.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ehtPa0ihaQ0Y"
   },
   "source": [
    "\n",
    "**Step 2 - Telling what library to import**\n",
    "\n",
    "In the text area, we type the name of the library we want to import. A library is like an extra thing we bring in to give us more coding abilities. In our case, we will type out the **plotly.express** library. Plotly is a popular library in Python that provides functions for fancy-pants data visualizations.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LJxNySTNaXwG"
   },
   "source": [
    "\n",
    "**Step 3 - Renaming the library so it’s easy to remember**\n",
    "\n",
    "Once you are done, put the **import** and **plotly.express** together in a single variable. This handy feature helps cut down on all the typing later on. You can call it whatever is easiest for you to remember. In the example below, we’ve put everything into **px** so it’s easier to remember\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PWikY6xDaca_"
   },
   "source": [
    "\n",
    "**Step 4 - Connect the blocks to run the code**\n",
    "\n",
    "Connect the blocks and run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GaON4A9XsAANFz2?format=png&name=240x240)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#blocks code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ex5KHzMuafWf"
   },
   "source": [
    "#### Freehand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bXQ3xEvXanNk"
   },
   "source": [
    "\n",
    "**Step 1 - Starting the import**\n",
    "\n",
    "First, we need to set up a “command” to tell the computer what to do. In this case, “command” is to “import” to bring the add-on package in.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7M5lf6WRaoXv"
   },
   "source": [
    "\n",
    "**Step 2 - Telling what library to import**\n",
    "\n",
    "In the text area, we type the name of the library we want to import. A library is like an extra thing we bring in to give us more coding abilities. In our case, we will type out the **plotly.express** library. Plotly is a popular library in Python that provides functions for fancy-pants data visualizations.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q6-9o7bJapLm"
   },
   "source": [
    "\n",
    "**Step 3 - Renaming the library so it’s easy to remember**\n",
    "\n",
    "Once you are done, put the ‘import’ and ‘package’ together in a single variable. This handy feature helps cut down on all the typing later on. Feel free to use whatever name you want that will help you remember it later on. In the example below, we’ve put everything into **px**.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hT4xV2j3ap63"
   },
   "source": [
    "\n",
    "**Step 4 - Run the code**\n",
    "\n",
    "Hit ‘control’ and ‘enter’ at the same time to run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GaON1ppW0AIrmJh?format=png&name=360x360)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aqCzo9tga09g"
   },
   "source": [
    "**Your Turn**: Now it’s your turn! We’re going to bring in Plotly Express to help with those visualizations. Let’s start the import making sure to rename it before we run it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "8teoZE9qa5dJ"
   },
   "outputs": [],
   "source": [
    "import plotly.express as px"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rXUSjC6sa6MW"
   },
   "source": [
    "**Explanation**: *Congrats! Your attempts finally made it! Now you have successfully imported the **plotly.express** package as the variable **px**.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8icFTIPgbAmj"
   },
   "source": [
    "### Goal 5: View the heatmap of the correlation matrix.\n",
    "\n",
    "We’ve run the correlations saw the numbers. How about let’s look at a heatmap that uses colors to show how strongly the variables are correlated with each other?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3AuBTf78fpF4"
   },
   "source": [
    "#### Blockly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dZJHhm7FfsXj"
   },
   "source": [
    "\n",
    "**Step 1 - Call the image show function from plotly**\n",
    "\n",
    "To make a heatmap, we first need to call the imshow() function with our plotly library (px).\n",
    "\n",
    "From the VARIABLE menu, drag the DO block for the **px** variable, which is a reference to the plotly library.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5v-tPO4bftK9"
   },
   "source": [
    "\n",
    "**Step 2 - Tell what data to use for image show to create the heatmap**\n",
    "\n",
    "Inside the method, we provide the correlation matrix to show. In this case, our correlation matrix is **corrMatrix**.\n",
    "\n",
    "To do that in the VARIABLES menu, we will tell the **imshow** function to look at the **corrMatrix** variable.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9-fb7Pf3fuBq"
   },
   "source": [
    "\n",
    "**Step 3 - Connect the blocks to run the code**\n",
    "\n",
    "Connect the blocks and run the code!\n",
    "\n",
    "\n",
    "![](https://pbs.twimg.com/media/GZoflMaWEAUJ1fw?format=png&name=360x360)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#blocks code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "imDd_NJJf4Mm"
   },
   "source": [
    "#### Freehand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "35glfuPRf5ef"
   },
   "source": [
    "**Step 1 - Call the Image show function from plotly**\n",
    "\n",
    "To make a heatmap, we first need to call the imshow() function with our plotly library (px).\n",
    "px.imshow()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RWTAC9g_gB--"
   },
   "source": [
    "\n",
    "**Step 2 - Tell what data to use for image show to create the heatmap**\n",
    "\n",
    "Inside the method, we provide the correlation matrix to show. In this case, our correlation matrix is **corrMatrix**.\n",
    "\n",
    "`px.imshow(corrMatrix)`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vmPqWCwvgE_H"
   },
   "source": [
    "\n",
    "**Step 3 - Run the code**\n",
    "\n",
    "Hit ‘control’ and ‘enter’ at the same time to run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GZpFgE_W8AMWBhq?format=png&name=360x360)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aEYX_jQAgJVs"
   },
   "source": [
    "**Your Turn**: Now you give it a try! Create your own heatmap that will give you a visual that shows the relationship between the different values in the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "id": "tG55M--qgPip",
    "outputId": "1ab64a51-7c3d-4711-f796-c1894016dfab"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script charset=\"utf-8\" src=\"https://cdn.plot.ly/plotly-2.35.2.min.js\"></script>                <div id=\"c982eb3a-c855-48a5-869d-4fef0fc0812b\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"c982eb3a-c855-48a5-869d-4fef0fc0812b\")) {                    Plotly.newPlot(                        \"c982eb3a-c855-48a5-869d-4fef0fc0812b\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"x\":[\"TrafficVolume\",\"AverageSpeed\",\"CO2Emissions\",\"NoiseLevel\"],\"y\":[\"TrafficVolume\",\"AverageSpeed\",\"CO2Emissions\",\"NoiseLevel\"],\"z\":[[1.0,-0.10936924994867558,0.8717541573027857,0.8179536333719912],[-0.10936924994867558,1.0,-0.420516096388613,-0.3565440896138057],[0.8717541573027857,-0.420516096388613,1.0,0.9627570970491449],[0.8179536333719912,-0.3565440896138057,0.9627570970491449,1.0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}\\u003cbr\\u003ey: %{y}\\u003cbr\\u003ecolor: %{z}\\u003cextra\\u003e\\u003c\\u002fextra\\u003e\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\"},\"coloraxis\":{\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]},\"margin\":{\"t\":60}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('c982eb3a-c855-48a5-869d-4fef0fc0812b');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "px.imshow(corrMatrix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-i9faWjQgV-n"
   },
   "source": [
    "**Explanation**: *Bolded The heatmap generated by px.imshow(corrMatrix) visually represents the correlation matrix, providing a clear and intuitive illustration of the strong positive relationships between Traffic Volume and CO2 Emissions/Noise Level, as well as the negative relationships between Average Speed and these pollutants*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MmTkgnDEgbTA"
   },
   "source": [
    "## Correlations for Ordinal Data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NpVUTnbZgc9-"
   },
   "source": [
    "\n",
    "What happens if the variable types are a bit different? For ordinal data there are measures of correlation based on ranks. We’ll work through this so their ranks replace the original variables.\n",
    "\n",
    "*Relationship between Ordinal Data*\n",
    "\n",
    "**Spearman’s rank correlation coefficient** uses rank numbers to help us find out how different ordinal variables are related. Remember **Pearson’s correlation coefficient** from before? We can't use Pearson correlation for ordinal data directly, but we can make it work with a trick we convert the ordinal values to ranks.\n",
    "\n",
    "For example, if our ordinal data is coded as small, medium,  and large, we can recode it in ranks as 1, 2, 3. Magic! Then we can calculate Pearson correlation with these ranks. This idea is so useful it has a special name, Spearman's rank coefficient. You can use Spearman's for ordinal variables or even for numeric variables with wild distributions.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f7YdVsPDgoN-"
   },
   "source": [
    "### Goal 6 - Calculate Correlations for Ordinal Data\n",
    "\n",
    "In our dataset, we got all different kinds of data types that we talked about in our Data Science and the Nature of Data notebook. As we look to help Kiana, let’s see if we can explore other relationships\n",
    "\n",
    "By default, the Pearson correlation method is used when we call the corr() method, which is really helpful when we have ratio data.\n",
    "\n",
    "But that’s not what we use when it comes to ordinal data. In order to use the Spearman correlation method, we can use the “method” parameter to define which correlation method we want to use.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F7AKoBkzg172"
   },
   "source": [
    "#### Blockly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8j1F1ZVYg3WA"
   },
   "source": [
    "\n",
    "**Step 1 - Write out the variable name you want to use for the correlation matrix**\n",
    "\n",
    "Create a new variable named **corrMatrix** and drag it into the VARIABLES menu. This variable will hold the correlation matrix calculated from the dataframe.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qS-bKWygg5Xh"
   },
   "source": [
    "\n",
    "**Step 2 - Call the corr() method of the dataframe**\n",
    "\n",
    "To get the correlation of variables, we first need to call the corr() method to help calculate the correlations. We’ll apply it to the data we have in our **dataframe** variable.\n",
    "\n",
    "From the VARIABLES menu, set the **corrMatrix** variable to the result of WITH the **dataframe** variable DO the corr function."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SGIRQlzMg7Jx"
   },
   "source": [
    "\n",
    "\n",
    "**Step 3 - Add spearman correlation method in corr() function**\n",
    "\n",
    "Then, we should pass the method argument to the corr function with the value 'spearman'. This specifies the correlation method to use, **method='spearman'**. To do this, using FREESTYLE, pass the method argument to the corr function with the value 'spearman'.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s-lyCGlKg9Vm"
   },
   "source": [
    "\n",
    "**Step 4 - Run correlations for only numeric values**\n",
    "\n",
    "We tell our code **numeric_only=True** (FREESTYLE menu), which means we only want to consider numeric columns in the calculation.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZIQCzLEog-J6"
   },
   "source": [
    "\n",
    "**Step 5 - Assign the correlation matrix to the variable you created**\n",
    "\n",
    "Finally, we need to store the correlation into a variable for later use. Here, we store the correlation into the variable **corrMatrix**.\n",
    "\n",
    "We will do this through Set **corrMatrix** variable to the **dataframe** DO **corr** block.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4F-1YXWmg_V0"
   },
   "source": [
    "\n",
    "**Step 6 - Display the correlation matrix**\n",
    "\n",
    "Lastly, we display the correlation variable. To do this, we drag the **corrMatrix** variable to the workspace, making it available for further use in our program. This step is more of a visualization step, allowing us to see the variable in the Blockly workspace.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tSp59YAphAID"
   },
   "source": [
    "\n",
    "**Step 7 - Connect the blocks to run the code**\n",
    "\n",
    "Connect the blocks and run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GaNPPz1XEAABJXF?format=png&name=small)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#blocks code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2lxIEaz1hoIy"
   },
   "source": [
    "#### Freestyle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lj-U9Jo2huTR"
   },
   "source": [
    "\n",
    "**Step 1 - Call the corr() method of the dataframe**\n",
    "\n",
    "To get the correlation of variables, we first need to call the corr() method to help calculate the correlations. We’ll apply it to the data we have in our **dataframe** variable.\n",
    "\n",
    "`dataframe.corr()`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kJapn6PShwNu"
   },
   "source": [
    "\n",
    "**Step 2 - Run correlations for only numeric values**\n",
    "\n",
    "The method will have one parameter **numeric_only=True** , which means showing correlations for only numeric values\n",
    "\n",
    "`dataframe.corr(numeric_only=True)`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-IGDGed-hxCW"
   },
   "source": [
    "\n",
    "**Step 3 -  Add spearman correlation method in corr() function**\n",
    "\n",
    "Then, we should pass the method argument to the corr function with the value 'spearman'. This specifies the correlation method to use, **method='spearman'**\n",
    "\n",
    "`dataframe.corr(method=”spearman”, numeric_only=True)`\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cCdCiaJWhxt-"
   },
   "source": [
    "\n",
    "**Step 4 - Assign the correlation matrix to the variable you created**\n",
    "\n",
    "Finally, we need to store the correlation into a variable for later use. Here, we store the correlation into the variable **corrMatrix**\n",
    "\n",
    "`corrMatrix= dataframe.corr(method=”spearman”, numeric_only=True)`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UFgLswiDhyb9"
   },
   "source": [
    "\n",
    "**Step 5 - Print the correlation matrix**\n",
    "\n",
    "Lastly, we print the correlation variable.\n",
    "\n",
    "`print(corrMatrix)`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "viSgnK8mhzZ5"
   },
   "source": [
    "\n",
    "\n",
    "**Step 6 - Run the code**\n",
    "\n",
    "Hit ‘control’ and ‘enter’ at the same time to run the code!\n",
    "\n",
    "\n",
    "![](https://pbs.twimg.com/media/GZpGTe1W0Ac89mL?format=png&name=small)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Qc0ke9fTicbm"
   },
   "source": [
    "**Your Turn**: Now it’s your turn! Start by applying the Pearson correlation method to the data, and after that you’ll add Spearman’s method into the mix!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "u1P9Qdv6idsu",
    "outputId": "25fa6c0a-3db8-4501-8278-fae8109bcc72"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "               TrafficVolume  AverageSpeed  CO2Emissions  NoiseLevel\n",
      "TrafficVolume       1.000000     -0.159457      0.881386    0.834421\n",
      "AverageSpeed       -0.159457      1.000000     -0.303421   -0.277511\n",
      "CO2Emissions        0.881386     -0.303421      1.000000    0.936003\n",
      "NoiseLevel          0.834421     -0.277511      0.936003    1.000000\n"
     ]
    }
   ],
   "source": [
    "corrMatrix = dataframe.corr(method='spearman',numeric_only=True)\n",
    "print(corrMatrix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F9IHEnrtid2l"
   },
   "source": [
    "**Explanation** *The Spearman rank correlation matrix reveals that traffic volume is strongly and positively correlated with CO2 emissions and noise level, indicating a monotonic increasing relationship between these variables. In contrast, Average Speed is negatively correlated with CO2 Emissions and Noise Level, suggesting that emissions and noise levels tend to decrease as speed increases. The strong correlation between CO2 Emissions and Noise Level implies that these two variables are closely related and may be driven by similar factors. Overall, the correlations suggest that Traffic Volume is a crucial driver of environmental pollution, while Average Speed may have a mitigating effect*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PHlIqrlXjGPo"
   },
   "source": [
    "\n",
    "## Correlations for Categorical Data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JlkWz0rtjGZJ"
   },
   "source": [
    "\n",
    "We’ve talked about associations in numbers, but what if we wanted to know how data in specific categories are related? We have a few tools that we can use to understand these relationships called tables. The first table is a **contingency table**. Contingency tables are helpful to use when you’re looking at data that only has two possible outcomes.\n",
    "\n",
    "|             | **Pass**    | **Fail**    |\n",
    "| ----------- | ----------- | ----------- |\n",
    "| Attended    | 25          | 5           |\n",
    "| Skipped     | 5           | 10          |\n",
    "\n",
    "\n",
    "\n",
    "To understand the relationship between the two categories, we need to know if there actually is one. For example, if we wanted to know if skipping a class (one category) has an effect on passing the same class (another category), we could put this in a table - passing/failing and skipping/attending. We would use the number of students who skipped and passed and those who skipped and failed and compare these numbers to the students who attended and passed and those who attended and failed. When we add these numbers together and compare them, the totals can help us predict what will happen when other students skip or attend class.\n",
    "\n",
    "So can we use data science to measure that relationship? The chi-square statistic comes in handy. Rather than eyeball or assume there is a difference, the **chi-square statistic** will tell us that the difference is statistically significant.\n",
    "\n",
    "So let’s dive in!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ksqW2IYjkq31"
   },
   "source": [
    "### Goal 7:  Calculate Correlations for Categorical Data\n",
    "\n",
    "You’ve now mastered ratio and ordinal data. You might be thinking “Well, what about categorical data?”. Let’s see how we can use crosstabs to explore categorical data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "K73uTj8Hlekd"
   },
   "source": [
    "#### Blockly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4TMNNUQHlgGF"
   },
   "source": [
    "\n",
    "**Step 1 - Write out the variable name you want to use for our contingency table**\n",
    "\n",
    "Create a new variable named **contingencyTable** and drag it into the VARIABLES menu. This variable will hold the correlation matrix calculated from the dataframe."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6VrHEwO8lhu8"
   },
   "source": [
    "\n",
    "\n",
    "**Step 2 - Call the crosstab() function**\n",
    "\n",
    "To create a contingency table, we first need to call the crosstab() method. From the VARIABLES menu, set the **contingencyTable** variable to the result of WITH **pd** DO the crosstab function.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xCoO-vItljG4"
   },
   "source": [
    "\n",
    "**Step 3 - Add parameters to the crosstab() function**\n",
    "\n",
    "In the next step, let’s look at what variables (column) to compare in our crosstab() method.\n",
    "\n",
    "Now, from the LISTS menu, get a *dictVariable*. Change it to **dataframe** and inform the name of the column \"TrafficCondition\" (use Quotes from the TEXT menu). Add it as a parameter of the crosstab function. To add a second parameter to it, repeat the process for the column \"Road Type\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zDsbQaPTlkAs"
   },
   "source": [
    "\n",
    "\n",
    "**Step 4 - Assign the crosstab to the variable you created**\n",
    "\n",
    "Finally, we need to store the contingency table result into a variable for later use. Here, we store it in the variable “contigencyTable”. From the VARIABLES menu, set the **contingencyTable** variable to the result of WITH **pd** DO the **crosstab** function.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FcQLajZ8lkyT"
   },
   "source": [
    "\n",
    "**Step 5 - Print the contingency table**\n",
    "\n",
    "From the TEXT menu, print the **contingencyTable** variable to the console, displaying the calculated contingency table.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dlI2Z-eCllcn"
   },
   "source": [
    "\n",
    "**Step 6 - Connect the blocks to run the code**\n",
    "\n",
    "Connect the blocks and run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/GaspkPFWQAAVv_n?format=jpg&name=medium)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#blocks code\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B3pqeiufmKhO"
   },
   "source": [
    "#### Freehand"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aQXf1pzynX8S"
   },
   "source": [
    "\n",
    "**Step 1 - Call the crosstab() function**\n",
    "\n",
    "To create a contingency table, we first need to call the **crosstab**() method.\n",
    "\n",
    "`pd.crosstab()`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QC_V-6QznZjX"
   },
   "source": [
    "\n",
    "**Step 2 -  Add parameters to the crosstab() function**\n",
    "\n",
    "In the next step, let’s look at what variables (column) to compare in our crosstab() method.The parameters for this table will be\n",
    "dataframe[‘TrafficCondition’] - the columns representing different traffic conditions\n",
    "dataframe[‘Road Type’] - the column representing different types of roads\n",
    "\n",
    "`pd.crosstab(dataframe[‘TrafficCondition’], dataframe[‘Road Type’])`\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "p1xnKLyjnaeu"
   },
   "source": [
    "\n",
    "**Step 3 - Assign the crosstab to the variable you created**\n",
    "\n",
    "Finally, we need to store the contingency table result into a variable for later use. Here, we store it in the variable **“contigencyTable”**.\n",
    "\n",
    "`contigencyTable= pd.crosstab(dataframe[‘TrafficCondition’], dataframe[‘Road Type’])`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ot0jxg6MnbOR"
   },
   "source": [
    "\n",
    "**Step 4 -  Print the contingency table**\n",
    "\n",
    "Lastly, we print the correlation variable.\n",
    "\n",
    "`print(contigencyTable)`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CeKdkcTYnb3B"
   },
   "source": [
    "\n",
    "**Step 5 - Run the code**\n",
    "\n",
    "Hit ‘control’ and ‘enter’ at the same time to run the code!\n",
    "\n",
    "![](https://pbs.twimg.com/media/Gasq8wuXcAAxTN-?format=jpg&name=medium)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SD_HmhUcoIpL"
   },
   "source": [
    "**Your Turn**: Now it’s your turn! Start by creating a variable for the contingency table. After you’ve added parameters, you’ll use the cross-tabulation method to get the results.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "tACIEsxuoLst",
    "outputId": "d4afc123-7e9e-4b84-ee31-e63b2728e0ce"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Road Type         City  Highway  Rural\n",
      "TrafficCondition                      \n",
      "Free Flow            3        3     44\n",
      "Heavy               44        3      3\n",
      "Moderate             6       38      6\n"
     ]
    }
   ],
   "source": [
    "contingencyTable = pd.crosstab(dataframe['TrafficCondition'],dataframe['Road Type'])\n",
    "print(contingencyTable)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jv7pYgMxoOk1"
   },
   "source": [
    "**Explanation** *The correlation matrix and contingency table provide insights into the relationships between traffic-related variables. The correlation matrix (not shown) likely reveals correlations between continuous variables, while the contingency table shows the distribution of Traffic Condition and Road Type. The contingency table indicates that most Free Flow traffic conditions occur on Rural roads, while Heavy traffic conditions are more common on City roads. Moderate traffic conditions are more evenly distributed across Road Types, with a slight majority on Highway roads. These findings suggest that Road Type is associated with different Traffic Conditions, with Rural roads tending to have lighter traffic and City roads experiencing heavier traffic*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-pLtUobGoSTc"
   },
   "source": [
    "\n",
    "## WHAT DID YOU LEARN?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iomlfG7-oUYe"
   },
   "source": [
    "\n",
    "[Correlations](## \"seeing how two different variables are related\") describe the strength and direction of a relationship between two variables. In this lesson, we learned about two important concepts called [Pearson's correlation coefficient](## \"measures linear relationships between two variables\") and [Spearman's rank correlation coefficient](## \"computes a measure of association using ranks\"). It helps determine whether and how strongly pairs of variables are related. We learned how to figure out coefficients and what they mean.\n",
    "\n",
    "We also learned that these tools have certain assumptions and limitations, which means they don’t work perfectly in every situation. We also explored how to visualize these relationships using [heatmaps](## \"a visual representation that uses colors to show the magnitude of values in a dataset\") and [contingency tables](## \"a type of table used to show the frequency distribution of variables\") for categorical data analysis.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-cUkaxCqoW4a"
   },
   "source": [
    "## WHAT’S NEXT?\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "seJ3GS0foYaC"
   },
   "source": [
    "[Clustering](Clustering.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7TYeizYxoqCw"
   },
   "source": [
    "## TELL ME MORE\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xiqmMVnRoq-K"
   },
   "source": [
    "- [Datawhys Measures of Association Notebook](https://github.com/memphis-iis/datawhys-content-notebooks-python/blob/master/Measures-of-association.ipynb)\n",
    "- [Datawhys Measures of Association Problem-Solving Notebook](https://github.com/memphis-iis/datawhys-content-notebooks-python/blob/master/Measures-of-association-PS.ipynb)\n",
    "- [What is Correlation?](https://youtu.be/PEfQCv9nvSo?si=q76qtB4STTqK8ZLY) - Simplilearn (video)\n",
    "- [The (Pearson) Correlation Coefficient Explained in One Minute](https://youtu.be/WpZi02ulCvQ?si=spmoPwDI6TxrBdHO) - One Minute Economics (video)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}