Deployment_Guide.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "02_Deployment.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RJQ5_qEdyLvz",
        "colab_type": "text"
      },
      "source": [
        "# 02 - Deployment\n",
        "\n",
        "* [Starlette](https://www.starlette.io/)\n",
        "* Follow the `Render` tutorial [here](https://github.com/render-examples/fastai-v3)\n",
        "\n",
        "* **Note**: You do not **need** to deploy on Render to get the code working, we can test locally on our machine! (which we will do today)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J-FaHcuTzdq8",
        "colab_type": "text"
      },
      "source": [
        "## What will we focus on?\n",
        "\n",
        "* Looking at how to format inputs/outputs for each model type and feeding it in. \n",
        "* Images, Tabular, NLP\n",
        "\n",
        "## What code do we change?\n",
        "* `server.py`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wo-u77GqzwHb",
        "colab_type": "text"
      },
      "source": [
        "# Images:\n",
        "\n",
        "* Different input types:\n",
        "  * URL\n",
        "  * File upload"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6_Y8FVlL4TxD",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "async def get_bytes(url):\n",
        "  async with aiohttp.ClientSession() as session:\n",
        "    async with session.get(url) as response:\n",
        "      return await response.read()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EiCTLwlQ4AfC",
        "colab_type": "text"
      },
      "source": [
        "An image upload"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oanc7RJbx642",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "@app.route('/analyze', methods=['POST'])\n",
        "async def analyze(request):\n",
        "  img_data = await request.form()\n",
        "  img_bytes = await (img_data['file'].read())\n",
        "  pred = learn.predict(BytesIO(img_bytes))[0]\n",
        "  return JSONResponse({\n",
        "      'results': str(pred)\n",
        "  })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iNr3NlPB4CIQ",
        "colab_type": "text"
      },
      "source": [
        "A URL"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "noh6yk1a4C2A",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "@app.route('/analyze', methods=['POST'])\n",
        "async def analyze(request):\n",
        "  img_bytes = await get_bytes(request.query_params[\"url\"])\n",
        "  pred = learn.predict(BytesIO(img_bytes))[0]\n",
        "  return JSONResponse({\n",
        "      'results' : str(pred)\n",
        "  })"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pe591V9T6FCy",
        "colab_type": "text"
      },
      "source": [
        "A zip file (see below on how to upload a `zip` or other file"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1rp6U5jK6Ghf",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import zipfile\n",
        "import csv\n",
        "\n",
        "@app.route('/analyze', methods=['POST'])\n",
        "async def analyze(request):\n",
        "  data = await request.form()\n",
        "  content = data['content']\n",
        "  zip_ref = zipfile.ZipFile(content, 'r')\n",
        "  mkdir('Downloaded_Images')\n",
        "  zipref.extractall('Downloaded_Images')\n",
        "  zip_ref.close()\n",
        "  path = Path('Downloaded_Images')\n",
        "  imgs = get_image_files(path)\n",
        "  learn = load_learner(path/export_file_name)\n",
        "  dl = test_dl(learn.dbunch, imgs)\n",
        "  _, __, preds = learn.get_preds(dl=dl, with_decoded=True)\n",
        "  rm -r 'Downloaded_Images'\n",
        "  resultsFile = open('results.csv', 'wb')\n",
        "  wr = csv.writer(resultsFile)\n",
        "  wr.writerows([preds])\n",
        "  return FileResponse('results.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LHVBE3G47Y1y",
        "colab_type": "text"
      },
      "source": [
        "Parsing a csv with image urls"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RzxsDi387amI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import csv\n",
        "import StringIO\n",
        "\n",
        "@app.route('/analyze', methods=['POST'])\n",
        "async def analyze(request):\n",
        "  data = await request.form()\n",
        "  content = await (data['file'].read())\n",
        "  s = str(content, 'utf-8')\n",
        "  data = StringIO(s)\n",
        "  mkdir('Downloaded_Images')\n",
        "  download_images('Downloaded_Images', urls=data)\n",
        "  path = Path('Downloaded_Images')\n",
        "  learn = load_learner(path/export_file_name)\n",
        "  imgs = get_image_files(path)\n",
        "  dl = test_dl(learn.dbunch, imgs)\n",
        "  _, __, preds = learn.get_preds(dl=dl, with_decoded=True)\n",
        "  rm -r 'Downloaded_Images'\n",
        "  resultsFile = open('results.csv', 'wb')\n",
        "  wr = csv.writer(resultsFile)\n",
        "  wr.writerows([preds])\n",
        "  return FileResponse('results.csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4ma0IvVA5FYH",
        "colab_type": "text"
      },
      "source": [
        "# Tabular"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ggo5cmwm8hNe",
        "colab_type": "text"
      },
      "source": [
        "Tabular is different. Most work will be done by sending large chuncks of data for analysis. Let's recreate what we did, but load it into Pandas"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hLJsXSMY5Gul",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import StringIO\n",
        "import csv\n",
        "\n",
        "@app.route('/analyze', methods=['POST'])\n",
        "async def analyze(request):\n",
        "  data = await request.form()\n",
        "  content = await (data['file'].read())\n",
        "  s = str(content, 'utf-8')\n",
        "  data = StringIO(s)\n",
        "  df = pd.read_csv(data)\n",
        "  learn = load_learner(path/export_file_name)\n",
        "  # if we want to do GPU:\n",
        "  # learn.model = learn.model.cuda()\n",
        "  dl = learn.dbunch.train_dl.new(df)\n",
        "  _, __, y = learn.get_preds(dl=dl, with_decoded=True)\n",
        "  df['Predictions'] = y\n",
        "  # if we want to store the results\n",
        "  path_res = Path('app/static/')\n",
        "  df.to_csv(path_res/'results.csv')\n",
        "\n",
        "  return FileResponse('results.csv', media_type='csv')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nH4qfhb--HlF",
        "colab_type": "text"
      },
      "source": [
        "We need to adjust the JavaScript to accept a form:\n",
        "\n",
        "`client.js`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e3KGo_MZ-MOJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "function analyze(){\n",
        "    var uploadFiles = el('file-input').files;\n",
        "    if (uploadFiles.length < 1) alert('Please select 1 file to analyze!');\n",
        "\n",
        "    el('analyze-button').innerHTML = 'Analyzing...';\n",
        "    var xhr = new XMLHttpRequest();\n",
        "    var loc = window.location\n",
        "    xhr.open('POST', `${loc.protocol}//${loc.hostname}:${loc.port}/analyze`, true);\n",
        "    xhr.onerror = function() {alert (xhr.responseText);}\n",
        "    xhr.onload = function(e) {\n",
        "        if (this.readyState === 4) {\n",
        "          el(\"result-label\").innerHTML = `Result = Good`;\n",
        "          \n",
        "          download('results.csv', 'results.csv');\n",
        "          xhr.send();\n",
        "        }\n",
        "        el(\"analyze-button\").innerHTML = \"Analyze\";\n",
        "      };\n",
        "\n",
        "      var fileData = new FormData();\n",
        "      fileData.append(\"file\", uploadFiles[0]);\n",
        "      xhr.send(fileData);\n",
        "    }\n",
        "  }"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pKZsDRx1_a5m",
        "colab_type": "text"
      },
      "source": [
        "# Text\n",
        "\n",
        "To write a simple function for text based models:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "61OsZ_1q_gmV",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "@app.route('/analyze', methods=['POST'])\n",
        "async def analyze(request):\n",
        "  data = await request.form()\n",
        "  content = data['content']\n",
        "  pred = learn.predict(content)[0]\n",
        "  return JSONResponse({'result': pred})"
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}