Skip to content
Deepesh Pathak edited this page Jun 13, 2018 · 1 revision

Origami Demo Creation Pipeline

Introduction

Origami is an AI-as-a-service solution that allows researchers to easily convert their deep learning models into an online service that is widely accessible to everyone without the need to set up the infrastructure, resolve the dependencies, and build a web service around the deep learning model. Demo Creation Pipeline This design doc deal with the implementation of new demo creation pipeline for Origami. The demo creation pipeline will have the following three stages on a high level

  • Bundling user model code with origami-lib
  • Demo deployment.
  • Demo access.

Basic idea behind Origami

Origami-lib ideology

Bundling user model code with origami-lib

The new demo creation pipeline is aimed at abstracting away the whole procedure related to the deployment of a machine learning model from the user, helping the user to focus entirely on his part of developing the machine learning model.

Flow

New Demo creation pipeline for origami

The new demo creation pipeline will have the above-given structure, at the end of this pipeline the user will have a complete package through which he can deploy his machine learning model and then can in turn access it via a web interface using origami.

Demo Deployment

After the first step the user will have a bundled directory with his code wrapped around origami-lib ready to be deployed. The directory structure at this moment will look like

The Dockerfile here is used for deploying demo in a container. Origami demos by default will run on port 6000 these ports inside the container will be mapped to actual ports while deploying the demo.

For the deployment of the demo on cloud we have two options

  • Deployment on CloudCV servers.
  • Deployment on users own server.

The deployment procedure for a demo is assisted by docker which eases the whole process for origami user. Since CloudCV does not have the computational power to deploy all the demos, user can use their own servers to deploy demos.

Demo deployment on CloudCV servers

Once the demo bundle has been formed, the next step is the deploy the demo. In order to deploy demo on CloudCV servers the user will follow the above given pipeline. We will use Github for remote repository management where the users bundled code will be present. Once the user has created a demo on the website he will be provided with a deploy button associated with the demo. This will trigger demo deployment.

On triggering the deployment, first Origami will check if Origami has been authorized by user for GIthub Oauth. If not user will be taken to CloudCV/Origami github authorization page. Origami will use standard Github oauth procedure(https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/) to get the user access token. This token will then be further used to interact with users Github repository.

After each new code push to the repository user will be required to trigger the demo deploy using website. We can later also extend github webhooks to customize automatic deployment of demos. Once the demo build is triggered and github oauth access has been verified. Django server running will trigger the deploy procedure using origami-daemon. The reason for using a separate service for this rather than using django-server itself is that, it might be possible that we want to deploy demos on some other server than on which our website is running, since both these services will require drastically different processing power.

This daemon will be running in background on the server will be listening on a port say(6001), now django server will make a request to origami-daemon with the command deploy_trigger and with parameters - user_access_token, user_id, remote_repo_link Each demo will have a unique identity calculated using md5(user_id + remote_repo_link). origami-daemon will maintain a queue and will push the action to queue, then when executing the action it will go through the following process:

  • Check if a demo container is already running using the unique identity of the demo and a stats file/db we will maintain to keep track containers running demo(more on this later). If the demo is running then kill the container and prune the image.
  • Pull the repository into a temporary directory, we will create a docker container based on the check if .origami is present in the repository root(This is for further extension, where we can use this to use or pass extra information) for it to be deployed.
  • If no .origami is present update the stats with not_linked.

Once we have the bundled demo code, origami-daemon will build an image using the Dockerfile in demo root. Once an image is build, daemon will spin the container mapping it to an external port which is free using stats file.

Each demo will have a log file associated, which will be present on the server in a directory with demo_id.log name. This will be mounted on the docker container as volume and all the logs from demo will be writterned to this. Once the demo is deployed origami-daemon will change the status in stats file/db to running and will notify request origin(django server) about the status.

Stats DB/File will have the following structure user_id : Origami user ID the demo is associated with container_id: Container/Demo unique ID calculated above. port: Exposed port the container is bound to. repolink/origami_demo_id: Repo link for the demo. status: Status of the demo -> container running/stopped/not_linked We can expose this data from either a json formatted file or from a local sqlite db.

Origami-daemon provide access to these stats and logs from the demo container to django server. Any idea how we can identify during request from django-server to daemon that it is actually django server. I was thinking of a TLS enabled connection where django server will be provided with a secret for identification. If anyone has a better option I would really love to hear.

Deployment on user’s server

Demo access

Demo access will be through Origami web interface. For demo deployed on user’s server there is not much to think of since we can’t control the access and resources. On CloudCV server we will have limited resources in terms of processing power so demo access will be made using a queue maintained on the server. This queue will be maintained by origami-daemon itself. The reason being:

  • There will be no race cases when using the same daemon since the demo is being redeployed can only be accessed once its redeployed.
  • Running two scripts is not a wise idea when we can incorporate the feature into a single one itself. The remaining flow for access of demo including the web UI will remain same.

Status

In Progress :)

For a better view of the doc look here