Skip to content

Deployment Using Docker Swarm

Jump to bottom
Brian Ball edited this page Oct 24, 2023 · 6 revisions

Deprecation Warning

With the 3.1 release, cloud-based deployment of OpenStudio Server will be updated to leverage the widely adopted open source tools, Kubernetes and Helm. A major benefit of this change will be to provide a deployment pathway for multiple cloud providers, including but not limited to AWS. Additional details related to this update are available on the wiki. This 3.0 release is the final release of OpenStudio Server that will include code to start and stop AWS remote instances. The openstudio-server-helm repo provides documentation and scripts to facilitate deployment and debugging of OSS on a variety of cloud platforms.

Prerequisites

This document assumes that you have installed docker for your platform.

Docker-based deployment is only recommended for teams that have general understanding of web application stacks and specific expertise running Docker, debugging Docker Swarm, and administering Linux systems.

Deploying with Docker Swarm

OpenStudio Server can be deployed to machines running Docker.

  1. Scripts and sample config files for docker swarm setup can be found at local docker scripts.

  2. Modify the docker-compose.yml for the host machine. This defines the Docker stack to deploy. These instructions assume the file is present at $HOME/docker-compose.yml. The file should be created by copying docker-compose.aws.yml and updating all image names to match docker-compose.deploy.yml. To use a specific version of OpenStudio Server, update the :latest tag to appropriate version. For example image: nrel/openstudio-server:2.7.0.

  3. log in to the target machine.

  4. Initialize Docker Swarm:

$ docker swarm init
Swarm initialized: current node (ud4ke0ytm8w9odxmj95a0meqs) is now a manager.
<remainder omitted>
  1. Set the environment variables used for configuring the deployment. Suggested values are:
  • AWS_MAX_REQUESTS: round((number_of_cpus + 10) * 1.2)
  • AWS_MONGO_CORES: ceil(number_of_cpus / 64)
  • AWS_WEB_CORES: ceil(number_of_cpus / 32)
  • AWS_MAX_POOL: 16 * WEB_CORES
  • AWS_REX_MEM: 512 * MAX_POOL
  • AWS_OS_SERVER_NUMBER_OF_WORKERS: number_of_cpus - WEB_CORES - MONGO_CORES - 3
  1. Deploy the stack
$ docker stack deploy osserver --compose-file=$HOME/docker-compose.yml
Creating service osserver_worker
Creating service osserver_rserve
Creating service osserver_db
Creating service osserver_web
Creating service osserver_web-background
Creating service osserver_queue
  1. Confirm that stack is running:
$ docker service ls
ID                  NAME                      MODE                REPLICAS            IMAGE                           PORTS
ubq7vw1ikcuj        osserver_db               replicated          1/1                 mongo:6.0.7                    *:27017->27017/tcp
i3mw78ht7ao1        osserver_queue            replicated          1/1                 redis:6.0.9                    *:6379->6379/tcp
fx0cz4rogbdn        osserver_rserve           replicated          1/1                 nrel/openstudio-rserve:latest
i067gzxmrnde        osserver_web              replicated          1/1                 nrel/openstudio-server:latest   *:80->80/tcp, *:443->443/tcp
fxt4x9haeamd        osserver_web-background   replicated          1/1                 nrel/openstudio-server:latest
xbk3bw4iaakc        osserver_worker           replicated          1/1                 nrel/openstudio-server:latest
  1. Wait until web application is responding on port 80 and then scale workers.
$ while ( nc -zv 127.0.0.1 80 3>&1 1>&2- 2>&3- ) | awk -F ":" '$3 != " Connection refused" {exit 1}'; do sleep 5; done

$ docker service scale osserver_worker=$AWS_OS_SERVER_NUMBER_OF_WORKERS
osserver_worker scaled to 42

All set! When you are done, remove the stack:

$ docker stack rm osserver
Removing service osserver_db
Removing service osserver_queue
Removing service osserver_rserve
Removing service osserver_web
Removing service osserver_web-background
Removing service osserver_worker
Removing network osserver_default

Debugging

Besides running out of storage space on the device, the following sections aim at providing help on challenges faced by the majority of users. Contributions to this section are welcomed.

Resetting the docker environment

First all self-healing deployments must be stopped by terminating all stacks and services, which will allow exiting swarm mode.

$ docker stack ls
NAME                SERVICES
osserver            5

$ docker stack rm osserver
Removing service osserver_db
Removing service osserver_rserve
Removing service osserver_web
Removing service osserver_web-background
Removing service osserver_worker
Removing network osserver_default

$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
nd3zz8c0y4rn        registry            replicated          1/1                 registry:2.6        *:5000->5000/tcp

$ docker service rm $(docker service ls -q)
nd3zz8c0y4rn

$ docker swarm leave -f
Node left the swarm.

Next, remove all local artifacts, beginning with containers, followed by volumes, and ending with images.

$ docker ps -a
CONTAINER ID        IMAGE                                     COMMAND                  CREATED             STATUS                       PORTS               NAMES
e2bcca7269ea        127.0.0.1:5000/openstudio-server:latest   "rails-entrypoint ..."   2 days ago          Exited (137) 8 minutes ago                       osserver_web.1.xmrm7r647gl3x7b7eehkkagma

$ docker rm -f $(docker ps -aq)
e2bcca7269ea

$ docker volume ls
DRIVER              VOLUME NAME
local               dbdata
local               osdata
local               regdata

$ docker volume rm $(docker volume ls -q)
dbdata
osdata
regdata

$ docker image rm -f $(docker image ls -aq)
<output omitted>

$ docker image ls -a
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

Finally, examine the docker daemon configuration and confirm each option is specified as desired with the official docker daemon command line reference. In the example below, the configuration options to validate are -H fd://. The default options for AWS deployments are specified in the DOCKERD_OPTIONS environment variable in the AMI build template.

$ ps aux | grep dockerd
root      1234  x.y  m.n 1234567 123456 ?      Ssl  Jan01  01:23 /usr/bin/dockerd -H fd://

Sample script for restarting the OpenStudio Server

This script will automate restarting the OpenStudio Server application if it has not crashed unexpectedly. If it has, please follow the instructions in the reset section.

#!/bin/bash -e
cd $HOME
docker stack rm osserver
while [ $(docker ps -q | wc -l) != 1 ]; do sleep 5; done
sleep 10
docker volume rm osdata
docker volume rm dbdata
docker stack deploy osserver --compose-file=$HOME/docker-compose.yml
while ( nc -zv 127.0.0.1 80 3>&1 1>&2- 2>&3- ) | awk -F ":" '$3 != " Connection refused" {exit 1}'; do sleep 5; done
docker service scale osserver_worker=42
echo 'osserver stack redeployed'
Clone this wiki locally