Skip to content
This repository has been archived by the owner on Jul 11, 2023. It is now read-only.

142 docker improvements #147

Merged
merged 9 commits into from
Jul 18, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# add git-ignore syntax here of things you don't want copied into docker image

.git
*Dockerfile*
*docker-compose*
node_modules
13 changes: 0 additions & 13 deletions .env-example

This file was deleted.

8 changes: 1 addition & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,4 @@ coverage
.DS_STORE
npm-debug.log

# Docker stuff
.env
docker/*.crt
docker/*.key
docker/*.srl
docker/*.csr
.vscode

17 changes: 17 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
// Use IntelliSense to learn about possible Node.js debug attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Attach 9229 --inspect",
"type": "node",
"request": "attach",
"protocol": "inspector",
"port": 9229,
"localRoot": "${workspaceRoot}",
"remoteRoot": "/usr/src/app"
}
]
}
56 changes: 35 additions & 21 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,22 +1,36 @@
# Copyright (c) 2016 code4hr <[email protected]> (http://code4hr.org/)
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
FROM node:alpine
FROM node:6.11
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to use the Alpine image here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also use the alpine version of the node Docker image.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry guys, just realized I had custom notifications for CfA repos to goto my CfA email which I really only check weekly. 😨

Short answer is we don't care as much about tiny single-instance images as we do about ease of use, and with alpine not using glibc or apt-get it'll make it trickier and a higher learning curve for new people. For example: I was getting build failures on npm install in alpine due to bugs unique to alpines c complier. Using default node skips all those issues.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really? What kinds of issues?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most common is npm module build failures that require c compiler with some issue related to their lack of glibc. Things are getting better all the time, but doesn't mean we need to be bleeding edge in this area for little benefit


RUN apk update && apk add python make g++
RUN mkdir -p /usr/src/app

# set our node environment, either development or production
# defaults to production, compose overrides this to development on build and run
ARG NODE_ENV=production
ENV NODE_ENV $NODE_ENV

# default to port 3000 for node, and 5858 or 9229 for debug
ARG PORT=3000
ENV PORT $PORT
EXPOSE $PORT 5858 9229

# check every 30s to ensure this service returns HTTP 200
# HEALTHCHECK CMD curl -fs http://localhost:$PORT || exit 1

# install dependencies first, in a different location for easier app bind mounting for local development
WORKDIR /usr/src
COPY package.json /usr/src/
RUN npm install && npm cache clean
# note that even with these two ENV's, node will still try to use the node_modules you
# bind-mount in with compose files or -v docker commands, so ensure you remove that subdir
# on your dev host before running docker-compose
ENV PATH /data/node_modules/.bin:$PATH
ENV NODE_PATH=/usr/src/node_modules

# copy in our source code last, as it changes the most
WORKDIR /usr/src/app
COPY . /usr/src/app

# if you want to use npm start instead, then use `docker run --init in production`
# so that signals are passed properly. Note the code in index.js is needed to catch Docker signals
# using node here is still more graceful stopping then npm with --init afaik
# I still can't come up with a good production way to run with npm and graceful shutdown
CMD [ "node", "server.js" ]
26 changes: 26 additions & 0 deletions docker-compose-proxy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
version: '3.3'
Copy link
Collaborator

@rydente rydente Jul 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't like the idea of multiple Docker Compose files, it makes the original intent of just using docker-compose up a bit more difficult when differentiating between environments.

Although I heard from a previous co-worker that at your talk, you mentioned Docker Compose eventually going away and/or only used as a development tool? Is that because of the improvements in Swarm?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're correct that Docker Compose is only intended for local development. It's preferred to use Docker Swarm instead in production.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, what about this then?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what we're talking about here.

the goal of a single docker-compose up is a local-dev-only goal. CI/CD will do all the extra things we'd need for testing and production deploy.

docker-compose cli is for dev and testing only. It doesn't understand multi-server clusters like Swarm

docker-compose.yml file is for dev, test, and prod docker Swarm. (using docker stack deploy in swarm)

We'll likely use override files with docker-compose config for production deploy, but not sure yet until we get everything needed into compose file and see what the dev-vs-prod differences are. We can't use extends, but overrides work great.

Does that help?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put another way:

The process I typically go through is making local dev as easy as possible because it's hand-typed commands by humans.

Ideally the dev should be able to install Docker, then:

git clone xxxxx
docker-compose up

Then get to work ;)

CI/CD will be automated so it can have more complex commands that combine compose files with docker-compose config and dealing with secrets, env vars, etc.

Copy link
Collaborator

@rydente rydente Jul 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, how is that the case (Compose is for local dev only) when they have explicit production environment support on the roadmap?

Not to mention, how would you do the same thing with Compose does (orchestrating lots of containers with a single configuration file) using Swarm only? Would that mean you'd have to translate the Compose config into a bash script or some sort of configuration management platform like Chef or Ansible to get this set up in production?

If that's the case, then why use Compose at all?

EDIT: I just saw docker stack deploy, so does that mean we'd use this Compose config to create a DAB, then deploy the DAB to a Swarm cluster? That would answer pretty much all my questions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're getting it. docker-compose cli and docker-compose.yml file have different scopes and purposes. The yaml file is used in dev+test+prod (everywhere). It is used by both docker-compose cli for dev/test and docker cli for prod swarm. Different parts of that file work in different scenarios.

No DAB needed in swarm, DAB is legacy and you can use compose file directly.

Local dev:

docker-compose up
That uses docker-compose.yml to start all the containers with proper settings and build stuff. build: objects under each service in compose file are used by docker-compose cli only and ignored by docker stack commands. See docs on build: in yaml

Docker Swarm Prod:

docker stack deploy -c docker-compose.yml mystackname
Will deploy all the containers with proper deploy settings like how many replicas of each container to run, healthchecks, how to update the service, etc. See deploy: docs

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also @rydente that readme for compose you referenced predates any of the modern features like Docker Swarm, and needs to be clarified. I recommend ignoring it. Eventually, the docker-compose cli is expected to go away, as its features are implemented in the docker cli, not become more important as that readme suggests.

Copy link
Collaborator

@rydente rydente Jul 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sweet, I didn't know they did away with DABs, awesome! Thanks for the info, Bret!

Re: that readme, thank you. I think someone should make a PR/issue... 🤔

HMMMMM

But how would the CLI go away? Do they plan on using a single Docker engine to make that happen, but still use the file format for the orchestration?

services:
web:
image: nginx:alpine
volumes:
- ./docker/nginx.conf:/etc/nginx/nginx.conf
- ./docker/okcandidate.crt:/etc/nginx/okcandidate.crt
- ./docker/okcandidate.key:/etc/nginx/okcandidate.key
- /data/nginx/cache
ports:
- "80:80"
- "443:443"

app:
image: rydente/node:alpine-gyp
env_file: .env
command: "ash -c 'npm i && npm start'"
working_dir: /usr/src/app
volumes:
- .:/usr/src/app
- /usr/src/app/node_modules

db:
env_file: .env
image: postgres:alpine

63 changes: 27 additions & 36 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,45 +1,36 @@
# Copyright (c) 2016 code4hr <[email protected]> (http://code4hr.org/)
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
version: '3.3'

services:
web:
image: nginx:alpine
volumes:
- ./docker/nginx.conf:/etc/nginx/nginx.conf
- ./docker/okcandidate.crt:/etc/nginx/okcandidate.crt
- ./docker/okcandidate.key:/etc/nginx/okcandidate.key
- /data/nginx/cache
ports:
- "80:80"
- "443:443"

app:
image: rydente/node:alpine-gyp
env_file: .env
command: "ash -c 'npm i && npm start'"
working_dir: /usr/src/app
image: code4hr/okc-node
build:
context: .
args:
NODE_ENV: development
entrypoint: /usr/src/app/docker/docker-entrypoint.sh
command: ../node_modules/.bin/nodemon server.js --inspect=0.0.0.0:9229
environment:
NODE_ENV: development
OKC_DB_HOST: db
OKC_DB_NAME: okcandidate_dev
OKC_DB_USER: blaine
OKC_DB_PASS: complicatedPassword
OKC_SESSION_SECRET_KEY: someGobbledygookThatIsAtLeast32CharactersLong
GOOGLE_API_KEY: google_api_key
volumes:
- .:/usr/src/app
- /usr/src/app/node_modules
ports:
- "80:3000"
- "9229:9229"

db:
env_file: .env
image: postgres:alpine
image: postgres:9.6.2-alpine
environment:
POSTGRES_DB: okcandidate_dev
POSTGRES_USER: blaine
POSTGRES_PASSWORD: complicatedPassword
ports:
- "5432:5432"


8 changes: 8 additions & 0 deletions docker/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/sh
set -e

# remove node_modules as a subdir, even if bind-mounted from host
# the correct node_modules was installed under /usr/src/node_modules
rm -rf /usr/src/app/node_modules

exec "$@"
4 changes: 4 additions & 0 deletions nodemon.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"verbose": true,
"ignore": ["dist/*"]
}
Loading