Add Dockerfile #21

intermittentnrg · 2023-10-16T16:41:29Z

A work in progress submitted for any feedback.

Since some animations take some time to render, especially with --exposure-time, and before I added #19, I thought I will run it as container from pipeline.

You may notice I don't usually write python. Submitted for early feedback.

Also it should build container from github actions and push image to github container registry.

intermittentnrg · 2023-10-16T18:42:40Z

Added GH actions workflow to build amd64 and arm64 images and push to github container registry for current repo

Docker images can be seen here https://github.com/intermittentnrg/grafanimate/pkgs/container/grafanimate

Build from workflow here: https://github.com/intermittentnrg/grafanimate/pull/3/checks

amotl

Thanks for Dockerizing grafanimate. On the publishing-to-GHCR details, I would like to use our well-established recipe as demonstrated at ¹²³⁴, unless you have any objections about it.

amotl · 2023-10-16T22:10:05Z

Hi again,

I've cherry-picked your Dockerfile into GH-24, and added our well-known recipe about OCI->GHCR publishing to it, following your intentions.

With kind regards,
Andreas.

intermittentnrg · 2023-10-16T22:19:15Z

Yes I used https://github.com/docker/build-push-action instructions and examples also, but went with parallel github worker per platform per example here https://docs.docker.com/build/ci/github-actions/multi-platform/#distribute-build-across-multiple-runners. Building arm64 images with qemu can be very slow, but seems not to be an issue for grafanimate.

Also if you have any suggestions on the Dockerfile it would be great. It's a first attempt. Should maybe switch to from ubuntu to alpine for base image...

amotl · 2023-10-16T22:26:18Z

Hi again,

Also if you have any suggestions on the Dockerfile it would be great. It's a first attempt. Should maybe switch to from ubuntu to alpine for base image...

I already started to build upon your proposal on behalf of GH-24. We discourage Alpine-based images, due to subtle differences compared to glibc. Specifically when running a huge software stack including Firefox, this is even more important.

With kind regards,
Andreas.

intermittentnrg · 2023-10-16T22:28:21Z

There is a alpine package for firefox available though. https://wiki.alpinelinux.org/wiki/Firefox

I try to use alpine if possible and have used it for python applications in production environment without issues.

amotl · 2023-10-16T22:37:29Z

We love the musl project, and it is becoming more mature every day. Still, we didn't regain enough trust, and don't bother too much about it. There are a few more pointers on this discussion - admitted, it's already from a few years ago ;].

Consider using openjdk:8-jre-alpine as base image crate/docker-crate#144

Other than potential differences between musl and glibc, the package management is also different than Debian, so we would need to re-learn, or maintain a different stack. It is not worth the time for us, we are clearly on the Debian side of "maintenance convenience and peace of mind", optimizing for re-using well-established recipes across all the projects we maintain.

amotl · 2023-10-16T22:58:42Z

Dockerfile

+RUN python3 setup.py develop
+
+ENV PATH /usr/lib/firefox-esr:/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:
+CMD xvfb-run python -c 'from grafanimate.commands import run; run()' --header-layout=no-chrome --scenario=scenarios.py:ontario_windsolar --output=./animations --video-fps=30 --video-framerate=30


We need to do something here, as we can't ship an OCI container including a specific command. However, I am struggling to generalize this, see #24 (comment). Maybe you can help?

I just pushed dcee772 please squash this into my Add dockerfile commit that you cherry picked.

The new Dockerfile looks like this: https://github.com/panodata/grafanimate/blob/dockerize/Dockerfile. xauth has been added, the ENTRYPOINT is apparently still wrong, and doesn't do DWIM yet.

Works in dcee772

$ docker run --rm grafanimate Error: Parameter --scenario is mandatory Usage: grafanimate [options] [--target=<target>]... grafanimate --version grafanimate (-h | --help)

$ docker run --rm grafanimate --header-layout=no-chrome --scenario=scenarios.py:gb_windsolar --output=./animations --video-fps=30 --video-framerate=30 --exposure-time=0.2 2023-10-16 22:53:27,111 [grafanimate.grafana ] INFO : Starting GrafanaWrapper on https://grafana-backup.int.mog.se/d/e0a625dd-20a9-447a-92d8-55eef5b6b077/generation-mix-grafanimate-gb?orgId=1&kiosk=1 2023-10-16 22:53:27,111 [grafanimate.marionette ] INFO : Starting Marionette Gecko wrapper 2023-10-16 22:53:27,112 [grafanimate.marionette ] INFO : Found "firefox" program at /usr/lib/firefox-esr/firefox-esr 2023-10-16 22:53:27,112 [grafanimate.marionette ] INFO : Check for running instance of Marionette/Firefox at localhost:2828 2023-10-16 22:53:27,112 [grafanimate.marionette ] INFO : Will launch new Marionette/Firefox instance mozversion INFO | application_buildid: 20230918121054 mozversion INFO | application_changeset: 17bedcda2f5c28b7fbc136d300bdbbf04f382e00 .............. 2023-10-16 22:53:39,052 [grafanimate.grafana ] INFO : Timewarp to 2022-10-01 00:00:00 -> 2022-10-31 00:00:00 2023-10-16 22:53:39,134 [grafanimate.animations ] INFO : Waiting for 0.2 seconds (exposure time) 2023-10-16 22:53:39,335 [grafanimate.grafana ] INFO : Timewarp to 2022-10-01 04:00:00 -> 2022-10-31 04:00:00 2023-10-16 22:53:39,428 [grafanimate.animations ] INFO : Waiting for 0.2 seconds (exposure time)

I have not tested the results all the way... ohh and ffmpeg will be missing. But it runs!

Does docker run --rm grafanimate --version work for you?

$ docker run --rm -it grafanimate -- --version grafanimate 0.7.0

It does with extra -- before --version
This is confusing, let's move to your PR.

I can confirm that, but the same happens over there as well.

Also works, it does not have to be the -- infix, let's just use foo: ;]

$ docker run --rm local/grafanimate foo --version grafanimate 0.7.0

Both funny and confusing. 😆

intermittentnrg · 2023-10-16T22:59:58Z

It's basically just apk add firefox-esr instead of apt install firefox-esr`. But yeah we can discuss this later. Might add a Dockerfile.alpine later tho and see how it goes...

amotl · 2023-10-16T23:11:59Z

It's basically just apk add firefox-esr instead of apt install firefox-esr.

It is not about this specific recipe. It is about that we want to use the same kinds of recipes across many projects, and need to keep a reasonably universal setup to keep our sanity. Thanks for your understanding ;].

Per instructions https://github.com/krallin/tini#using-tini

amotl · 2023-10-17T08:38:04Z

Dockerfile

+
+COPY . /app
+
+ENTRYPOINT ["/tini", "--", "xvfb-run", "python", "-c", "from grafanimate.commands import run; run()"]


Thank you for discovering the right solution here, I've added it to the other PR with 7419f1c.

amotl · 2023-10-17T08:38:17Z

grafanimate/marionette.py

-        candidates = where.where("firefox")
-        candidates += [
+        candidates = [
+            "/usr/lib/firefox-esr/firefox-esr",
            "/Applications/Firefox.app/Contents/MacOS/firefox-bin",
        ]
+        candidates += where.where("firefox")


Thanks again. Added with f784511.

amotl · 2023-10-17T09:12:38Z

GH-24 has been merged, therefore I am closing this. Thanks for all the fish.

amotl requested changes Oct 16, 2023

View reviewed changes

amotl mentioned this pull request Oct 16, 2023

Dockerize #24

Merged

2 tasks

amotl reviewed Oct 16, 2023

View reviewed changes

intermittentnrg added 4 commits October 17, 2023 02:36

Add Dockerfile

18fd477

Build docker and push to ghcr.io from pipeline

45909d9

Use matrix build for docker platforms

bf933d8

fixup! Add Dockerfile

43ad434

intermittentnrg force-pushed the dockerfile branch from dcee772 to 43ad434 Compare October 17, 2023 00:36

intermittentnrg added 2 commits October 17, 2023 02:48

Add tini

03be161

Per instructions https://github.com/krallin/tini#using-tini

Enable docker cache of setup.py

41ab012

intermittentnrg force-pushed the dockerfile branch from f06bae8 to 41ab012 Compare October 17, 2023 00:55

amotl reviewed Oct 17, 2023

View reviewed changes

amotl closed this Oct 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Dockerfile #21

Add Dockerfile #21

intermittentnrg commented Oct 16, 2023 •

edited

Loading

intermittentnrg commented Oct 16, 2023 •

edited

Loading

amotl left a comment

amotl commented Oct 16, 2023

intermittentnrg commented Oct 16, 2023 •

edited

Loading

amotl commented Oct 16, 2023

intermittentnrg commented Oct 16, 2023

amotl commented Oct 16, 2023

amotl Oct 16, 2023

intermittentnrg Oct 16, 2023

amotl Oct 16, 2023 •

edited

Loading

intermittentnrg Oct 16, 2023

amotl Oct 16, 2023

intermittentnrg Oct 16, 2023

amotl Oct 16, 2023 •

edited

Loading

intermittentnrg commented Oct 16, 2023

amotl commented Oct 16, 2023 •

edited

Loading

amotl Oct 17, 2023

amotl Oct 17, 2023

amotl commented Oct 17, 2023


		COPY . /app

		ENTRYPOINT ["/tini", "--", "xvfb-run", "python", "-c", "from grafanimate.commands import run; run()"]

Add Dockerfile #21

Add Dockerfile #21

Conversation

intermittentnrg commented Oct 16, 2023 • edited Loading

intermittentnrg commented Oct 16, 2023 • edited Loading

amotl left a comment

Choose a reason for hiding this comment

Footnotes

amotl commented Oct 16, 2023

intermittentnrg commented Oct 16, 2023 • edited Loading

amotl commented Oct 16, 2023

intermittentnrg commented Oct 16, 2023

amotl commented Oct 16, 2023

amotl Oct 16, 2023

Choose a reason for hiding this comment

intermittentnrg Oct 16, 2023

Choose a reason for hiding this comment

amotl Oct 16, 2023 • edited Loading

Choose a reason for hiding this comment

intermittentnrg Oct 16, 2023

Choose a reason for hiding this comment

amotl Oct 16, 2023

Choose a reason for hiding this comment

intermittentnrg Oct 16, 2023

Choose a reason for hiding this comment

amotl Oct 16, 2023 • edited Loading

Choose a reason for hiding this comment

intermittentnrg commented Oct 16, 2023

amotl commented Oct 16, 2023 • edited Loading

amotl Oct 17, 2023

Choose a reason for hiding this comment

amotl Oct 17, 2023

Choose a reason for hiding this comment

amotl commented Oct 17, 2023

intermittentnrg commented Oct 16, 2023 •

edited

Loading

intermittentnrg commented Oct 16, 2023 •

edited

Loading

intermittentnrg commented Oct 16, 2023 •

edited

Loading

amotl Oct 16, 2023 •

edited

Loading

amotl Oct 16, 2023 •

edited

Loading

amotl commented Oct 16, 2023 •

edited

Loading