Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Close #LGVISIUM-92: Add GitHub actions and push Docker images to GHCR #93

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .github/workflows/pre-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Pre-release

on:
push:
branches:
- main
workflow_dispatch:

env:
REGISTRY: ghcr.io
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

jobs:
create-pre-release:
runs-on: ubuntu-latest
name: Build and push Docker image and create a new GitHub pre-release

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set environment variables
run: |
echo VERSION=$(cat VERSION).$GITHUB_RUN_NUMBER >> $GITHUB_ENV
echo BASE_IMAGE_NAME=$REGISTRY/$(echo ${GITHUB_REPOSITORY,,}) >> $GITHUB_ENV
echo COMMITED_AT=$(git show -s --format=%cI `git rev-parse HEAD`) >> $GITHUB_ENV

- name: Collect Docker image metadata (api)
id: meta-api
uses: docker/metadata-action@v5
with:
images: ${{ env.BASE_IMAGE_NAME }}-api
labels: |
org.opencontainers.image.created=${{ env.COMMITED_AT }}
org.opencontainers.image.version=v${{ env.VERSION }}
org.opencontainers.image.authors=Stijn Vermeeren <[email protected]>
flavor: |
latest=false
tags: |
type=edge
type=semver,pattern=v{{version}},value=${{ env.VERSION }}

- name: Log in to the GitHub container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push Docker image (api)
uses: docker/build-push-action@v5
with:
context: ./
push: true
build-args: |
VERSION=${{ env.VERSION }}
tags: ${{ steps.meta-api.outputs.tags }}
labels: ${{ steps.meta-api.outputs.labels }}
cache-from: type=registry,ref=${{ env.BASE_IMAGE_NAME }}-api:edge
cache-to: type=inline

- name: Create GitHub pre-release
run: |
gh api \
--method POST \
--header "Accept: application/vnd.github+json" \
/repos/${GITHUB_REPOSITORY}/releases \
-f tag_name='v${{ env.VERSION }}' \
-f target_commitish='main' \
-f name='${{ env.VERSION }}' \
-F prerelease=true \
96 changes: 96 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
name: Release

on:
release:
types: [released]
workflow_dispatch:
inputs:
TAG_NAME:
description: "Tag name"
required: true

env:
REGISTRY: ghcr.io
TAG_NAME: ${{ github.event.inputs.TAG_NAME || github.event.release.tag_name }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

jobs:
retag-docker-image:
runs-on: ubuntu-latest
name: Push updated Docker image

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set environment variables
run: |
echo VERSION=${TAG_NAME#v} >> $GITHUB_ENV
echo BASE_IMAGE_NAME=$REGISTRY/$(echo ${GITHUB_REPOSITORY,,}) >> $GITHUB_ENV
echo COMMITED_AT=$(git show -s --format=%cI `git rev-parse HEAD`) >> $GITHUB_ENV

- name: Collect Docker image metadata (api)
id: meta-api
uses: docker/metadata-action@v5
with:
images: ${{ env.BASE_IMAGE_NAME }}-api
labels: |
org.opencontainers.image.created=${{ env.COMMITED_AT }}
org.opencontainers.image.version=v${{ env.VERSION }}
org.opencontainers.image.authors=Stijn Vermeeren <[email protected]>
tags: |
type=semver,pattern=v{{version}}

- name: Log in to the GitHub container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push Docker image (api)
uses: docker/build-push-action@v5
with:
context: ./
push: true
build-args: |
VERSION=${{ env.VERSION }}
tags: ${{ steps.meta-api.outputs.tags }}
labels: ${{ steps.meta-api.outputs.labels }}
cache-from: type=registry,ref=${{ env.BASE_IMAGE_NAME }}-api:edge
cache-to: type=inline

patch-changelog:
runs-on: ubuntu-latest
name: Patch CHANGELOG.md and update GitHub release notes

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set environment variables
run: |
echo GIT_BRANCH_NAME=mark-version-${TAG_NAME#v}-as-released >> $GITHUB_ENV
echo GIT_COMMIT_MESSAGE=Mark version ${TAG_NAME#v} as released >> $GITHUB_ENV
echo RELEASE_ID=$(gh api -H "Accept: application/vnd.github+json" /repos/${GITHUB_REPOSITORY}/releases/tags/${TAG_NAME} | jq '.id') >> $GITHUB_ENV

- name: Get changelog for this specific release and update release notes
run: |
gh api \
--method PATCH \
--header "Accept: application/vnd.github+json" \
/repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID} \
-f body="$(./get-changelog.sh)"
dcleres marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will discuss with Oliver in detail how this script works. Even without the script, the pipeline was running which is surprizing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that without the script, it just skips the automatic updating of the changelog?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I would have expected the pipeline to crash, but the behavior you described is the one observed.


- name: Checkout new branch and patch changelog
run: |
git checkout -b $GIT_BRANCH_NAME
sed -i "/^\#\# \[Unreleased\]$/a \\\n\#\# $TAG_NAME - $(date '+%Y-%m-%d')" CHANGELOG.md

- name: Commit, push and create pull request
run: |
git config --global user.email "[email protected]"
git config --global user.name "Github build action"
git commit -am "$GIT_COMMIT_MESSAGE"
git push --set-upstream origin $GIT_BRANCH_NAME
gh pr create --title "$GIT_COMMIT_MESSAGE" --body ""
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Changelog

## [Unreleased]

### Changed


### Fixed


## v1.0.0 - 2024-10-07

### Added

- Initial Version of the API

### Changed


### Fixed

10 changes: 10 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Use an official Python runtime as a parent image - Use the latest slim version as the base image
FROM python:3.12-slim

# Set arguments to be passed from build-args
ARG VERSION

# Set the working directory in the container
WORKDIR /app

Expand All @@ -12,16 +15,23 @@ RUN pip install --no-cache-dir pip-tools \
&& pip-compile --generate-hashes \
&& pip-sync

# Curl installation step for health check
RUN apt-get update && apt-get install -y curl

# Copy the rest of the application source code into the container
COPY ./src /app/src
COPY ./config /app/config

# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV PYTHONPATH=/app/src
ENV APP_VERSION=$VERSION

# Expose port 8000 for the FastAPI Borehole app
EXPOSE 8000

# Run a health check to ensure the container is healthy
HEALTHCHECK CMD curl --silent --fail http://localhost:8000/health || exit 1

# Command to run the FastAPI Borehole app with Uvicorn
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
42 changes: 41 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,7 +288,7 @@ Additional endpoints and their functionalities can be found in the project's sou
To stop the FastAPI server, press `Ctrl + C` in the terminal where the server is running. Please refer to the [FastAPI documentation](https://fastapi.tiangolo.com) for more information on how to work with FastAPI and build APIs using this framework.


## API as Docker Image
## Build API as Local Docker Image

The borehole application offers a given amount of functionalities (extract text, number, and coordinates) through an API. To build this API using a Docker Container, you can run the following commands.

Expand Down Expand Up @@ -449,6 +449,46 @@ docker stop <container_id>
Replace `<container_id>` with the ID of the running container, which can be obtained by running `docker ps`.


## Use the Docker Image from the GitHub Container Registry

This repository provides a Docker image hosted in the GitHub Container Registry (GHCR) that can be used to run the application easily. Below are the steps to pull and run the Docker image.

1. **Pull the Docker Image from the GitHub Container Registry**

```bash
docker pull ghcr.io/swisstopo/swissgeol-boreholes-dataextraction-api:edge
```

1. a. **Run the docker image from the Terminal**

```bash
docker run -d --name swissgeol-boreholes-dataextraction-api -e AWS_ACCESS_KEY_ID=XXX -e AWS_SECRET_ACCESS_KEY=YYY -e AWS_ENDPOINT=ZZZ -p 8000:8000 ghcr.io/swisstopo/swissgeol-boreholes-dataextraction-api:TAG
```

Adjust the port mapping (8000:8000) based on the app's requirements.

NOTE: Do not forget to specify your AWS Credentials.

1. b. **Run the docker image from the Docker Desktop App**

Open the Docker Desktop app and navigate to `Images`, you should be able to see the image you just pulled from GHCR. Click on the image and click on the `Run` button on the top right of the screen.

![](assets/img/docker-1.png){ width=400px }

Then open the `Optional Settings` menu and specify the port and the AWS credentials

![](assets/img/docker-2.png){ width=800px }


2. **Verify the Container is Running**

To check if the container is running, use:

```bash
docker ps
```


## AWS Lambda Deployment

AWS Lambda is a serverless computing service provided by Amazon Web Services that allows you to run code without managing servers. It automatically scales your applications by executing code in response to triggers. You only pay for the compute time used.
Expand Down
1 change: 1 addition & 0 deletions VERSION
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.0
Binary file added assets/img/docker-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/img/docker-2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
23 changes: 23 additions & 0 deletions get-changelog.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/bash
# This script gets all changelog entries from CHANGELOG.md since last release.

set -e

tempDir="$(mktemp -d)"
tempFile=$tempDir/gh_release_notes.md

# Get changelog entries since last release
cat CHANGELOG.md | \
grep -Pazo '(?s)(?<=\#{2} \[Unreleased\]\n{2}).*?(?=\n\#{2} v|$)' \
> $tempFile

# Improve readability and add some icons
sed -i -E 's/(###) (Added)/\1 🚀 \2/' $tempFile
sed -i -E 's/(###) (Changed)/\1 🔨 \2/' $tempFile
sed -i -E 's/(###) (Fixed)/\1 🐛 \2/' $tempFile
sed -i 's/\x0//g' $tempFile

cat $tempFile

# Cleanup temporary files
trap 'rm -rf -- "$tempDir"' EXIT
11 changes: 10 additions & 1 deletion src/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,16 @@ async def validation_exception_handler(request: Request, exc: RequestValidationE
@app.get("/health", tags=["health"])
def get_health():
"""Check the health of the application."""
return {"status": "ok"}
return "Healthy"


####################################################################################################
### Version
####################################################################################################
@app.get("/version")
def get_version():
"""Return the version of the application."""
return {"version": os.getenv("APP_VERSION")}


####################################################################################################
Expand Down
Loading