Merge branch 'mailcow:master' into master

.github/ISSUE_TEMPLATE/pr_to_nighty_template.yml

@@ -1,13 +1,3 @@
-## :memo:  Brief description
-<!-- Diff summary - START -->
-<!-- Diff summary - END -->
-
-
-## :computer:  Commits
-<!-- Diff commits - START -->
-<!-- Diff commits - END -->
-
-
 ## :file_folder:  Modified files
 <!-- Diff files - START -->
-<!-- Diff files - END -->
+<!-- Diff files - END -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,38 @@
+<!-- _Please make sure to review and check all of these items, otherwise we might refuse your PR:_ -->
+
+## Contribution Guidelines
+
+* [ ] I've read the [contribution guidelines](https://github.com/mailcow/mailcow-dockerized/blob/master/CONTRIBUTING.md) and wholeheartedly agree them
+
+<!-- _NOTE: this tickbox is needed to fullfil on order to get your PR reviewed._ -->
+
+## What does this PR include?
+
+### Short Description
+
+<!-- Please write a short description, what your PR does here. -->
+
+###  Affected Containers
+
+<!-- Please list all affected Docker containers here, which you commited changes to -->
+
+<!--
+
+Please list them like this:
+
+- container1
+- container2
+- container3
+etc.
+
+-->
+
+## Did you run tests?
+
+### What did you tested?
+
+<!-- Please write shortly, what you've tested (which components etc.). -->
+
+### What were the final results? (Awaited, got)
+
+<!-- Please write shortly, what your final tests results were. What did you awaited? Was the outcome the awaited one? -->

.github/workflows/rebuild_backup_image.yml

@@ -26,7 +26,7 @@ jobs:
           password: ${{ secrets.BACKUPIMAGEBUILD_ACTION_DOCKERHUB_TOKEN }}
 
       - name: Build and push
-        uses: docker/build-push-action@v5
+        uses: docker/build-push-action@v6
         with:
           context: .
           platforms: linux/amd64,linux/arm64

CONTRIBUTING.md

@@ -1,33 +1,52 @@
-# Contribution Guidelines (Last modified on 18th December 2023)
+# Contribution Guidelines
+**_Last modified on 15th August 2024_**
 
 First of all, thank you for wanting to provide a bugfix or a new feature for the mailcow community, it's because of your help that the project can continue to grow!
 
-## Pull Requests (Last modified on 18th December 2023)
+As we want to keep mailcow's development structured we setup these Guidelines which helps you to create your issue/pull request accordingly.
+
+**PLEASE NOTE, THAT WE MIGHT CLOSE ISSUES/PULL REQUESTS IF THEY DON'T FULLFIL OUR WRITTEN GUIDELINES WRITTEN INSIDE THIS DOCUMENT**. So please check this guidelines before you propose a Issue/Pull Request.
+
+## Topics
+
+- [Pull Requests](#pull-requests)
+- [Issue Reporting](#issue-reporting)
+    - [Guidelines](#issue-reporting-guidelines)
+    - [Issue Report Guide](#issue-report-guide)
+
+## Pull Requests
+**_Last modified on 15th August 2024_**
 
 However, please note the following regarding pull requests:
 
 1. **ALWAYS** create your PR using the staging branch of your locally cloned mailcow instance, as the pull request will end up in said staging branch of mailcow once approved. Ideally, you should simply create a new branch for your pull request that is named after the type of your PR (e.g. `feat/` for function updates or `fix/` for bug fixes) and the actual content (e.g. `sogo-6.0.0` for an update from SOGo to version 6 or `html-escape` for a fix that includes escaping HTML in mailcow).
-2. Please **keep** this pull request branch **clean** and free of commits that have nothing to do with the changes you have made (e.g. commits from other users from other branches). *If you make changes to the `update.sh` script or other scripts that trigger a commit, there is usually a developer mode for clean working in this case.
-3. **Test your changes before you commit them as a pull request.** <ins>If possible</ins>, write a small **test log** or demonstrate the functionality with a **screenshot or GIF**. *We will of course also test your pull request ourselves, but proof from you will save us the question of whether you have tested your own changes yourself.*
-4. Please **ALWAYS** create the actual pull request against the staging branch and **NEVER** directly against the master branch. *If you forget to do this, our moobot will remind you to switch the branch to staging.*
-5. Wait for a merge commit: It may happen that we do not accept your pull request immediately or sometimes not at all for various reasons. Please do not be disappointed if this is the case. We always endeavor to incorporate any meaningful changes from the community into the mailcow project.
-6. If you are planning larger and therefore more complex pull requests, it would be advisable to first announce this in a separate issue and then start implementing it after the idea has been accepted in order to avoid unnecessary frustration and effort!
+2. **ALWAYS** report/request issues/features in the english language, even though mailcow is a german based company. This is done to allow other GitHub users to reply to your issues/requests too which did not speak german or other languages besides english.
+3. Please **keep** this pull request branch **clean** and free of commits that have nothing to do with the changes you have made (e.g. commits from other users from other branches). *If you make changes to the `update.sh` script or other scripts that trigger a commit, there is usually a developer mode for clean working in this case.*
+4. **Test your changes before you commit them as a pull request.** <ins>If possible</ins>, write a small **test log** or demonstrate the functionality with a **screenshot or GIF**. *We will of course also test your pull request ourselves, but proof from you will save us the question of whether you have tested your own changes yourself.*
+5. **Please use** the pull request template we provide once creating a pull request. *HINT: During editing you encounter comments which looks like: `<!-- CONTENT -->`. These can be removed or kept, as they will not rendered later on GitHub! Please only create actual content without the said comments.* 
+6. Please **ALWAYS** create the actual pull request against the staging branch and **NEVER** directly against the master branch. *If you forget to do this, our moobot will remind you to switch the branch to staging.*
+7. Wait for a merge commit: It may happen that we do not accept your pull request immediately or sometimes not at all for various reasons. Please do not be disappointed if this is the case. We always endeavor to incorporate any meaningful changes from the community into the mailcow project.
+8. If you are planning larger and therefore more complex pull requests, it would be advisable to first announce this in a separate issue and then start implementing it after the idea has been accepted in order to avoid unnecessary frustration and effort!
 
 ---
 
-## Issue Reporting (Last modified on 18th December 2023)
+## Issue Reporting
+**_Last modified on 15th August 2024_**
 
 If you plan to report a issue within mailcow please read and understand the following rules:
 
+### Issue Reporting Guidelines
+
 1. **ONLY** use the issue tracker for bug reports or improvement requests and NOT for support questions. For support questions you can either contact the [mailcow community on Telegram](https://docs.mailcow.email/#community-support-and-chat) or the mailcow team directly in exchange for a [support fee](https://docs.mailcow.email/#commercial-support).
 2. **ONLY** report an error if you have the **necessary know-how (at least the basics)** for the administration of an e-mail server and the usage of Docker. mailcow is a complex and fully-fledged e-mail server including groupware components on a Docker basement and it requires a bit of technical know-how for debugging and operating.
-3. **ONLY** report bugs that are contained in the latest mailcow release series. *The definition of the latest release series includes the last major patch (e.g. 2023-12) and all minor patches (revisions) below it (e.g. 2023-12a, b, c etc.).* New issue reports published starting from January 1, 2024 must meet this criterion, as versions below the latest releases are no longer supported by us.
-4. When reporting a problem, please be as detailed as possible and include even the smallest changes to your mailcow installation. Simply fill out the corresponding bug report form in detail and accurately to minimize possible questions.
-5. **Before you open an issue/feature request**, please first check whether a similar request already exists in the mailcow tracker on GitHub. If so, please include yourself in this request.
-6. When you create a issue/feature request: Please note that the creation does <ins>**not guarantee an instant implementation or fix by the mailcow team or the community**</ins>.
-7. Please **ALWAYS** anonymize any sensitive information in your bug report or feature request before submitting it.
-
-### Quick guide to reporting problems:
+3. **ALWAYS** report/request issues/features in the english language, even though mailcow is a german based company. This is done to allow other GitHub users to reply to your issues/requests too which did not speak german or other languages besides english.
+4. **ONLY** report bugs that are contained in the latest mailcow release series. *The definition of the latest release series includes the last major patch (e.g. 2023-12) and all minor patches (revisions) below it (e.g. 2023-12a, b, c etc.).* New issue reports published starting from January 1, 2024 must meet this criterion, as versions below the latest releases are no longer supported by us.
+5. When reporting a problem, please be as detailed as possible and include even the smallest changes to your mailcow installation. Simply fill out the corresponding bug report form in detail and accurately to minimize possible questions.
+6. **Before you open an issue/feature request**, please first check whether a similar request already exists in the mailcow tracker on GitHub. If so, please include yourself in this request.
+7. When you create a issue/feature request: Please note that the creation does <ins>**not guarantee an instant implementation or fix by the mailcow team or the community**</ins>.
+8. Please **ALWAYS** anonymize any sensitive information in your bug report or feature request before submitting it.
+
+### Issue Report Guide
 1. Read your logs; follow them to see what the reason for your problem is.
 2. Follow the leads given to you in your logfiles and start investigating.
 3. Restarting the troubled service or the whole stack to see if the problem persists.
@@ -36,4 +55,4 @@ If you plan to report a issue within mailcow please read and understand the foll
 6. [Create an issue](https://github.com/mailcow/mailcow-dockerized/issues/new/choose) over at our GitHub repository if you think your problem might be a bug or a missing feature you badly need. But please make sure, that you include **all the logs** and a full description to your problem.
 7. Ask your questions in our community-driven [support channels](https://docs.mailcow.email/#community-support-and-chat).
 
-## When creating an issue/feature request or a pull request, you will be asked to confirm these guidelines.
+## When creating an issue/feature request or a pull request, you will be asked to confirm these guidelines.

data/Dockerfiles/acme/Dockerfile

@@ -1,8 +1,8 @@
-FROM alpine:3.18
+FROM alpine:3.20
+
+LABEL maintainer = "The Infrastructure Company GmbH <[email protected]>"
 
-LABEL maintainer "The Infrastructure Company GmbH <[email protected]>"
 
-ARG PIP_BREAK_SYSTEM_PACKAGES=1
 RUN apk upgrade --no-cache \
   && apk add --update --no-cache \
   bash \
@@ -15,9 +15,7 @@ RUN apk upgrade --no-cache \
   tini \
   tzdata \
   python3 \
-  py3-pip \
-  && pip3 install --upgrade pip \
-  && pip3 install acme-tiny
+  acme-tiny --repository=http://dl-cdn.alpinelinux.org/alpine/edge/community/
 
 COPY acme.sh /srv/acme.sh
 COPY functions.sh /srv/functions.sh

data/Dockerfiles/acme/acme.sh

@@ -33,6 +33,10 @@ if [[ "${ONLY_MAILCOW_HOSTNAME}" =~ ^([yY][eE][sS]|[yY])+$ ]]; then
   ONLY_MAILCOW_HOSTNAME=y
 fi
 
+if [[ "${AUTODISCOVER_SAN}" =~ ^([yY][eE][sS]|[yY])+$ ]]; then
+  AUTODISCOVER_SAN=y
+fi
+
 # Request individual certificate for every domain
 if [[ "${ENABLE_SSL_SNI}" =~ ^([yY][eE][sS]|[yY])+$ ]]; then
   ENABLE_SSL_SNI=y
@@ -119,7 +123,7 @@ done
 log_f "Database OK"
 
 log_f "Waiting for Nginx..."
-until $(curl --output /dev/null --silent --head --fail http://nginx:8081); do
+until $(curl --output /dev/null --silent --head --fail http://nginx.${COMPOSE_PROJECT_NAME}_mailcow-network:8081); do
   sleep 2
 done
 log_f "Nginx OK"
@@ -133,7 +137,7 @@ log_f "Resolver OK"
 # Waiting for domain table
 log_f "Waiting for domain table..."
 while [[ -z ${DOMAIN_TABLE} ]]; do
-  curl --silent http://nginx/ >/dev/null 2>&1
+  curl --silent http://nginx.${COMPOSE_PROJECT_NAME}_mailcow-network/ >/dev/null 2>&1
   DOMAIN_TABLE=$(mysql --socket=/var/run/mysqld/mysqld.sock -u ${DBUSER} -p${DBPASS} ${DBNAME} -e "SHOW TABLES LIKE 'domain'" -Bs)
   [[ -z ${DOMAIN_TABLE} ]] && sleep 10
 done
@@ -211,7 +215,11 @@ while true; do
       ADDITIONAL_SAN_ARR+=($i)
     fi
   done
+
+  if [[ ${AUTODISCOVER_SAN} == "y" ]]; then
+  # Fetch certs for autoconfig and autodiscover subdomains
   ADDITIONAL_WC_ARR+=('autodiscover' 'autoconfig')
+  fi
 
   if [[ ${SKIP_IP_CHECK} != "y" ]]; then
   # Start IP detection

data/Dockerfiles/acme/reload-configurations.sh

@@ -2,32 +2,32 @@
 
 # Reading container IDs
 # Wrapping as array to ensure trimmed content when calling $NGINX etc.
-NGINX=($(curl --silent --insecure https://dockerapi/containers/json | jq -r ".[] | {name: .Config.Labels[\"com.docker.compose.service\"], project: .Config.Labels[\"com.docker.compose.project\"], id: .Id}" | jq -rc "select( .name | tostring | contains(\"nginx-mailcow\")) | select( .project | tostring | contains(\"${COMPOSE_PROJECT_NAME,,}\")) | .id" | tr "\n" " "))
-DOVECOT=($(curl --silent --insecure https://dockerapi/containers/json | jq -r ".[] | {name: .Config.Labels[\"com.docker.compose.service\"], project: .Config.Labels[\"com.docker.compose.project\"], id: .Id}" | jq -rc "select( .name | tostring | contains(\"dovecot-mailcow\")) | select( .project | tostring | contains(\"${COMPOSE_PROJECT_NAME,,}\")) | .id" | tr "\n" " "))
-POSTFIX=($(curl --silent --insecure https://dockerapi/containers/json | jq -r ".[] | {name: .Config.Labels[\"com.docker.compose.service\"], project: .Config.Labels[\"com.docker.compose.project\"], id: .Id}" | jq -rc "select( .name | tostring | contains(\"postfix-mailcow\")) | select( .project | tostring | contains(\"${COMPOSE_PROJECT_NAME,,}\")) | .id" | tr "\n" " "))
+NGINX=($(curl --silent --insecure https://dockerapi.${COMPOSE_PROJECT_NAME}_mailcow-network/containers/json | jq -r ".[] | {name: .Config.Labels[\"com.docker.compose.service\"], project: .Config.Labels[\"com.docker.compose.project\"], id: .Id}" | jq -rc "select( .name | tostring | contains(\"nginx-mailcow\")) | select( .project | tostring | contains(\"${COMPOSE_PROJECT_NAME,,}\")) | .id" | tr "\n" " "))
+DOVECOT=($(curl --silent --insecure https://dockerapi.${COMPOSE_PROJECT_NAME}_mailcow-network/containers/json | jq -r ".[] | {name: .Config.Labels[\"com.docker.compose.service\"], project: .Config.Labels[\"com.docker.compose.project\"], id: .Id}" | jq -rc "select( .name | tostring | contains(\"dovecot-mailcow\")) | select( .project | tostring | contains(\"${COMPOSE_PROJECT_NAME,,}\")) | .id" | tr "\n" " "))
+POSTFIX=($(curl --silent --insecure https://dockerapi.${COMPOSE_PROJECT_NAME}_mailcow-network/containers/json | jq -r ".[] | {name: .Config.Labels[\"com.docker.compose.service\"], project: .Config.Labels[\"com.docker.compose.project\"], id: .Id}" | jq -rc "select( .name | tostring | contains(\"postfix-mailcow\")) | select( .project | tostring | contains(\"${COMPOSE_PROJECT_NAME,,}\")) | .id" | tr "\n" " "))
 
 reload_nginx(){
   echo "Reloading Nginx..."
-  NGINX_RELOAD_RET=$(curl -X POST --insecure https://dockerapi/containers/${NGINX}/exec -d '{"cmd":"reload", "task":"nginx"}' --silent -H 'Content-type: application/json' | jq -r .type)
+  NGINX_RELOAD_RET=$(curl -X POST --insecure https://dockerapi.${COMPOSE_PROJECT_NAME}_mailcow-network/containers/${NGINX}/exec -d '{"cmd":"reload", "task":"nginx"}' --silent -H 'Content-type: application/json' | jq -r .type)
   [[ ${NGINX_RELOAD_RET} != 'success' ]] && { echo "Could not reload Nginx, restarting container..."; restart_container ${NGINX} ; }
 }
 
 reload_dovecot(){
   echo "Reloading Dovecot..."
-  DOVECOT_RELOAD_RET=$(curl -X POST --insecure https://dockerapi/containers/${DOVECOT}/exec -d '{"cmd":"reload", "task":"dovecot"}' --silent -H 'Content-type: application/json' | jq -r .type)
+  DOVECOT_RELOAD_RET=$(curl -X POST --insecure https://dockerapi.${COMPOSE_PROJECT_NAME}_mailcow-network/containers/${DOVECOT}/exec -d '{"cmd":"reload", "task":"dovecot"}' --silent -H 'Content-type: application/json' | jq -r .type)
   [[ ${DOVECOT_RELOAD_RET} != 'success' ]] && { echo "Could not reload Dovecot, restarting container..."; restart_container ${DOVECOT} ; }
 }
 
 reload_postfix(){
   echo "Reloading Postfix..."
-  POSTFIX_RELOAD_RET=$(curl -X POST --insecure https://dockerapi/containers/${POSTFIX}/exec -d '{"cmd":"reload", "task":"postfix"}' --silent -H 'Content-type: application/json' | jq -r .type)
+  POSTFIX_RELOAD_RET=$(curl -X POST --insecure https://dockerapi.${COMPOSE_PROJECT_NAME}_mailcow-network/containers/${POSTFIX}/exec -d '{"cmd":"reload", "task":"postfix"}' --silent -H 'Content-type: application/json' | jq -r .type)
   [[ ${POSTFIX_RELOAD_RET} != 'success' ]] && { echo "Could not reload Postfix, restarting container..."; restart_container ${POSTFIX} ; }
 }
 
 restart_container(){
   for container in $*; do
     echo "Restarting ${container}..."
-    C_REST_OUT=$(curl -X POST --insecure https://dockerapi/containers/${container}/restart --silent | jq -r '.msg')
+    C_REST_OUT=$(curl -X POST --insecure https://dockerapi.${COMPOSE_PROJECT_NAME}_mailcow-network/containers/${container}/restart --silent | jq -r '.msg')
     echo "${C_REST_OUT}"
   done
 }

data/Dockerfiles/backup/Dockerfile

@@ -1,3 +1,3 @@
-FROM debian:bullseye-slim
+FROM debian:bookworm-slim
 
 RUN apt update && apt install pigz

data/Dockerfiles/clamd/Dockerfile

@@ -1,6 +1,6 @@
-FROM alpine:3.19
+FROM alpine:3.20
 
-LABEL maintainer "The Infrastructure Company GmbH <[email protected]>"
+LABEL maintainer = "The Infrastructure Company GmbH <[email protected]>"
 
 RUN apk upgrade --no-cache \
   && apk add --update --no-cache \

data/Dockerfiles/dockerapi/Dockerfile

@@ -1,6 +1,6 @@
-FROM alpine:3.19
+FROM alpine:3.20
 
-LABEL maintainer "The Infrastructure Company GmbH <[email protected]>"
+LABEL maintainer = "The Infrastructure Company GmbH <[email protected]>"
 
 ARG PIP_BREAK_SYSTEM_PACKAGES=1
 WORKDIR /app

data/Dockerfiles/dockerapi/modules/DockerApi.py

@@ -358,8 +358,8 @@ def container_post__exec__rspamd__worker_password(self, request_json, **kwargs):
         for line in cmd_response.split("\n"):
           if '$2$' in line:
             hash = line.strip()
-            hash_out = re.search('\$2\$.+$', hash).group(0)
-            rspamd_passphrase_hash = re.sub('[^0-9a-zA-Z\$]+', '', hash_out.rstrip())
+            hash_out = re.search(r'\$2\$.+$', hash).group(0)
+            rspamd_passphrase_hash = re.sub(r'[^0-9a-zA-Z\$]+', '', hash_out.rstrip())
             rspamd_password_filename = "/etc/rspamd/override.d/worker-controller-password.inc"
             cmd = '''/bin/echo 'enable_password = "%s";' > %s && cat %s''' % (rspamd_passphrase_hash, rspamd_password_filename, rspamd_password_filename)
             cmd_response = self.exec_cmd_container(container, cmd, user="_rspamd")