Merge branch 'release-10.3.8' into stable

CHANGELOG.md

@@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project follows MariaDB versioning.
 
+## [10.3.8] - 2018-07-29
+### Changed
+  - Upgrade MariaDB version to 10.3.8
+  - Now use mariabackup instead of xtrabackup
+
 ## [10.2.14-1] - 2018-04-09
 ### Fixed
   - User and database init
@@ -153,6 +158,8 @@ and this project follows MariaDB versioning.
   - Upgrade baseimage to baseimage:0.9.0
   - Change docker command from docker.io to docker
 
+[10.3.8]: https://github.com/osixia/docker-mariadb/compare/v10.2.14-1...v10.3.8
+[10.2.14-1]: https://github.com/osixia/docker-mariadb/compare/v10.2.14...v10.2.14-1
 [10.2.14]: https://github.com/osixia/docker-mariadb/compare/v10.2.13...v10.2.14
 [10.2.13]: https://github.com/osixia/docker-mariadb/compare/v10.2.12...v10.2.13
 [10.2.12]: https://github.com/osixia/docker-mariadb/compare/v10.2.11...v10.2.12

Makefile

@@ -1,5 +1,5 @@
 NAME = osixia/mariadb
-VERSION = 10.2.14-1
+VERSION = 10.3.8
 
 .PHONY: build build-nocache test tag-latest push push-latest release git-tag-version
 

README.md

@@ -6,45 +6,46 @@
 
 [hub]: https://hub.docker.com/r/osixia/mariadb/
 
-Latest release: 10.2.14-1 - MariaDB 10.2.14 - [Changelog](CHANGELOG.md) | [Docker Hub](https://hub.docker.com/r/osixia/mariadb/) 
+Latest release: 10.3.8 - MariaDB 10.3.8 - [Changelog](CHANGELOG.md) | [Docker Hub](https://hub.docker.com/r/osixia/mariadb/) 
 
-**A docker image to run MariaDB with XtraBackup.**
+**A docker image to run MariaDB with perdiodic backups.**
 > [MariaDB](https://mariadb.org/)
 
-> [Percona XtraBackup](https://www.percona.com/doc/percona-xtrabackup/2.4/index.html)
-
-- [Quick start](#quick-start)
-- [Beginner Guide](#beginner-guide)
-	- [Create new database](#create-new-database)
-		- [Full example](#full-example)
-		- [Data persistence](#data-persistence)
-	- [Use an existing MariaDB database](#use-an-existing-mariadb-database)
-	- [Administrate your MariaDB server](#administrate-your-mariadb-server)
-	- [Use a custom my.cnf](#use-a-custom-mycnf)
-	- [SSL](#ssl)
-		- [Use autogenerated certificate](#use-autogenerated-certificate)
-		- [Use your own certificate](#use-your-own-certificate)
-		- [Disable SSL](#disable-ssl)
-	- [Fix docker mounted file problems](#fix-docker-mounted-file-problems)
-	- [Debug](#debug)
-- [Environment Variables](#environment-variables)
-	- [Set your own environment variables](#set-your-own-environment-variables)
-		- [Use command line argument](#use-command-line-argument)
-		- [Link environment file](#link-environment-file)
-		- [Make your own image or extend this image](#make-your-own-image-or-extend-this-image)
-- [Advanced User Guide](#advanced-user-guide)
-	- [Extend osixia/mariadb:10.2.14-1 image](#extend-osixiamariadb10214-1-image)
-	- [Make your own MariaDB image](#make-your-own-mariadb-image)
-	- [Tests](#tests)
-	- [Kubernetes](#kubernetes)
-	- [Under the hood: osixia/light-baseimage](#under-the-hood-osixialight-baseimage)
-- [Security](#security)
-- [Changelog](#changelog)
+> [MariaDB Backup](https://mariadb.com/kb/en/library/mariadb-backup/)
+
+- [osixia/mariadb](#osixiamariadb)
+	- [Quick start](#quick-start)
+	- [Beginner Guide](#beginner-guide)
+		- [Create new database](#create-new-database)
+			- [Full example](#full-example)
+			- [Data persistence](#data-persistence)
+		- [Use an existing MariaDB database](#use-an-existing-mariadb-database)
+		- [Administrate your mariadb server](#administrate-your-mariadb-server)
+		- [Use a custom my.cnf](#use-a-custom-mycnf)
+		- [SSL](#ssl)
+			- [Use autogenerated certificate](#use-autogenerated-certificate)
+			- [Use your own certificate](#use-your-own-certificate)
+			- [Disable SSL](#disable-ssl)
+		- [Fix docker mounted file problems](#fix-docker-mounted-file-problems)
+		- [Debug](#debug)
+	- [Environment Variables](#environment-variables)
+		- [Set your own environment variables](#set-your-own-environment-variables)
+			- [Use command line argument](#use-command-line-argument)
+			- [Link environment file](#link-environment-file)
+			- [Make your own image or extend this image](#make-your-own-image-or-extend-this-image)
+	- [Advanced User Guide](#advanced-user-guide)
+		- [Extend osixia/mariadb:10.3.8 image](#extend-osixiamariadb1038-image)
+		- [Make your own MariaDB image](#make-your-own-mariadb-image)
+		- [Tests](#tests)
+		- [Kubernetes](#kubernetes)
+		- [Under the hood: osixia/light-baseimage](#under-the-hood-osixialight-baseimage)
+	- [Security](#security)
+	- [Changelog](#changelog)
 
 ## Quick start
 Run MariaDB docker image:
 
-	docker run --name my-mariadb-container --detach osixia/mariadb:10.2.14-1
+	docker run --name my-mariadb-container --detach osixia/mariadb:10.3.8
 
 This start a new container with a MariaDB server running inside.
 
@@ -134,7 +135,7 @@ Or you can set your custom config at run time, by mounting your **my.cnf** file
 #### Use autogenerated certificate
 By default SSL is enable, a certificate is created with the container hostname (it can be set by docker run --hostname option eg: db.my-company.com).
 
-	docker run --hostname db.my-company.com --detach osixia/mariadb:10.2.14-1
+	docker run --hostname db.my-company.com --detach osixia/mariadb:10.3.8
 
 #### Use your own certificate
 
@@ -144,22 +145,22 @@ You can set your custom certificate at run time, by mounting a directory contain
 	--env MARIADB_SSL_CRT_FILENAME=my-cert.crt \
 	--env MARIADB_SSL_KEY_FILENAME=my-cert.key \
 	--env MARIADB_SSL_CA_CRT_FILENAME=the-ca.crt \
-	--detach osixia/mariadb:10.2.14-1
+	--detach osixia/mariadb:10.3.8
 
 Other solutions are available please refer to the [Advanced User Guide](#advanced-user-guide)
 
 #### Disable SSL
 Add --env MARIADB_SSL=false to the run command :
 
-    docker run --env MARIADB_SSL=false --detach osixia/mariadb:10.2.14-1
+    docker run --env MARIADB_SSL=false --detach osixia/mariadb:10.3.8
 
 ### Fix docker mounted file problems
 
 You may have some problems with mounted files on some systems. The startup script try to make some file adjustment and fix files owner and permissions, this can result in multiple errors. See [Docker documentation](https://docs.docker.com/v1.4/userguide/dockervolumes/#mount-a-host-file-as-a-data-volume).
 
 To fix that run the container with `--copy-service` argument :
 
-		docker run [your options] osixia/mariadb:10.2.14-1 --copy-service
+		docker run [your options] osixia/mariadb:10.3.8 --copy-service
 
 
 ### Debug
@@ -169,11 +170,11 @@ Available levels are: `none`, `error`, `warning`, `info`, `debug` and `trace`.
 
 Example command to run the container in `debug` mode:
 
-	docker run --detach osixia/mariadb:10.2.14-1 --loglevel debug
+	docker run --detach osixia/mariadb:10.3.8 --loglevel debug
 
 See all command line options:
 
-	docker run osixia/mariadb:10.2.14-1 --help
+	docker run osixia/mariadb:10.3.8 --help
 
 
 ## Environment Variables
@@ -193,7 +194,7 @@ Used when the container is started without an existing database:
 	```
 	If you want to set this variable at docker run command add the tag `#PYTHON2BASH:` and convert the yaml in python:
 
-		docker run --env MARIADB_ROOT_ALLOWED_NETWORKS="#PYTHON2BASH:['localhost','127.0.0.1','::1']" --detach osixia/mariadb:10.2.14-1
+		docker run --env MARIADB_ROOT_ALLOWED_NETWORKS="#PYTHON2BASH:['localhost','127.0.0.1','::1']" --detach osixia/mariadb:10.3.8
 
 	To convert yaml to python online: http://yaml-online-parser.appspot.com/
 
@@ -209,7 +210,7 @@ Used when the container is started without an existing database:
 	```
 	If you want to set this variable at docker run command add the tag `#PYTHON2BASH:` and convert the yaml in python:
 
-		docker run --env MARIADB_DATABASES="#PYTHON2BASH:[{'products': [{'user': 'password'}, {'user2': 'passw0rd'}]},'posts','tomatoes']" --detach osixia/mariadb:10.2.14-1
+		docker run --env MARIADB_DATABASES="#PYTHON2BASH:[{'products': [{'user': 'password'}, {'user2': 'passw0rd'}]},'posts','tomatoes']" --detach osixia/mariadb:10.3.8
 
 	To convert yaml to python online: http://yaml-online-parser.appspot.com/
 
@@ -223,7 +224,7 @@ Used when the container is started without an existing database:
 	```
 	If you want to set this variable at docker run command add the tag `#PYTHON2BASH:` and convert the yaml in python:
 
-		docker run --env MARIADB_DATABASES="#PYTHON2BASH:[{'boby': 'mcD0nald'},{'billy': 'th3k1ng'},{'tomatoes': 'ketchup'}]" --detach osixia/mariadb:10.2.14-1
+		docker run --env MARIADB_DATABASES="#PYTHON2BASH:[{'boby': 'mcD0nald'},{'billy': 'th3k1ng'},{'tomatoes': 'ketchup'}]" --detach osixia/mariadb:10.3.8
 
 	To convert yaml to python online: http://yaml-online-parser.appspot.com/
 
@@ -254,14 +255,14 @@ Other environment variables:
 #### Use command line argument
 Environment variables can be set by adding the --env argument in the command line, for example:
 
-	docker run --env MARIADB_ROOT_USER="JaxTeller" --env MARIADB_ROOT_PASSWORD="Sons Of Anarchy" --detach osixia/mariadb:10.2.14-1
+	docker run --env MARIADB_ROOT_USER="JaxTeller" --env MARIADB_ROOT_PASSWORD="Sons Of Anarchy" --detach osixia/mariadb:10.3.8
 
 #### Link environment file
 
 For example if your environment file is in :  /data/environment/my-env.yaml
 
 	docker run --volume /data/environment/my-env.yaml:/container/environment/01-custom/env.yaml \
-	--detach osixia/mariadb:10.2.14-1
+	--detach osixia/mariadb:10.3.8
 
 Take care to link your environment file to `/container/environment/XX-somedir` (with XX < 99 so they will be processed before default environment files) and not  directly to `/container/environment` because this directory contains predefined baseimage environment files to fix container environment (INITRD, LANG, LANGUAGE and LC_CTYPE).
 
@@ -271,13 +272,13 @@ This is the best solution if you have a private registry. Please refer to the [A
 
 ## Advanced User Guide
 
-### Extend osixia/mariadb:10.2.14-1 image
+### Extend osixia/mariadb:10.3.8 image
 
 If you need to add your custom TLS certificate, bootstrap config or environment files the easiest way is to extends this image.
 
 Dockerfile example:
 
-    FROM osixia/mariadb:10.2.14-1
+    FROM osixia/mariadb:10.3.8
     MAINTAINER Your Name <[email protected]>
 
     ADD ssl-certs /container/service/mariadb/assets/certs

image/Dockerfile

@@ -1,21 +1,18 @@
 # Use osixia/light-baseimage
 # sources: https://github.com/osixia/docker-light-baseimage
 FROM osixia/light-baseimage:1.1.1
-MAINTAINER Bertrand Gouny <[email protected]>
 
 # MariaDB version
-ARG MARIADB_MAJOR=10.2
-ARG MARIADB_VERSION=10.2.14+maria~stretch
+ARG MARIADB_MAJOR=10.3
+ARG MARIADB_VERSION=1:10.3.8+maria~stretch
 
 # Add mysql user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
 # https://github.com/docker-library/mariadb/blob/master/10.0/Dockerfile
 RUN groupadd -r mysql && useradd -r -g mysql mysql
 
-# Add MariaDB and Percona repositories
+# Add MariaDB repository
 RUN apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0xF1656F24C74CD1D8 \
     && echo "deb http://ftp.igh.cnrs.fr/pub/mariadb/repo/$MARIADB_MAJOR/debian stretch main" > /etc/apt/sources.list.d/mariadb.list \
-    && apt-key adv --keyserver keys.gnupg.net --recv-keys 0x9334A25F8507EFA5 \
-    && echo "deb https://repo.percona.com/apt stretch main" > /etc/apt/sources.list.d/percona.list \
     && { \
 		     echo 'Package: *'; \
 		     echo 'Pin: release o=MariaDB'; \
@@ -27,14 +24,15 @@ RUN apt-key adv --recv-keys --keyserver keyserver.ubuntu.com 0xF1656F24C74CD1D8
 # sources: https://github.com/osixia/docker-light-baseimage/blob/stable/image/tool/add-multiple-process-stack
 #          https://github.com/osixia/docker-light-baseimage/blob/stable/image/tool/add-service-available
 #          https://github.com/osixia/docker-light-baseimage/blob/stable/image/service-available/:ssl-tools/download.sh
-# Install MariaDB and xtrabackup
+# Install MariaDB and MariaDB Backup
 RUN apt-get -y update \
     && /container/tool/add-multiple-process-stack \
     && /container/tool/add-service-available :ssl-tools \
     && LC_ALL=C DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        mariadb-server=$MARIADB_VERSION \
-       percona-xtrabackup-24 \
-       qpress \
+       mariadb-backup=$MARIADB_VERSION \
+       gzip \
+    && apt-get remove -y --purge --auto-remove curl ca-certificates \
     && apt-get clean \
     && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
 

image/service/mariadb-backup/assets/tool/mariadb-backup

@@ -0,0 +1,19 @@
+#!/bin/bash -e
+
+# Usage: /sbin/mariadb-backup
+
+source /container/run/environment.sh
+
+backupPath="/data/backup"
+
+# delete backups that are over $MARIADB_BACKUP_TTL days
+find "${backupPath}" -type f -mtime +"${MARIADB_BACKUP_TTL}" -exec rm -rf {} \;
+
+dateFileFormat="+%Y%m%dT%H%M%S"
+newBackupFile="${backupPath}/$(date "${dateFileFormat}").gz"
+
+mariabackup --backup --user="${MARIADB_BACKUP_USER}" --password="${MARIADB_BACKUP_PASSWORD}" --stream=xbstream  | gzip > "${newBackupFile}"
+
+chmod 600 "${newBackupFile}"
+
+exit 0

image/service/mariadb-backup/assets/tool/mariadb-restore

@@ -0,0 +1,48 @@
+#!/bin/bash -e
+
+# Usage: /sbin/mariadb-restore dir
+
+# Example : /sbin/mariadb-restore 2018-10-04-17:28:56
+#           will restore data in /data/backup/2018-10-04-17:28:56
+
+
+FILE=$1
+
+source /container/run/environment.sh
+
+# stop mariadb
+sv stop /container/run/process/mariadb || true
+
+# backup mariadb database
+[ -d /tmp/mysql/restore/ ] && rm -rf /tmp/mysql/restore
+
+dateFileFormat="+%Y%m%dT%H%M%S"
+date=$(date "${dateFileFormat}")
+
+mkdir -p "/tmp/mysql/backup/${date}" /tmp/mysql/restore/zip/ /tmp/mysql/restore/data/
+mv /var/lib/mysql/* "/tmp/mysql/backup/${date}" || true
+
+# decompress backup
+cp "/data/backup/${FILE}" /tmp/mysql/restore/zip/data.gz
+gzip -d /tmp/mysql/restore/zip/data.gz
+
+(
+cd /tmp/mysql/restore/data
+mbstream  -x </tmp/mysql/restore/zip/data
+)
+
+# prepare backup
+mariabackup --prepare --target-dir /tmp/mysql/restore/data \
+      --user="${MARIADB_BACKUP_USER}" --password="${MARIADB_BACKUP_PASSWORD}"
+
+# restore backup
+mariabackup --copy-back --target-dir /tmp/mysql/restore/data\
+      --user="${MARIADB_BACKUP_USER}" --password="${MARIADB_BACKUP_PASSWORD}"
+
+# fix file permissions
+chown -R mysql:mysql /var/lib/mysql/
+
+# restart mariadb
+sv start /container/run/process/mariadb
+
+exit 0

image/service/mariadb-backup/startup.sh

@@ -0,0 +1,25 @@
+#!/bin/bash -e
+
+# set -x (bash debug) if log level is trace
+# https://github.com/osixia/docker-light-baseimage/blob/stable/image/tool/log-helper
+log-helper level eq trace && set -x
+
+# add image tools
+ln -sf "${CONTAINER_SERVICE_DIR}"/mariadb-backup/assets/tool/* /sbin/
+
+# add cron jobs
+ln -sf "${CONTAINER_SERVICE_DIR}/mariadb-backup/assets/cronjobs" /etc/cron.d/mariadb-backup
+chmod 600 "${CONTAINER_SERVICE_DIR}/mariadb-backup/assets/cronjobs"
+
+
+FIRST_START_DONE="${CONTAINER_STATE_DIR}/docker-mariadb-mariadb-backup-first-start-done"
+# container first start
+if [ ! -e "${FIRST_START_DONE}" ]; then
+
+  # adapt cronjobs file
+  sed -i "s|{{ MARIADB_BACKUP_CRON_EXP }}|${MARIADB_BACKUP_CRON_EXP}|g" "${CONTAINER_SERVICE_DIR}/mariadb-backup/assets/cronjobs"
+
+  touch "${FIRST_START_DONE}"
+fi
+
+exit 0

image/service/mariadb/process.sh

@@ -6,7 +6,7 @@ log-helper level eq trace && set -x
 
 if [ "${MARIADB_SSL,,}" == "true" ]; then
   log-helper info "Start mariadb with ssl..."
-  exec /usr/bin/mysqld_safe --ssl --ssl-cipher=$MARIADB_SSL_CIPHER_SUITE --ssl-ca=${CONTAINER_SERVICE_DIR}/mariadb/assets/certs/$MARIADB_SSL_CA_CRT_FILENAME --ssl-cert=${CONTAINER_SERVICE_DIR}/mariadb/assets/certs/$MARIADB_SSL_CRT_FILENAME --ssl-key=${CONTAINER_SERVICE_DIR}/mariadb/assets/certs/$MARIADB_SSL_KEY_FILENAME
+  exec /usr/bin/mysqld_safe --ssl --ssl-cipher="${MARIADB_SSL_CIPHER_SUITE}" --ssl-ca="${CONTAINER_SERVICE_DIR}/mariadb/assets/certs/${MARIADB_SSL_CA_CRT_FILENAME}" --ssl-cert="${CONTAINER_SERVICE_DIR}/mariadb/assets/certs/${MARIADB_SSL_CRT_FILENAME}" --ssl-key="${CONTAINER_SERVICE_DIR}/mariadb/assets/certs/${MARIADB_SSL_KEY_FILENAME}"
 else
   log-helper info "Start mariadb..."
   exec /usr/bin/mysqld_safe