Skip to content

Latest commit

 

History

History
218 lines (198 loc) · 11.5 KB

README.md

File metadata and controls

218 lines (198 loc) · 11.5 KB

(IN DEVELOPMENT)

Backupper

The tool is for backuping file from FTP/SFTP source and save it another FTP/SFTP backup destination.

Features

  • Schedule backuping with CRON expression in config file
  • Download files from FTP/SFTP sources
  • Executing commands on source server (SFTP only)
  • Upload files to FTP/SFTP destinations
  • Upload files to Telegram with Bot API (max 50 MB files)
  • Limit the file count in target folder (if limit is reached, oldest backups will be deleted)
  • Limit the total file size in target folder (if limit is reached, oldest backups will be deleted)
  • Limit the target folder by duration (oldest backups than date will be deleted)

Config

Main config file is config.json in the root folder of the project. Also you can use config.example.json as a template.

Main Structure

{
  "timezone": "Europe/Istanbul",
  "dateFormat": "2006-01-02__15-04-05",
  "backups": []
}
Key Description Type
timezone TZ identifier string
dateFormat the format of the date to be added to the file name (Golang time format) string
logLevel Log level (debug, info, warn, error, dpanic, panic, fatal) string
backups Backup schedules array

Backup Schedule Structure

{
  "name": "backup-title",
  "cronExpr": "*/60 * * * * *",
  "callbackUrl": "http://example.com/callback",
  "deleteLocal": true,
  "source": {
    "type": "sftp",
    "info": {
      "host": "127.0.0.1",
      "port": 22,
      "user": "root",
      "pass": "",
      "privateKeyFile": "~/.ssh/custom_id_rsa",
      "privateKeyPass": "",
      "variables": {
        "foo": "bar",
        "bar": 123,
        "baz": true
      },
      "beforeCommands": [
        "echo $foo > /tmp/backupper/$BACKUP_ID/foo.txt",
        "echo $bar > /tmp/backupper/$BACKUP_ID/bar.txt",
        "echo $baz > /tmp/backupper/$BACKUP_ID/baz.txt",
        "cd /opt/foo/bar",
        "zip -r /tmp/backupper/$BACKUP_ID/foo_backup.zip ."
      ],
      "downloads": [
        "foo_backup.zip",
        "foo.txt",
        "bar.txt",
        "baz.txt"
      ],
      "afterCommands": [
        "rm -rf /tmp/backupper/$BACKUP_ID/"
      ]
    }
  },
  "destination": {
    "type": "ftp",
    "deleteAfterUpload": true,
    "info": {
      "host": "backup-ftp.example.com",
      "port": 21,
      "user": "root",
      "pass": "p4ssw0rd",
      "target": "/up/backups/foo/",
      "limitByCount": 3,
      "LimitBySize": 1073741824,
      "limitByDate": "2 DAYS"
    }
  }
}
Key Description Type
name Name of the backup schedule string
cronExpr CRON expression (also supports cronSeconds) string
callbackUrl Callback URL to be called after backup process is completed string
deleteLocal Delete local files after upload process is completed bool
source Source server information object
destination Destination server information object

Source

Key Description Type
type Source server type (ftp/sftp) string
info Source server information object

Source Info (FTP)

Key Description Type
host FTP server host string
port FTP server port int
user FTP server username string
pass FTP server password string
downloads Files to be downloaded from FTP server (only files, not directories) array

Source Info (SFTP)

Key Description Type
host SFPT server host string
port SFTP server port int
user SFTP server username string
pass SFTP server password string
privateKeyFile Private key file path string
passphrase Private key passphrase string
variables Custom variables to be used in SSH commands object
beforeCommands SSH Commands to be executed before download process array
downloads Files to be downloaded from SFTP server (only files, not directories) array
afterCommands SSH Commands to be executed after download process array

SFTP - SSH Command Variables

Variable Description Type
$BACKUP_ID Unique ID of the backup process (generated by the tool) (Nano unix timestamp) string
$BACKUP_NAME Name of the backup schedule string

Destination

Key Description Type
type Destination server type (ftp/sftp/telegram_bot) string
deleteAfterUpload Delete files after upload process is completed bool
info Destination server information object

Destination Info (Telegram with Bot API) (max 50 MB files)

Key Description Type
token Telegram bot token from @BotFather string
chatID Telegram chat ID (channel/group) or public username string

Destination Info (FTP)

Key Description Type
host FTP server host string
port FTP server port int
user FTP server username string
pass FTP server password string
target Target folder on FTP server string
limitByCount Limit the file count in target folder int
limitBySize Limit the total file size in target folder (bytes) int
limitByDate Limit the target folder by duration (duration format) string

Destination Info (SFTP)

Key Description Type
host SFTP server host string
port SFTP server port int
user SFTP server username string
pass SFTP server password string
privateKeyFile Private key file path string
passphrase Private key passphrase string
target Target folder on SFTP server string
limitByCount Limit the file count in target folder int
limitBySize Limit the total file size in target folder (bytes) int
limitByDate Limit the target folder by duration (duration format) string

limitByDate - Duration Format

Format Date Range
5 MINUTES All files before 5 minutes
1 HOUR All files before 1 hour
2 DAYS All files before 2 days
3 WEEKS All files before 3 weeks
4 MONTHS All files before 4 months
5 YEARS All files before 5 years

Multiple durations can be used together. (e.g. 1 HOUR 30 MINUTES)

Callback Post Data

{
  "backup_date": "2023-07-20T15:27:50+03:00",
  "backup_destination": "sftp",
  "backup_destination_result": {
    "totalUploadedFiles": 1,
    "totalUploadedSize": 5820073
  },
  "backup_duration": "19.7989235s",
  "backup_id": "1689856070674044500",
  "backup_name": "test-backup",
  "backup_source": "sftp",
  "backup_ts": 1689856070
}
Key Description Type
backup_date Backup date (RFC3339) string
backup_destination Destination server type (ftp/sftp) string
backup_destination_result Destination server upload result object
backup_duration Backup duration (time.Duration string) string
backup_id Unique ID of the backup process (generated by the tool) (Nano unix timestamp) int
backup_name Name of the backup schedule string
backup_source Source server type (ftp/sftp) string
backup_ts Backup timestamp (Unix seconds) int

Used Modules