General Information

Installing and configuring Urbackup server and client.

“UrBackup is an easy to setup Open Source client/server backup system, that through a combination of image and file backups accomplishes both data safety and a fast restoration time.”

Official Site: https://www.urbackup.org/

Documentation: https://www.urbackup.org/administration_manual.html

Checklist

• Distro(s): Enterprise Linux
• Server and client system available.

Installing the Urbackup Server

• Add CentOS repo

  wget https://download.opensuse.org/repositories/home:uroni/CentOS_7/home:uroni.repo -O /etc/yum.repos.d/urbackup-server.repo

  • Or select different distro here: https://software.opensuse.org/download.html?project=home%3Auroni&package=urbackup-server

• Install server

  yum install urbackup-server

• Enable and start

  systemctl enable urbackup-server
  systemctl start urbackup-server

Installing a Nginx web proxy frontend.

• Obtain a SSL certificate from a certificate authority or generate a self signed cert
• Install Nginx
• Configure urbackup proxy pass

  vim /etc/nginx/conf.d/urbackup.conf
   
  server {
    server_name NAME.DOMAIN;
   
    # Redirect all requests to https
    return 301 https://$server_name$request_uri;
  }
   
  server {
    listen 443 ssl;
    server_name NAME.DOMAIN;
   
    # SSL Cert and Key
    ssl_certificate ssl/NAME.DOMAIN.crt;
    ssl_certificate_key ssl/NAME.DOMAIN.key;
   
    # SSL Protocols, Ciphers
    ssl_protocols TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
   
    # HSTS header
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
   
    location / { 
      proxy_pass http://localhost:55414/;
    }   
  }

• Enable and start

  systemctl enable nginx
  systemctl start nginx

Optional - Needed if not using web proxy

firewall-cmd --add-port=55413/tcp --permanent
firewall-cmd --add-port=55414/tcp --permanent

Optional - Support client backups over the internet

firewall-cmd --add-port=55415/tcp --permanent

Web proxy http and https

firewall-cmd --add-service=http --permanent
firewall-cmd --add-service=https --permanent

Needed? - Incoming client discovery

firewall-cmd --add-port=35622/udp --permanent

• I had trouble with client discovery until I added this to the server side incoming firewall rules. (even though the documentation says this is outgoing only on the server side)

Reload and list rules

firewall-cmd --reload
firewall-cmd --list-all

Installing the Urbackup client.

• Install the 2.2.5 Linux client

  TF=$(mktemp) && wget "https://hndl.urbackup.org/Client/2.2.5/UrBackup%20Client%20Linux%202.2.5.sh" -O $TF && sudo sh $TF; rm $TF

  • You will be prompted for:
    • Proceed with install into /usr/local [Y/n]
    • Snapshot mechanism used for backups (1, 2, or 4)
      • Using a snapshot method, a block snapshot is made of your data and that is used for the backup. (This is typically more reliable than a straight up file copy operation)
      • 1) dattobd volume snapshot kernel module from https://github.com/datto/dattobd
        • Install instructions: https://github.com/datto/dattobd/blob/master/INSTALL.md
        • Good: No extra space needed, creates copy on write special device (not tested, just read about it)
        • Bad: Extra kernel module install
      • 2) LVM - Logical Volume Manager snapshots
        • Good: Native snapshots (if LVM already setup on system)
        • Bad: Snapshots require free space in the same volume group as the logical volume being backed up, equal to the used size. (snapshots removed after backup completes)
      • 4) Use no snapshot mechanism
        • Good: Simple, no add-ons, no extra space required
        • Bad: Since it is essentially a rsync, if files are modified during backup, there could be inconsistencies.

• Enable and start (should automatically do this)

  systemctl enable urbackupclientbackend
  systemctl start urbackupclientbackend

Client firewall ports

firewall-cmd --add-port=35621/tcp --permanent
firewall-cmd --add-port=35622/udp --permanent
firewall-cmd --add-port=35623/tcp --permanent

Reload and list rules

firewall-cmd --reload
firewall-cmd --list-all

• To uninstall the client

  /usr/local/sbin/uninstall_urbackupclient

Administrative tasks for Urbackup server and clients.

Official administrative documentation: https://www.urbackup.org/administration_manual.html

Web console

• Through web proxy: https://NAME.DOMAIN/
• No web proxy (direct): http://NAME.DOMAIN:55414

The first time you navigate to the web console, all admin settings are accessible to the world.

To set an admin user and password:

• Open the web console
• Navigate to “Settings” > “Users”
• Click the “Create User” button
  • Set a username and password
  • Set “Rights for” to “Administrator
  • Click “Create”
• Refresh the page and you will now be prompted to login

Some initial server config changes to make.

Login to the web console and navigate to “Settings”.

• Settings → General → Server
  • Backup storage path: Full path to file system location where you want backups stored
  • Server URL: Set to URL that you can access the web console (so clients can access and restore files)
  • Download client from update server: Check if you want the server to look for new Urbackup client versions and download them automatically.
  • Autoupdate clients: Check to keep clients auto updated
  • Cleanup time window: Set the days/hours during which cleanup jobs can run.
    • Click the ”?“ for cleanup window syntax.
    • Note: Its a good idea to have a window of an hour or two in which backups will not run so cleanup jobs can run successfully.

• Settings → General → File Backups
  • Two types of wildcards are available:
    • * (star) → Matches any character, including file system path slash (forward slash for Linux, back slash for Windows)
    • : (colon) → Matches any character, except file system path slash
  • Excluded files (with wildcards): List of all files/directories to exclude from backups (semi-colon separated).
    • Examples

      # Exclude movie files
      *.avi;*.mkv;*.mpg;*.mpeg
       
      # Exclude all 'tmp' and 'temp' directories (ANYWHERE in the path)
      */tmp/*;*/temp/*
       
      # Exclude any user's Downloads and Desktop directory and its contents
      /home/:/Downloads/*;/home/:/Desktop/*

  • Included files (with wildcards): List of all files to include from backups (semi-colon separated).
    • If you do not define any includes, all files in the 'Default directories to backup' will be backed up, except items in the 'Excluded files' list.
    • If there is one item in the 'Included files' list, only files that match are backed up.
    • Examples

      # Any user's specific hidden files and directories
      /home/:/.bash_aliases;/home/:/.bashrc;/home/:/.ssh/*;/home/:/.vim/*;/home/:/.vimrc
       
      # Any user's normal directories and their contents
      /home/:/bin/*;/home/:/Desktop/*;/home/:/Documents/*;/home/:/Downloads/*;/home/:/Music/*;/home/:/Pictures/*;/home/:/Videos/*

  • Default directories to backup: List all of the top level directories (semi-colon separated) that you want backed up. Exclusions/inclusions of those directory contents are set above. (Excluded files, Included files)
    • Example

      /home/;/etc/

• Settings → General → Permissions
  • Review the list of permissions and decide what you want each client to be able to do.

• Settings → General → Client
  • Delay after system startup: Delay in minutes after system startup before backups can begin on clients.
  • Backup window: Set the days/hours during which backups can run.
    • Click the ”?“ for backup window syntax.
    • Note: Its a good idea to have a window of an hour or two in which backups will not run so cleanup jobs can run successfully.
  • Perform autoupdates silently: Check if you want client software to update in the background.

• Settings → General → Archive
  • - to add when I get around to it

• Settings → General → Alerts
  • E-Mail addresses: enter email address(es) semi colon separated to get alerts.
  • Send ok mail after backup status goes from not ok to ok: check this to get the recovered email.

• Settings → General → Internet
  • - to add when I get around to it

• Settings → General → Advanced
  • Most defaults are good settings.
  • Database cache size: 512 MB (I increased this from 200)
  • Run backups with background priority: Check this to not impact the clients as much when backups run.

• Settings → Mail - Google SMTP example
  • Mail server name: smtp.gmail.com
  • Mail server port: 587
  • Mail server username: yourgoogleaccount@gmail.com
  • Mail server password: generate an application password here
  • Sender E-mail Address: yourgoogleaccount@gmail.com
  • Send mails only with SSL/TLS: Checked
  • Send test mail to this email: yourgoogleaccount@gmail.com

Various CLI commands on the Urbackup server.

Cleanup Commands

• Remove client that is marked for removal now

  urbackupsrv cleanup --amount 0%

• Remove files not known by urbackup database

  urbackupsrv remove-unknown

Client side configuration options.

Client side snapshot config can be changed after installation if desired

• Edit the client side snapshot configuration
  • /usr/local/etc/urbackup/snapshot.cfg
    • dattobd snapshots

      create_filesystem_snapshot=dattobd_create_filesystem_snapshot
      remove_filesystem_snapshot=dattobd_remove_filesystem_snapshot

    • LVM snapshots

      create_filesystem_snapshot=lvm_create_filesystem_snapshot
      remove_filesystem_snapshot=lvm_remove_filesystem_snapshot

    • No shapshots

      rm /usr/local/etc/urbackup/snapshot.cfg
      touch /usr/local/etc/urbackup/no_filesystem_snapshot

• Restart the backend client service

  systemctl restart urbackupclientbackend

Some commands can be executed on the client side. All of these commands can also be executed via the Urbackup server web console.

Urbackup client control

• /usr/local/bin/urbackupclientctl

Note: Some of these commands (such as browse and restore) require root privileges to work.

• Backup Commands
  • Backup Status

    urbackupclientctl status

  • Start a full backup

    urbackupclientctl start --full

  • Start an incremental backup

    urbackupclientctl start --incremental

• Restore files
  • Browse backups or files/directories in backups

    urbackupclientctl browse --path /path/here/ --backupid last

    • Use to determine which file id version you will want to restore
  • Restore a file/directory from backup

    urbackupclientctl restore-start --path /path/of/fileordir/to/restore --backupid <id>

    • backupid is the id of the file version to restore from or “last” for last backup. The ids can be viewed via the “browse” command.

• Backed Up Directories
  • List directories that are being backed up

    urbackupclientctl list-backupdirs

  • Add directory to backup list

    urbackupclientctl add-backupdir --path /path/here/

  • Remove directory from backup list

    urbackupclientctl remove-backupdir --path /path/here/

  • Warning: If you modify directories to be backed up from an individual client (add or remove), it will no longer use the global file backup settings.

The backup location on the server can be rsync'ed to another location to create a backup of the backups.

This is useful if you were to attach removable storage to the server, run the rsync of the latest backups, and take the removable storage offsite.

Due to urbackup's space saving methods, rsync'ing the most current backup with the correct rsync options will provide the most accurate mirror.

Example

• Urbackup location: /home/urbackup/
• The clients directory has symlinks to the latest backups: /home/urbackup/clients/
• Rsync example: sync the latest backups to an external drive

  /usr/bin/rsync --archive --partial --progress --delete --copy-links -h /home/urbackup/clients/ /mnt/usb-storage/