linux_wiki:urbackup_server_and_client

Urbackup Server And Client

General Information

Installing and configuring Urbackup server and client.

“UrBackup is an easy to setup Open Source client/server backup system, that through a combination of image and file backups accomplishes both data safety and a fast restoration time.”

Official Site: https://www.urbackup.org/

Documentation: https://www.urbackup.org/administration_manual.html

Checklist

  • Distro(s): Enterprise Linux
  • Server and client system available.

Server Install

Installing the Urbackup Server

  • Install server
    yum install urbackup-server
  • Enable and start
    systemctl enable urbackup-server
    systemctl start urbackup-server

Installing a Nginx web proxy frontend.

  • Obtain a SSL certificate from a certificate authority or generate a self signed cert
  • Configure urbackup proxy pass
    vim /etc/nginx/conf.d/urbackup.conf
     
    server {
      server_name NAME.DOMAIN;
     
      # Redirect all requests to https
      return 301 https://$server_name$request_uri;
    }
     
    server {
      listen 443 ssl;
      server_name NAME.DOMAIN;
     
      # SSL Cert and Key
      ssl_certificate ssl/NAME.DOMAIN.crt;
      ssl_certificate_key ssl/NAME.DOMAIN.key;
     
      # SSL Protocols, Ciphers
      ssl_protocols TLSv1.2;
      ssl_ciphers HIGH:!aNULL:!MD5;
      ssl_prefer_server_ciphers on;
     
      # HSTS header
      add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
     
      location / { 
        proxy_pass http://localhost:55414/;
      }   
    }
  • Enable and start
    systemctl enable nginx
    systemctl start nginx
Port Protocol In/Out Description
55413 tcp Incoming FastCG for web interface (needed if no web proxy)
55414 tcp Incoming HTTP web interface (needed if no web proxy)
55415 tcp Incoming Internet clients connect for backups
35623 udp Outgoing Broadcast for client discovery
35623 tcp Incoming Broadcast for client discovery (maybe needed?)
443 tcp Incoming HTTPS Web Proxy (if used)
80 tcp Incoming HTTP Web Proxy (if used)


Optional - Needed if not using web proxy

firewall-cmd --add-port=55413/tcp --permanent
firewall-cmd --add-port=55414/tcp --permanent


Optional - Support client backups over the internet

firewall-cmd --add-port=55415/tcp --permanent


Web proxy http and https

firewall-cmd --add-service=http --permanent
firewall-cmd --add-service=https --permanent


Needed? - Incoming client discovery

firewall-cmd --add-port=35622/udp --permanent
  • I had trouble with client discovery until I added this to the server side incoming firewall rules. (even though the documentation says this is outgoing only on the server side)


Reload and list rules

firewall-cmd --reload
firewall-cmd --list-all

Client Install

Installing the Urbackup client.

  • Install the 2.2.5 Linux client
    TF=$(mktemp) && wget "https://hndl.urbackup.org/Client/2.2.5/UrBackup%20Client%20Linux%202.2.5.sh" -O $TF && sudo sh $TF; rm $TF
    • You will be prompted for:
      • Proceed with install into /usr/local [Y/n]
      • Snapshot mechanism used for backups (1, 2, or 4)
        • Using a snapshot method, a block snapshot is made of your data and that is used for the backup. (This is typically more reliable than a straight up file copy operation)
        • 1) dattobd volume snapshot kernel module from https://github.com/datto/dattobd
        • 2) LVM - Logical Volume Manager snapshots
          • Good: Native snapshots (if LVM already setup on system)
          • Bad: Snapshots require free space in the same volume group as the logical volume being backed up, equal to the used size. (snapshots removed after backup completes)
        • 4) Use no snapshot mechanism
          • Good: Simple, no add-ons, no extra space required
          • Bad: Since it is essentially a rsync, if files are modified during backup, there could be inconsistencies.
  • Enable and start (should automatically do this)
    systemctl enable urbackupclientbackend
    systemctl start urbackupclientbackend
Port Protocol In/Out Description
35621 tcp Incoming Transfers during file backups
35622 udp Incoming Broadcasts for discovery
35623 tcp Incoming Server commands and image backups


Client firewall ports

firewall-cmd --add-port=35621/tcp --permanent
firewall-cmd --add-port=35622/udp --permanent
firewall-cmd --add-port=35623/tcp --permanent


Reload and list rules

firewall-cmd --reload
firewall-cmd --list-all
  • To uninstall the client
    /usr/local/sbin/uninstall_urbackupclient

Urbackup Administration

Administrative tasks for Urbackup server and clients.

Official administrative documentation: https://www.urbackup.org/administration_manual.html

Web console

The first time you navigate to the web console, all admin settings are accessible to the world.

To set an admin user and password:

  • Open the web console
  • Navigate to “Settings” > “Users”
  • Click the “Create User” button
    • Set a username and password
    • Set “Rights for” to “Administrator
    • Click “Create”
  • Refresh the page and you will now be prompted to login

Some initial server config changes to make.


Login to the web console and navigate to “Settings”.

  • Settings → General → Server
    • Backup storage path: Full path to file system location where you want backups stored
    • Server URL: Set to URL that you can access the web console (so clients can access and restore files)
    • Download client from update server: Check if you want the server to look for new Urbackup client versions and download them automatically.
    • Autoupdate clients: Check to keep clients auto updated
    • Cleanup time window: Set the days/hours during which cleanup jobs can run.
      • Click the ”?“ for cleanup window syntax.
      • Note: Its a good idea to have a window of an hour or two in which backups will not run so cleanup jobs can run successfully.


  • Settings → General → File Backups
    • Two types of wildcards are available:
      • * (star) → Matches any character, including file system path slash (forward slash for Linux, back slash for Windows)
      • : (colon) → Matches any character, except file system path slash
    • Excluded files (with wildcards): List of all files/directories to exclude from backups (semi-colon separated).
      • Examples
        # Exclude movie files
        *.avi;*.mkv;*.mpg;*.mpeg
         
        # Exclude all 'tmp' and 'temp' directories (ANYWHERE in the path)
        */tmp/*;*/temp/*
         
        # Exclude any user's Downloads and Desktop directory and its contents
        /home/:/Downloads/*;/home/:/Desktop/*
    • Included files (with wildcards): List of all files to include from backups (semi-colon separated).
      • If you do not define any includes, all files in the 'Default directories to backup' will be backed up, except items in the 'Excluded files' list.
      • If there is one item in the 'Included files' list, only files that match are backed up.
      • Examples
        # Any user's specific hidden files and directories
        /home/:/.bash_aliases;/home/:/.bashrc;/home/:/.ssh/*;/home/:/.vim/*;/home/:/.vimrc
         
        # Any user's normal directories and their contents
        /home/:/bin/*;/home/:/Desktop/*;/home/:/Documents/*;/home/:/Downloads/*;/home/:/Music/*;/home/:/Pictures/*;/home/:/Videos/*
    • Default directories to backup: List all of the top level directories (semi-colon separated) that you want backed up. Exclusions/inclusions of those directory contents are set above. (Excluded files, Included files)
      • Example
        /home/;/etc/


  • Settings → General → Permissions
    • Review the list of permissions and decide what you want each client to be able to do.


  • Settings → General → Client
    • Delay after system startup: Delay in minutes after system startup before backups can begin on clients.
    • Backup window: Set the days/hours during which backups can run.
      • Click the ”?“ for backup window syntax.
      • Note: Its a good idea to have a window of an hour or two in which backups will not run so cleanup jobs can run successfully.
    • Perform autoupdates silently: Check if you want client software to update in the background.


  • Settings → General → Archive
    • FIXME - to add when I get around to it


  • Settings → General → Alerts
    • E-Mail addresses: enter email address(es) semi colon separated to get alerts.
    • Send ok mail after backup status goes from not ok to ok: check this to get the recovered email.


  • Settings → General → Internet
    • FIXME - to add when I get around to it


  • Settings → General → Advanced
    • Most defaults are good settings.
    • Database cache size: 512 MB (I increased this from 200)
    • Run backups with background priority: Check this to not impact the clients as much when backups run.


  • Settings → Mail - Google SMTP example
    • Mail server name: smtp.gmail.com
    • Mail server port: 587
    • Mail server username: yourgoogleaccount@gmail.com
    • Sender E-mail Address: yourgoogleaccount@gmail.com
    • Send mails only with SSL/TLS: Checked
    • Send test mail to this email: yourgoogleaccount@gmail.com

Various CLI commands on the Urbackup server.

Cleanup Commands

  • Remove client that is marked for removal now
    urbackupsrv cleanup --amount 0%
  • Remove files not known by urbackup database
    urbackupsrv remove-unknown

Client side configuration options.

Client side snapshot config can be changed after installation if desired

  • Edit the client side snapshot configuration
    • /usr/local/etc/urbackup/snapshot.cfg
      • dattobd snapshots
        create_filesystem_snapshot=dattobd_create_filesystem_snapshot
        remove_filesystem_snapshot=dattobd_remove_filesystem_snapshot
      • LVM snapshots
        create_filesystem_snapshot=lvm_create_filesystem_snapshot
        remove_filesystem_snapshot=lvm_remove_filesystem_snapshot
      • No shapshots
        rm /usr/local/etc/urbackup/snapshot.cfg
        touch /usr/local/etc/urbackup/no_filesystem_snapshot
  • Restart the backend client service
    systemctl restart urbackupclientbackend

Some commands can be executed on the client side. All of these commands can also be executed via the Urbackup server web console.

Urbackup client control

  • /usr/local/bin/urbackupclientctl


Note: Some of these commands (such as browse and restore) require root privileges to work.

  • Backup Commands
    • Backup Status
      urbackupclientctl status
    • Start a full backup
      urbackupclientctl start --full
    • Start an incremental backup
      urbackupclientctl start --incremental


  • Restore files
    • Browse backups or files/directories in backups
      urbackupclientctl browse --path /path/here/ --backupid last
      • Use to determine which file id version you will want to restore
    • Restore a file/directory from backup
      urbackupclientctl restore-start --path /path/of/fileordir/to/restore --backupid <id>
      • backupid is the id of the file version to restore from or “last” for last backup. The ids can be viewed via the “browse” command.


  • Backed Up Directories
    • List directories that are being backed up
      urbackupclientctl list-backupdirs
    • Add directory to backup list
      urbackupclientctl add-backupdir --path /path/here/
    • Remove directory from backup list
      urbackupclientctl remove-backupdir --path /path/here/
    • Warning: If you modify directories to be backed up from an individual client (add or remove), it will no longer use the global file backup settings.

Mirroring the Backups

The backup location on the server can be rsync'ed to another location to create a backup of the backups.

This is useful if you were to attach removable storage to the server, run the rsync of the latest backups, and take the removable storage offsite.

Due to urbackup's space saving methods, rsync'ing the most current backup with the correct rsync options will provide the most accurate mirror.

Example

  • Urbackup location: /home/urbackup/
  • The clients directory has symlinks to the latest backups: /home/urbackup/clients/
  • Rsync example: sync the latest backups to an external drive
    /usr/bin/rsync --archive --partial --progress --delete --copy-links -h /home/urbackup/clients/ /mnt/usb-storage/

  • linux_wiki/urbackup_server_and_client.txt
  • Last modified: 2019/05/25 23:50
  • (external edit)