Differences

This shows you the differences between two versions of the page.

--- linux_wiki:urbackup_server_and_client [2018/07/10 23:26]
billdozor [Client CLI]
+++ linux_wiki:urbackup_server_and_client [2019/05/25 23:50]
@@ Line 1: / Line 1: @@
-====== Urbackup Server And Client ======
-**General Information**
-Installing and configuring Urbackup server and client.
-"UrBackup is an easy to setup Open Source client/server backup system, that through a combination of image and file backups accomplishes both data safety and a fast restoration time."
-Official Site: https://www.urbackup.org/
-Documentation: https://www.urbackup.org/administration_manual.html
-**Checklist**
-  * Distro(s): Enterprise Linux
-  * Server and client system available.
-----
-====== Server Install ======
-Installing the Urbackup Server
-  * Add CentOS repo<code bash>wget https://download.opensuse.org/repositories/home:uroni/CentOS_7/home:uroni.repo -O /etc/yum.repos.d/urbackup-server.repo</code>
-    * Or select different distro here: https://software.opensuse.org/download.html?project=home%3Auroni&package=urbackup-server
-  * Install server<code bash>yum install urbackup-server</code>
-  * Enable and start<code bash>systemctl enable urbackup-server
-systemctl start urbackup-server</code>
-===== Install a Web Proxy Frontend =====
-Installing a Nginx web proxy frontend.
-  * Obtain a SSL certificate from a certificate authority or [[linux_wiki:openssl#self-signed_certificates|generate a self signed cert]]
-  * [[linux_wiki:nginx_http_server|Install Nginx]]
-  * Configure urbackup proxy pass<code bash>vim /etc/nginx/conf.d/urbackup.conf
-server {
-  server_name NAME.DOMAIN;
-  # Redirect all requests to https
-  return 301 https://$server_name$request_uri;
-}
-server {
-  listen 443 ssl;
-  server_name NAME.DOMAIN;
-  # SSL Cert and Key
-  ssl_certificate ssl/NAME.DOMAIN.crt;
-  ssl_certificate_key ssl/NAME.DOMAIN.key;
-  # SSL Protocols, Ciphers
-  ssl_protocols TLSv1.2;
-  ssl_ciphers HIGH:!aNULL:!MD5;
-  ssl_prefer_server_ciphers on;
-  # HSTS header
-  add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
-  location / {
-    proxy_pass http://localhost:55414/;
-  }
-}
-</code>
-  * Enable and start<code bash>systemctl enable nginx
-systemctl start nginx</code>
-===== Server Firewall Rules =====
-^  Port  ^  Protocol  ^  In/Out  ^  Description  ^
-|  55413  |  tcp  |  Incoming  |  FastCG for web interface (needed if no web proxy)  |
-|  55414  |  tcp  |  Incoming  |  HTTP web interface (needed if no web proxy)  |
-|  55415  |  tcp  |  Incoming  |  Internet clients connect for backups  |
-|  35623  |  udp  |  Outgoing  |  Broadcast for client discovery  |
-|  35623  |  tcp  |  Incoming  |  Broadcast for client discovery (**maybe needed?**)  |
-|  443  |  tcp  |  Incoming  |  HTTPS Web Proxy (if used)  |
-|  80  |  tcp  |  Incoming  |  HTTP Web Proxy (if used)  |
-\\
-**Optional** - Needed if **not** using web proxy<code bash>firewall-cmd --add-port=55413/tcp --permanent
-firewall-cmd --add-port=55414/tcp --permanent</code>
-\\
-**Optional** - Support client backups over the internet<code bash>firewall-cmd --add-port=55415/tcp --permanent</code>
-\\
-Web proxy http and https<code bash>firewall-cmd --add-service=http --permanent
-firewall-cmd --add-service=https --permanent</code>
-\\
-**Needed?** - Incoming client discovery<code bash>firewall-cmd --add-port=35622/udp --permanent</code>
-  * I had trouble with client discovery until I added this to the server side incoming firewall rules. (even though the documentation says this is outgoing only on the server side)
-\\
-Reload and list rules<code bash>firewall-cmd --reload
-firewall-cmd --list-all</code>
-----
-====== Client Install ======
-Installing the Urbackup client.
-  * Install the 2.2.5 Linux client<code bash>TF=$(mktemp) && wget "https://hndl.urbackup.org/Client/2.2.5/UrBackup%20Client%20Linux%202.2.5.sh" -O $TF && sudo sh $TF; rm $TF</code>
-    * You will be prompted for:
-      * Proceed with install into /usr/local [Y/n]
-      * Snapshot mechanism used for backups (**1, 2, or 4**)
-        * Using a snapshot method, a block snapshot is made of your data and that is used for the backup. (This is typically more reliable than a straight up file copy operation)
-        * **1) dattobd** volume snapshot kernel module from https://github.com/datto/dattobd
-          * Install instructions: https://github.com/datto/dattobd/blob/master/INSTALL.md
-          * Good: No extra space needed, creates copy on write special device (not tested, just read about it)
-          * Bad: Extra kernel module install
-        * **2) LVM** - Logical Volume Manager snapshots
-          * Good: Native snapshots (if LVM already setup on system)
-          * Bad: Snapshots require free space in the same volume group as the logical volume being backed up, equal to the used size. (snapshots removed after backup completes)
-        * **4) Use no snapshot** mechanism
-          * Good: Simple, no add-ons, no extra space required
-          * Bad: Since it is essentially a rsync, if files are modified during backup, there could be inconsistencies.
-  * Enable and start (should automatically do this)<code bash>systemctl enable urbackupclientbackend
-systemctl start urbackupclientbackend</code>
-===== Client Firewall Rules =====
-^  Port  ^  Protocol  ^  In/Out  ^  Description  ^
-|  35621  |  tcp  |  Incoming  |  Transfers during file backups  |
-|  35622  |  udp  |  Incoming  |  Broadcasts for discovery  |
-|  35623  |  tcp  |  Incoming  |  Server commands and image backups  |
-\\
-Client firewall ports<code bash>firewall-cmd --add-port=35621/tcp --permanent
-firewall-cmd --add-port=35622/udp --permanent
-firewall-cmd --add-port=35623/tcp --permanent</code>
-\\
-Reload and list rules<code bash>firewall-cmd --reload
-firewall-cmd --list-all</code>
-===== Client Uninstall =====
-  * To uninstall the client<code bash>/usr/local/sbin/uninstall_urbackupclient</code>
-----
-====== Urbackup Administration ======
-Administrative tasks for Urbackup server and clients.
-Official administrative documentation: https://www.urbackup.org/administration_manual.html
-===== Server Management Console =====
-Web console
-  * Through web proxy: https://NAME.DOMAIN/
-  * No web proxy (direct): http://NAME.DOMAIN:55414
-===== Server Admin Password =====
-The first time you navigate to the web console, all admin settings are accessible to the world.
-To set an admin user and password:
-  * Open the web console
-  * Navigate to "Settings" > "Users"
-  * Click the "Create User" button
-    * Set a username and password
-    * Set "Rights for" to "Administrator
-    * Click "Create"
-  * Refresh the page and you will now be prompted to login
-===== Server Config =====
-Some initial server config changes to make.
-----
-Login to the web console and navigate to "Settings".
-  * **Settings -> General -> Server**
-    * Backup storage path: Full path to file system location where you want backups stored
-    * Server URL: Set to URL that you can access the web console (so clients can access and restore files)
-    * Download client from update server: Check if you want the server to look for new Urbackup client versions and download them automatically.
-    * Autoupdate clients: Check to keep clients auto updated
-    * Cleanup time window: Set the days/hours during which cleanup jobs can run.
-      * Click the "?" for cleanup window syntax.
-      * **Note**: Its a good idea to have a window of an hour or two in which backups will not run so cleanup jobs can run successfully.
-\\
-  * **Settings -> General -> File Backups**
-    * Two types of wildcards are available:
-      * *  (star) -> Matches any character, including file system path slash (forward slash for Linux, back slash for Windows)
-      * :  (colon) -> Matches any character, **except file system path slash**
-    * Excluded files (with wildcards): List of all files/directories to exclude from backups (**semi-colon separated**).
-      * Examples<code bash># Exclude movie files
-*.avi;*.mkv;*.mpg;*.mpeg
-# Exclude all 'tmp' and 'temp' directories (ANYWHERE in the path)
-*/tmp/*;*/temp/*
-# Exclude any user's Downloads and Desktop directory and its contents
-/home/:/Downloads/*;/home/:/Desktop/*</code>
-    * Included files (with wildcards): List of all files to include from backups (**semi-colon separated**).
-      * If you do not define any includes, all files in the 'Default directories to backup' will be backed up, except items in the 'Excluded files' list.
-      * If there is one item in the 'Included files' list, only files that match are backed up.
-      * Examples<code bash># Any user's specific hidden files and directories
-/home/:/.bash_aliases;/home/:/.bashrc;/home/:/.ssh/*;/home/:/.vim/*;/home/:/.vimrc
-# Any user's normal directories and their contents
-/home/:/bin/*;/home/:/Desktop/*;/home/:/Documents/*;/home/:/Downloads/*;/home/:/Music/*;/home/:/Pictures/*;/home/:/Videos/*</code>
-    * Default directories to backup: List all of the top level directories (**semi-colon separated**) that you want backed up. Exclusions/inclusions of those directory contents are set above. (Excluded files, Included files)
-      * Example<code bash>/home/;/etc/</code>
-\\
-  * **Settings -> General -> Permissions**
-    * Review the list of permissions and decide what you want each client to be able to do.
-\\
-  * **Settings -> General -> Client**
-    * Delay after system startup: Delay in minutes after system startup before backups can begin on clients.
-    * Backup window: Set the days/hours during which backups can run.
-      * Click the "?" for backup window syntax.
-      * **Note**: Its a good idea to have a window of an hour or two in which backups will not run so cleanup jobs can run successfully.
-    * Perform autoupdates silently: Check if you want client software to update in the background.
-\\
-  * **Settings -> General -> Archive**
-    * FIXME - to add when I get around to it
-\\
-  * **Settings -> General -> Alerts**
-    * E-Mail addresses: enter email address(es) semi colon separated to get alerts.
-    * Send ok mail after backup status goes from not ok to ok: check this to get the recovered email.
-\\
-  * **Settings -> General -> Internet**
-    * FIXME - to add when I get around to it
-\\
-  * **Settings -> General -> Advanced**
-    * Most defaults are good settings.
-    * Database cache size: 512 MB    (I increased this from 200)
-    * Run backups with background priority: Check this to not impact the clients as much when backups run.
-\\
-  * **Settings -> Mail** - Google SMTP example
-    * Mail server name: smtp.gmail.com
-    * Mail server port: 587
-    * Mail server username: yourgoogleaccount@gmail.com
-    * Mail server password: [[https://security.google.com/settings/security/apppasswords|generate an application password here]]
-    * Sender E-mail Address: yourgoogleaccount@gmail.com
-    * Send mails only with SSL/TLS: Checked
-    * Send test mail to this email: yourgoogleaccount@gmail.com
-===== Server CLI =====
-Various CLI commands on the Urbackup server.
-**Cleanup Commands**
-  * Remove client that is marked for removal now<code bash>urbackupsrv cleanup --amount 0%</code>
-  * Remove files not known by urbackup database<code bash>urbackupsrv remove-unknown</code>
-----
-===== Client Config =====
-Client side configuration options.
-==== Snapshot Configs ====
-Client side snapshot config can be changed after installation if desired
-  * Edit the client side snapshot configuration
-    * /usr/local/etc/urbackup/snapshot.cfg
-      * dattobd snapshots<code bash>create_filesystem_snapshot=dattobd_create_filesystem_snapshot
-remove_filesystem_snapshot=dattobd_remove_filesystem_snapshot</code>
-      * LVM snapshots<code bash>create_filesystem_snapshot=lvm_create_filesystem_snapshot
-remove_filesystem_snapshot=lvm_remove_filesystem_snapshot</code>
-      * No shapshots<code bash>rm /usr/local/etc/urbackup/snapshot.cfg
-touch /usr/local/etc/urbackup/no_filesystem_snapshot</code>
-  * Restart the backend client service<code bash>systemctl restart urbackupclientbackend</code>
-----
-===== Client CLI =====
-Some commands can be executed on the client side. All of these commands can also be executed via the Urbackup server web console.
-Urbackup client control
-  * /usr/local/bin/urbackupclientctl
-\\
-**Note**: Some of these commands (such as browse and restore) require root privileges to work.
-  * Backup Commands
-    * Backup Status<code bash>urbackupclientctl status</code>
-    * Start a full backup<code bash>urbackupclientctl start --full</code>
-    * Start an incremental backup<code bash>urbackupclientctl start --incremental</code>
-\\
-  * Restore files
-    * Browse backups or files/directories in backups<code bash>urbackupclientctl browse --path /path/here/ --backupid last</code>
-      * Use to determine which file id version you will want to restore
-    * Restore a file/directory from backup<code bash>urbackupclientctl restore-start --path /path/of/fileordir/to/restore --backupid <id></code>
-      * backupid is the id of the file version to restore from or "last" for last backup. The ids can be viewed via the "browse" command.
-\\
-  * Backed Up Directories
-    * List directories that are being backed up<code bash>urbackupclientctl list-backupdirs</code>
-    * Add directory to backup list<code bash>urbackupclientctl add-backupdir --path /path/here/</code>
-    * Remove directory from backup list<code bash>urbackupclientctl remove-backupdir --path /path/here/</code>
-    * **Warning**: If you modify directories to be backed up from an individual client (add or remove), it will no longer use the global file backup settings.
-----
-====== Mirroring the Backups ======
-The backup location on the server can be rsync'ed to another location to create a backup of the backups.
-This is useful if you were to attach removable storage to the server, run the rsync of the latest backups, and take the removable storage offsite.
-Due to urbackup's space saving methods, rsync'ing the most current backup with the correct rsync options will provide the most accurate mirror.
-Example
-  * Urbackup location: /home/urbackup/
-  * The clients directory has symlinks to the latest backups: /home/urbackup/clients/
-  * Rsync example: sync the latest backups to an external drive<code bash>/usr/bin/rsync --archive --partial --progress --delete --copy-links -h /home/urbackup/clients/ /mnt/usb-storage/</code>
-----