Differences

This shows you the differences between two versions of the page.

--- linux_wiki:spacewalk [2018/03/06 10:19]
billdozor [Spacewalk]
+++ linux_wiki:spacewalk [2019/05/25 23:50]
@@ Line 1: / Line 1: @@
-====== Spacewalk ======
-**General Information**
-Spacewalk is a centralized system update and config server.\\
-  * Official Site: https://github.com/spacewalkproject/spacewalk
-**Checklist**
-  * Distro(s): Enterprise Linux
-  * Other: [[https://github.com/spacewalkproject/spacewalk/wiki/HowToInstall|Spacewalk server]] setup
-----
-====== Spacecmd ======
-Spacecmd is the command line interface to Spacewalk.\\
-Details here: [[https://www.owlbearconsulting.com/doku.php?id=linux_wiki:spacecmd_spacewalk_cmd_line|Spacecmd]]
-----
-====== Register System with Spacewalk ======
-A [[linux_wiki:spacewalk_register|Spacewalk registration script]] has been created to ease registration.
-----
-===== Re-Register =====
-If you need to re-register a client for any reason, you need the "--force" option when executing rhnreg_ks.
-  * Delete system from Spacewalk<code bash>spacecmd system_delete <SYSTEM></code>
-  * Register system with the --force option<code bash>
-sw_activation_key="1-my-system-key"
-sw_server="my-spacewalk-server.local"
-rhnreg_ks --force --serverUrl=https://${sw_server}/XMLRPC --sslCACert=/usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT --activationkey=${sw_activation_key}
-</code>
-----
-====== Client Check Ins ======
-  * rhnsd => By default, a system will run the rhnsd daemon.
-    * This daemon checks into Spacewalk via rhn_check every 240 mins (4 hours). It is tunable down to a minimum of 60 mins (1 hour).
-  * cron => An alternative is to create a cron job to execute every 30 mins and run rhn_check.
-    * If you do this, you can disable rhnsd, as it provides no other functionality than to run "run_check".
-  * osad => Using osad on the client and osa-dispatcher on the server (with a jabberd daemon as well) is also available.
-    * **This setup is fragile and not recommended**.
-If you do not want to wait for the next automatic check in (via rhnsd or cron), you can force a group of systems to check in by running the "rhn_check" command locally on that system.\\
-To loop through a group of systems and have them check in:
-Example: Loop through the dev system group and have them check in
-<code bash>
-for NODE in $(spacecmd group_listsystems dev); do echo "=>${NODE}"; ssh -qt ${NODE} "sudo /usr/sbin/rhn_check"; done
-</code>
-----
-===== rhnsd config =====
-rhnsd is a daemon that will run rhn_check every 240 mins (by default).
-Configure: /etc/sysconfig/rhn/rhnsd
-<code bash>
-INTERVAL=240
-</code>
-  * Change INTERVAL time to check in time in minutes.
-Ensure it is enabled and started
-  * EL7<code bash>systemctl enable rhnsd
-systemctl start rhnsd</code>
-  * EL6<code bash>chkconfig rhnsd on
-service rhnsd start</code>
-----
-===== rhn_check Cron Job =====
-The alternative to using rhnsd (if you do not want a daemon running or desire more frequent check ins) is a cron job.
-Configure: /etc/cron.d/spacewalk-checkin
-<code bash>
-# Do not e-mail root/anyone about this job
-MAILTO=""
-# .---------------- minute (0 - 59)
-# |  .------------- hour (0 - 23)
-# |  |  .---------- day of month (1 - 31)
-# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
-# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
-# |  |  |  |  |
-# *  *  *  *  * user-name  command to be executed
-# Check in every 30 mins
-*/30 * * * * root /usr/sbin/rhn_check
-</code>
-Optional: Disable rhnsd
-  * EL7<code bash>systemctl disable rhnsd
-systemctl stop rhnsd</code>
-  * EL6<code bash>chkconfig rhnsd off
-service rhnsd stop</code>
-----
-===== osad =====
-Another option for client communication is using the osad daemon (xmpp/jabber) on the client and osa-dispatcher on the server.
-  * This type of setup is very fragile and needs constant babysitting.
-  * The amount of administration overhead and how often it breaks is not worth the effort for faster execution of scheduled jobs.
-  * **It is HIGHLY recommended to not go this route** and the rest of this documentation assumes rhnsd or the cron job with rhn_check.
-----
-====== Channel Management ======
-About Channels
-  * Systems are subscribed to "Channels"
-  * Channel subscriptions can be changed at any time in the Spacewalk portal, across any amount of systems.
-  * Channels have Repositories assigned to them
-  * This allows for a single repo to back multiple channels
-----
-===== Channel Freezing/Snapshots =====
-In order to facilitate the same updates being applied to the Development, System Test, and the Production environments, it is necessary to clone the original Channels.\\
-This creates a "snapshot in time" of the available packages/errata on the date of clone.\\
-**Note: This copies metadata of the Channel and does not duplicate repo packages**
-To Clone an entire Channel tree:
-  * Login to a system with spacecmd installed
-  * Clone the original base tree to a "snapshot-<date>_" prefix tree
-  * Clone can be performed with spacecmd in batch or interactive mode:
-    * Batch Clone Example => Clone the CentOS 6 tree, giving it the prefix "ss-20151215_" (for the snapshot date)<code bash>spacecmd softwarechannel_clonetree centos6_x86-64_base --prefix "ss-20151103_" --gpg-copy</code>
-      * The above will clone the entire tree(base and child channels), give the shown prefix, copy gpg data, and copy errata data.
-    * Interactive Clone Example<code bash>spacecmd {SSM:0}> softwarechannel_clonetree
-Source Channels:
-centos6_x86-64_base
-centos7_x86-64_base
-Select source channel: centos6_x86-64_base
-Prefix: ss-20151215_
-Copy source channel GPG details? [y/N]: y
-Original State (No Errata) [y/N]: N</code>
-----
-===== Errata Setup =====
-As of 12/15/2015, CentOS does not generate an "updateinfo.xml" file in their repodata directories. This file is responsible for the package to errata mappings. (RHEL, Fedora, EPEL, and Oracle all do this)
-For a workaround, use a script to scrape the CentOS mailing archive lists for the errata.
-  * Git hub project: https://github.com/mike-wendt/spacewalk-centos-errata
-    * This is a bash based project that is a wrapper for the perl based project, making it easy to implement.
-    * Original perl project's git page: https://github.com/stevemeier/cefs
-    * Original perl project's main web page: http://cefs.steve-meier.de/
-The "spacewalk-centos-errata" project is installed to:
-  * Main Dir: /opt/spacewalk-centos-errata/
-    * com.redhat.rhsa-all.xml => File downloaded by the "errata-import.pl" script that contains Red Hat errata info.
-    * errata-import.pl => main perl script that does the work
-    * errata.latest.xml => File downloaded by the "errata-import.pl" script that contains CentOS errata info. (from the mailing lists)
-    * **errata-sync.sh** => Configuration file and parent script that launches "errata-import.pl" with the proper Spacewalk credentials and channel IDs to scan.
-      * **Edit this file to make login credential changes or to include other channels for inclusion in errata scanning.**
-    * install.sh => Downloads the latest "errata-import" script, extracts, and creates the cron job
-  * Cron Job installed to:
-    * /etc/cron.d/spacewalk-centos-errata<code bash>MAILTO=""
-01 * * * root /bin/bash /opt/spacewalk-centos-errata/errata-sync.sh 2>&1 > /opt/spacewalk-centos-errata/errata.log</code>
-----
-====== Config Management ======
-A system is automatically subscribed to the proper configuration channels when it is registered via its Activation Key.
-  * Configuration is NOT pushed to the system automatically.
-  * The config files can be deployed while on the client system or pushed to the client using the Spacewalk server portal or spacecmd.
-----
-===== Compare Configs =====
-To compare the centrally managed files to a system's local config files:
-  * Login to the Spacewalk Web Portal
-  * Find the target system via one of these methods:
-    * Searching in the top right
-    * Browsing all systems by clicking "Systems" on the left navigation
-    * Browsing system groups
-  * Click the systems name
-  * On the systems Overview page, click on the "Configuration" tab underneath the system name.
-  * On the right under "Configurable Actions", click on "Compare all managed files to system"
-  * Click "Schedule Compare"
-    * Refresh the Configuration Overview page or click on the systems "Events" tab to watch status.
-  * On the systems Configuration > Overview page, at the bottom under "Recent Events", locate the "Last Spacewalk System Comparison"
-    * Click the "View Details" link to see which files differ.
-  * Under the Config Files list, click on the "Differences exist" link to view the differences.
-----
-===== Download (Pull) Configs =====
-The various ways to download config files while on the client system.
-Download all config files, from all subscribed config channels
-<code bash>
-rhncfg-client get
-</code>
-Download a specific managed config file
-<code bash>
-rhncfg-client get /etc/resolv.conf
-</code>
-Download all config files from a specific Config Channel ID
-<code bash>
-for FILE in $(rhncfg-client list | awk /config-channel-id/'{print $3}'); do rhncfg-client get ${FILE}; done
-</code>
-----
-===== Deploy (Push) Configs =====
-To deploy configs from the server to a client.
-==== Portal Deploy ====
-  * Login to the Spacewalk Web Portal.
-  * At the top, click on the "Systems" tab.
-  * Find the target system via one of these methods:
-    * Searching in the top right
-    * Browsing all systems by clicking "Systems" on the left navigation
-    * Browsing system groups
-  * Click the systems name
-  * On the systems Overview page, click on the "Configuration" tab underneath the system name.
-  * Click "Deploy Files"
-  * Check the config files to deploy, then click "Deploy Files" at the bottom right.
-  * On the "Confirm Deploy Files" page, set a scheduled day/time or click "Schedule Deploy" to deploy immediately.
-==== Spacecmd Deploy ====
-List config channels a system is subscribed to
-<code bash>
-spacecmd system_listconfigchannels
-</code>
-List config files that a system is subscribed to
-<code bash>
-spacecmd system_listconfigfiles
-</code>
-Deploy all of those config files
-<code bash>
-spacecmd system_deployconfigfiles <SYSTEMS>
-</code>
-  * <SYSTEMS> can be:
-    * a single system name
-    * multiple system names space separated
-    * "group:GROUPNAME"
-----
-===== Create a Local Managed File Override =====
-Some systems will need to have different config files than the centrally managed ones.
-\\
-To create exceptions, or local managed overrides:
-  * Login to the Spacewalk Web Portal.
-  * Find the system (Systems tab at the top)
-  * Click on the system name to go to its Overview page.
-On the system's Details > Overview page:
-  * Click the "Configuration" tab underneath the hostname (not the main Configuration tab in black up top) > "View/Modify Files"
-  * To to right of the File Name to override, click "Override this file"
-  * Click "Import Files" > scroll down under "Import Existing Files"
-  * Check the file to override that exists on the system, click "Import Configuration Files"
-  * After file has been successfully imported, click "Configuration" > "View/Modify Files" > "Local Sandbox"
-  * Check the file > click "Copy Latest to System Channel"
-  * The file will now show up under "Locally-Managed Files" and **will NOT be over written by any centrally managed config file deploys**.
-----
-====== Server Services ======
-Spacewalk server services.
-===== Removing osa/jabber =====
-We won't be using osa-dispatcher or jabberd services, so these can safely be disabled.
-  * EL7<code bash>systemctl disable osa-dispatcher
-systemctl disable jabberd
-systemctl stop osa-dispatcher
-systemctl stop jabberd</code>
-  * EL6<code bash>chkconfig osa-dispatcher off
-chkconfig jabberd off
-service osa-dispatcher stop
-service jabberd stop</code>
-Remove osa and jabber from the main spacewalk-service script.
-  * Edit the script: vim /usr/sbin/spacewalk-service
-  * Find the variable "SERVICES=" and remove "osa-dispatcher" and "jabberd"
-----
-===== Normal Status of Spacewalk Services =====
-After removing osa-dispatcher and jabberd, the status output looks like this:
-<code bash>
-/usr/sbin/spacewalk-service status
-postmaster (pid  29875) is running...
-tomcat6 (pid 29992) is running...                          [  OK  ]
-httpd (pid  30115) is running...
-rhn-search is running (30168).
-cobblerd (pid 30204) is running...
-RHN Taskomatic is running (30236).
-</code>
-  * postmaster => Spacewalk Postgres Database
-  * tomcat6 => Spacewalk application
-  * httpd => Spacewalk portal website
-  * rhn-search => Searching functionality within the portal
-  * cobblerd => Provisioning capability
-  * RHN Taskomatic => Scheduled jobs viewable in the Spacewalk portal
-----
-====== Spacewalk SSL Certificates ======
-The SSL Certificates on the Spacewalk server is used for:
-  * Spacewalk Portal (Apache httpd server)
-**Before manipulating either client or CA cert**
-  * SSH to the Spacewalk server and switch to root
-  * Backup the current ssl-build directory (if it exists already)
-    * <code bash>cp -R /root/ssl-build /root/ssl-build.bak</code>
-----
-===== Client Certificate =====
-Client Certificate default locations:
-  * /etc/httpd/conf/ssl.crt/server.crt
-  * /etc/httpd/conf/ssl.csr/server.csr
-  * /etc/httpd/conf/ssl.key/server.key
-Client Certificate Update Procedure
-  * Order certificate renewal from certificate provider
-  * Download certificate, copy to server's /root/ directory
-  * SSH to the Spacewalk server and switch to root
-  * Copy the current CA cert in use to the ssl-build directory
-    * <code bash>cp /var/www/html/pub/RHN-ORG-TRUSTED-SSL-CERT /root/ssl-build/</code>
-  * Copy NEW client certificate into ssl-build/my-spacewalk-server
-    * <code bash>cp server.crt /root/ssl-build/my-spacewalk-server/</code>
-  * Copy existing client key and CSR into ssl-build/my-spacewalk-server
-    * <code bash>cp /etc/httpd/conf/ssl.key/server.key /root/ssl-build/my-spacewalk-server/
-cp /etc/httpd/conf/ssl.csr/server.csr /root/ssl-build/my-spacewalk-server/</code>
-  * Verify that NEW client cert will work with CA cert
-    * <code bash>openssl verify -CAfile /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT /root/ssl-build/my-spacewalk-server/server.crt</code>
-  * Generate the new client cert RPM
-    * <code bash>rhn-ssl-tool --gen-server --rpm-only --dir /root/ssl-build</code>
-  * Remove old SSL key pair package
-    * <code bash>rpm -e rhn-org-httpd-ssl-key-pair-my-spacewalk-server-1.0-1.noarch</code>
-  * Install new SSL key pair package
-    * <code bash>rpm -ivh /root/ssl-build/my-spacewalk-server/rhn-org-httpd-ssl-key-pair-my-spacewalk-server-1.0-2.noarch.rpm</code>
-  * Restart Spacewalk services
-    * <code bash>spacewalk-service restart</code>
-----
-===== CA Certificate =====
-CA Chain Certificate locations
-  * RPM build location: /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT
-  * Locally installed location: /usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT
-  * Publicly available for clients to download: /var/www/html/pub/RHN-ORG-TRUSTED-SSL-CERT
-    * Also packaged in: /var/www/html/pub/rhn-org-trusted-ssl-cert-1.0-1.noarch.rpm
-Updating the CA certificate will not have to be done very often; only when:
-  * CA cert expires
-  * You change certificate providers
-**WARNING**
-  * Updating the CA certificate on the Spacewalk server will break all communication between the server and the clients.
-  * Each client will need to update to the new CA cert individually before communication can be restored.
-CA Certificate Update Procedure
-  * Download the new single .pem file containing all the certs from the certificate provider.
-  * Copy the PEM file to the Spacewalk server
-  * SSH to the Spacewalk server and switch to root
-  * Cat/view the contents of the PEM file
-    * The top BEGIN/END block is the client cert (server.crt)
-    * The rest is the certificate chain
-      * Copy this into a new file; "RHN-ORG-TRUSTED-SSL-CERT"
-  * Copy into ssl-build directory
-    * <code>cp RHN-ORG-TRUSTED-SSL-CERT /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT</code>
-  * Verify CA cert with the server cert
-    * <code bash>openssl verify -CAfile /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT /root/ssl-build/my-spacewalk-server/server.crt</code>
-  * Generate CA chain RPM
-    * <code bash>rhn-ssl-tool --gen-ca --rpm-only --dir /root/ssl-build</code>
-  * Copy new CA chain cert and RPM into Spacewalk's public directory (for client installation later)
-    * <code bash>cp /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT /var/www/html/pub/
-cp ssl-build/rhn-org-trusted-ssl-cert-1.0-2.noarch.rpm /var/www/html/pub/</code>
-  * Install new CA chain cert on the Spacewalk server
-    * <code bash>rpm -ivh /root/ssl-build/rhn-org-trusted-ssl-cert-1.0-2.noarch.rpm</code>
-  * Update the database
-    * <code bash>rhn-ssl-dbstore -vvv --ca-cert /root/ssl-build/RHN-ORG-TRUSTED-SSL-CERT</code>
-  * Restart the Spacewalk services
-    * <code bash>spacewalk-service restart</code>
-  * **Login to each client and update the CA chain**
-    * <code bash>rpm -ivh https://my-spacewalk-server.local/pub/rhn-org-trusted-ssl-cert-1.0-2.noarch.rpm</code>
-      * Each client will have no communication to the Spacewalk server until this is complete.
-----