Show pageOld revisionsBacklinksBack to top This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong. ====== Command Many Systems: Part1 Basics ====== **General Information** Introduction to sending commands in serial and parallel to a large number of systems. **Checklist** * Distro(s): Any * Other: Use of [[linux_wiki:ssh_pub_priv_keys|SSH Public/Private keys]] so nothing prompts for a password in the loop. ---- ====== Systems in Serial ====== For some things, it is better to do a rolling serial change...such as restarting a web server. This example restarts the Apache webserver on a list of systems serially, and sleeps 10 seconds in between each. 1) Populate a text file with the list of systems, one per line. Example: host_list.txt <code> webserver01 webserver02 webserver03 ... </code> 2) Execute a for loop through that list: <code bash> for NODE in $(cat host_list.txt) do echo -e "\nWorking on ${NODE}..." ssh admin@${NODE} 'service httpd restart' echo "---- Done with ${NODE} ----" sleep 10 done </code> ---- ====== Systems in Parallel ====== If you would rather get something done very quickly, commands can be sent in parallel. 1) Create a script that will operate on 1 system. This script takes 1 argument; a system hostname. <code bash single_host.sh> #!/bin/bash HOST=$1 echo "Executing on ${HOST}..." ssh admin@${HOST} 'command1;command2;command3' </code> 2) Create a script that will call that script. This script takes two+ arguments: * Max Number of forks * Hostnames (space seperated) <code bash parallel_hosts.sh> #!/bin/bash MAX_FORKS=$1 shift FORKS=0 for HOST in $@; do if [ ${FORKS} -ge ${MAX_FORKS} ]; then wait FORKS=0 fi (single_host.sh ${HOST}) & FORKS=$(( ${FORKS} + 1 )) done wait </code> 3) Call the parallel_hosts.sh script. In this example, parallel_hosts.sh will spawn a max of 10 processes at a time: <code bash> parallel_hosts.sh 10 $(cat host_list.txt) </code> ===== Parallel Explanation ===== * parallel_hosts.sh is called with two arguments: 10 and an expanded list of host names * The first argument (10) is assigned to MAX_FORKS * shift (with no number) moves all arguments down a position, so the 10 drops off, and now $1 is the first host name from the text file. * The for loop goes through each argument ($@) starting at $1 * If FORKS is ever equal to the MAX_FORKS, wait is used to pause until all child processes have completed. * Otherwise, call single_host.sh ${HOST}, creating a child process and sending it the current host in the loop to execute on. * Finally, after the loop is complete, a final wait is used to ensure that any other single_host.sh child processes can finish cleanly. ---- linux_wiki/command_many_systems_part1_basics.txt Last modified: 2019/05/25 23:50(external edit)