Show pageOld revisionsBacklinksBack to top This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong. ====== Load Balancing with HAProxy And Keepalived ====== **General Information** Creating a highly available pair of load balancers with HAProxy and Keepalived. **Checklist** * Number of systems * 2 servers to be load balancers * 2 servers for web servers (in the example) * Distro(s): Enterprise Linux 7 ---- ====== Network Addressing Setup ====== Network configuration used in the examples below. Load Balancers * Server "lb01" -> 10.1.2.1 (eth0) * Server "lb02" -> 10.1.2.2 (eth0) * "lbvip" -> 10.1.2.3 (load balancer virtual IP - floats between servers) Web Servers (used in haproxy example config) * web01 -> 10.1.2.50 * web02 -> 10.1.2.51 \\ {{ haproxy_keepalived_example.jpg |}} \\ ---- ====== Install ====== Install the required packages on the load balancer servers * KeepAliveD (high availability)<code bash>yum install keepalived</code> * HA-Proxy (load balancing)<code bash>yum install haproxy</code> ---- ====== Configure ====== Configuring keepalived and haproxy. ---- ===== Keepalived ===== Keepalived utlizes a Linux kernel implementation of VRRP. (Virtual Router Redundancy Protocol) Official Site: http://www.keepalived.org/ * Configure all nodes with these keepalive settings (/etc/keepalived/keepalived.conf). **Example**:<code bash>! Configuration File for keepalived vrrp_script check_haproxy { script "killall -0 haproxy" # check the haproxy process timeout 1 interval 2 # every 2 seconds weight 2 # add 2 points if OK } vrrp_instance VI_1 { state BACKUP # All instances 'BACKUP' to prevent VIP flapping interface eth0 virtual_router_id 51 priority 100 # All instances same priority to prevent VIP flapping advert_int 1 authentication { auth_type PASS auth_pass PASSWORDHERE } virtual_ipaddress { 10.1.2.3 } track_script { check_haproxy } }</code> ---- ===== HA-Proxy ===== HAProxy is a TCP/HTTP load balancer. Official Site: http://www.haproxy.org/ ==== Main Config ==== * Configure HA-Proxy (/etc/haproxy/haproxy.cfg) * Remove all example frontend and backend config sections (leave default section) * Add a section for the HAProxy Stats page<code bash>#--------------------------------------------------------------------- # HAProxy Stats #--------------------------------------------------------------------- listen stats # SSL Mode and Cert bind *:9000 ssl crt /etc/pki/tls/mycertfiles.pem mode http # Enable Stats and Hide Version stats enable stats hide-version # Authentication realm. This can be set to anything. Escape space characters with a backslash. stats realm HAProxy\ Statistics # The virtual URL to access the stats page stats uri /haproxy_stats # The user/pass you want to use. Change this password! stats auth admin:adminpassword</code> * The pem certificate file is a concatenation of the SSL key, cert, and certificate authority. Example<code bash>cat mykey.key mycert.crt myCAs.crt >> mycertfiles.pem</code> ==== Frontend/Backend Configs ==== * Create new directory to hold frontend/backend config files<code bash>mkdir /etc/haproxy/config.d</code> * Create new frontend/backend config files (Example: /etc/haproxy/config.d/http.cfg) * Add New frontend/backend sections **Example**:<code bash>#--------------------------------------------------------------------- # fe_http frontend which proxys to the backends #--------------------------------------------------------------------- frontend fe_http *:80 # Log format option httplog # Timeout Settings #no option http-server-close #timeout client 1m #default: 50s #-- ACLs - Match HTTP Requests --# acl url_web path_beg -i /mywebsite #-- Backend Selection based on ACLs --# use_backend be_web_pool1 if url_web # If not using ACLs for backend selection or to have a fall back selection #default_backend be_web_pool1 #--------------------------------------------------------------------- # Backend Configuration #--------------------------------------------------------------------- backend be_web_pool1 # Replace "/mywebsite/" with "/" at the beginning of the request reqirep ^([^\ ]*\ /)mywebsite[/]?(.*) \1\2 # Backend Protocol mode http #-- Timeout Settings --# #timeout connect 1m #default: 5s #timeout server 2m #default: 50s #-- Health check options --# # Use http layer 7 check instead of default layer 4 port check option httpchk HEAD / # inter: How often to execute a health check (default: 2s) # rise: Number of consecutive checks before server is UP (default: 2) # fall: Number of consecutive checks before server is DOWN (default: 3) default-server inter 5s rise 2 fall 3 # timeout check: Fail health check after x seconds of no response (default: 10s) timeout check 12s #-- Balancing --# balance leastconn # fullconn: does nothing since we are not using minconn (just makes the dashboard less confusing) fullconn 1000 server web01 10.1.2.50:80 check maxconn 500 server web02 10.1.2.51:80 check maxconn 500</code> * Ensure each additional config file in config.d/ is setup in haproxy's environment options(/etc/sysconfig/haproxy)<code bash># Config files specifying frontend/backends OPTIONS="-f /etc/haproxy/config.d/http.cfg"</code> * Multiple config files example:<code bash>OPTIONS="-f /etc/haproxy/config.d/http.cfg -f /etc/haproxy/config.d/otherfrontend.cfg"</code> ==== Additional Config Examples ==== **Session Persistence** * Cookies: Application layer persistence (app needs to support cookies)<code bash> #-- Balancing --# balance leastconn # Use Cookie for Session Persistence cookie SERVERID insert indirect nocache # fullconn: does nothing since we are not using minconn (just makes the dashboard less confusing) fullconn 1000 server web01 10.1.2.50:80 check cookie web01 maxconn 500 server web02 10.1.2.51:80 check cookie web02 maxconn 500</code> * Source IP: Affinity based on source IP hash (app doesn't need to know about it)<code bash> #-- Balancing --# balance source # fullconn: does nothing since we are not using minconn (just makes the dashboard less confusing) fullconn 1000 server web01 10.1.2.50:80 check maxconn 500 server web02 10.1.2.51:80 check maxconn 500</code> ---- ===== Logging ===== Setup logging for HAProxy. * Create a Rsyslog drop in file for HA-Proxy (/etc/rsyslog.d/haproxy.conf)<code bash>## HA-Proxy Rsyslog Config ## # Load UDP Modules $ModLoad imudp # Run UDP server $UDPServerRun 514 # Allow only localhost $AllowedSender UDP, 127.0.0.1 # Send local2 haproxy logs to /var/log/haproxy.log local2.none /var/log/messages local2.* /var/log/haproxy.log</code> * Restart rsyslog<code bash>systemctl restart rsyslog</code> ---- ====== Operate ====== Operating the load balancers. ---- ===== Services ===== Start and enable the services on each node. * HA-Proxy<code bash>systemctl start haproxy systemctl enable haproxy</code> * Keepalived<code bash>systemctl start keepalived systemctl enable keepalived</code> ---- ===== Reboots ===== Reboot procedure and dependencies. * Load Balancers (lb01, lb02) can be rebooted 1 at a time to avoid service interruption. * Determine the **inactive system** (the system that does NOT have the virtual IP as a secondary address<code bash>ip addr sh</code> * Reboot the **inactive system**<code bash>reboot</code> * Once the inactive system is up, verify keepalived and haproxy are running<code bash>systemctl status keepalived haproxy</code> * Stop keepalived on the active system in order to force a fail over<code bash>systemctl stop keepalived</code> * Verify connections to the frontend listeners go away<code bash>netstat -anpt | grep haproxy | grep -v 9000</code> * Reboot the system with keepalived stopped and no more client connections<code bash>reboot</code> ---- linux_wiki/load_balancing_haproxy_and_keepalived.txt Last modified: 2019/05/25 23:50(external edit)