本文是我給公司內(nèi)部寫(xiě)的一個(gè)簡(jiǎn)單的配置文檔，文中只有配置步驟，省掉了原理說(shuō)明部分。

polygun2000原創(chuàng)，轉(zhuǎn)載請(qǐng)注明: 來(lái)源于polygun2000博客 http://blog.sina.com.cn/polygun2000

一、功能需求

1.四層負(fù)載均衡(TCP)和七層負(fù)載均衡(HTTP)

2.會(huì)話保持

3.IP地址透?jìng)?/span>

二、系統(tǒng)結(jié)構(gòu)

三、系統(tǒng)組件介紹

haproxy:  http://haproxy.

1.基于 TCP 和 HTTP 協(xié)議的高效能負(fù)載均衡器(不同于nginx,haproxy本身不具有web server功能)。

2.基于GPL協(xié)議,開(kāi)源軟件。

3.高效,穩(wěn)定,安全性高,適合重負(fù)載使用,支持10GE網(wǎng)卡。

4.負(fù)載均衡算法靈活: 輪詢，靜態(tài)輪詢,最小連接數(shù)，源地址hash，基于url等。

5.支持透明代理,限速等高級(jí)功能。

tproxy: http://www./support/community/products/tproxy

1.支持透明代理的內(nèi)核補(bǔ)丁,自2.6.28以后已經(jīng)進(jìn)入主線內(nèi)核。

2.結(jié)合haproxy可以使用戶IP地址透?jìng)鹘o后端服務(wù)器。

keepalived: http://www.

1.用來(lái)防止路由器出現(xiàn)單點(diǎn)故障的熱備份軟件，最早用于與LVS結(jié)合。

2.使用VRRP協(xié)議。

四、配置過(guò)程簡(jiǎn)述

五、具體配置步驟

1.環(huán)境準(zhǔn)備

硬件選擇: E5-2600CPU+Intel服務(wù)器網(wǎng)卡

操作系統(tǒng): 最小化安裝CentOS 6.3 x86_64

a.關(guān)閉網(wǎng)卡中斷調(diào)節(jié)
[root@ modprobe.d]# vi /etc/modprobe.d/intel-nic.conf
options igb InterruptThrottleRate=0,0,0,0
或者
options ixgbe InterruptThrottleRate=0,0

b.設(shè)置網(wǎng)卡中斷CPU親和

set_irq_affinity.sh腳本包含在Intel官方的ixgbe驅(qū)動(dòng)中，下載地址:
https://downloadcenter.intel.com/download/14687/Network-Adapter-Driver-for-PCI-E-10-Gigabit-Network-Connections-under-Linux-

安裝163,epel源

[root@haproxy ~]#yum install wget

[root@haproxy ~]#wget http://mirrors.163.com/.help/CentOS6-Base-163.repo

[root@haproxy ~]#wget http://dl./pub/epel/6/i386/epel-release-6-8.noarch.rpm

[root@haproxy ~]#mv CentOS6-Base-163.repo /etc/yum.repos.d/CentOS-Base.repo

[root@haproxy ~]#rpm -ivhepel-release-6-8.noarch.rpm

[root@haproxy ~]#yum update

2.編譯安裝pcre

[root@haproxy ~]#yum install gcc gcc-c++ make zlib-devel bzip2-devel

[root@haproxy ~]#wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.32.tar.bz2

[root@haproxy ~]#tar xvjf pcre-8.32.tar.bz2

[root@haproxy ~]#./configure --prefix=/usr \

--docdir=/usr/share/doc/pcre-8.32 \

--enable-utf --enable-unicode-properties \

--enable-pcregrep-libz --enable-pcregrep-libbz2

[root@haproxy ~]#make

[root@haproxy ~]#make check

[root@haproxy ~]#make install

3.編譯安裝haproxy

[root@haproxy ~]#yum install openssl-devel

[root@haproxy ~]#wget http://haproxy./download/1.5/src/devel/haproxy-1.5-dev17.tar.gz

[root@haproxy ~]#tar xvzfhaproxy-1.5-dev17.tar.gz

[root@haproxy ~]#cd haproxy-1.5-dev17

[root@haproxy ~]#make TARGET=linux26 USE_STATIC_PCRE=1 \

USE_REGPARM=1 USE_LINUX_TPROXY=1 USE_OPENSSL=1 USE_ZLIB=1 ARCH=x86_64

[root@haproxy ~]#make install

4.創(chuàng)建haproxy啟動(dòng)腳本

來(lái)源: http:///2010/11/04/a-custom-init-d-start-up-script-for-haproxy-start-stop-restart-reload-checkconfig/

直接下載連接: http:///downloads/haproxy/haproxy.init

[root@haproxy ~]#vi /etc/init.d/haproxy

#----------------------------

#!/bin/sh

#

# custom haproxy init.d script, by Mattias Geniar

#

# haproxy        starting and stopping the haproxy load balancer

#

# chkconfig: 345 55 45

# description: haproxy is a TCP loadbalancer

# probe: true

# Source function library.

. /etc/rc.d/init.d/functions

# Source networking configuration.

. /etc/sysconfig/network

# Check that networking is up.

[ ${NETWORKING} = "no" ] && exit 0

[ -f /usr/local/sbin/haproxy ] || exit 0

[ -f /etc/haproxy/haproxy.conf ] || exit 0

# Define our actions

checkconfig() {

       # Check the config file for errors

       /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf

       if [ $? -ne 0 ]; then

                echo "Errors found in configuration file."

               return 1

       fi

       # We're OK!

       return 0

}

start() {

       # Check config

       /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf

       if [ $? -ne 0 ]; then

               echo "Errors found in configuration file."

               return 1

       fi

       echo -n "Starting HAProxy: "

       daemon /usr/local/sbin/haproxy -D -f /etc/haproxy/haproxy.conf -p /var/run/haproxy.pid

       RETVAL=$?

       echo

       [ $RETVAL -eq 0 ] && touch /var/lock/subsys/haproxy

       return $RETVAL

}

stop() {

       echo -n "Shutting down HAProxy: "

       killproc haproxy -USR1

       RETVAL=$?

       echo

       [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/haproxy

       [ $RETVAL -eq 0 ] && rm -f /var/run/haproxy.pid

       return $RETVAL

}

restart() {

       /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf

       if [ $? -ne 0 ]; then

               echo "Errors found in configuration file."

               return 1

       fi

       stop

       start

}

check() {

       /usr/local/sbin/haproxy -c -q -V -f /etc/haproxy/haproxy.conf

}

rhstatus() {

       status haproxy

}

reload() {

       /usr/local/sbin/haproxy -c -q -f /etc/haproxy/haproxy.conf

       if [ $? -ne 0 ]; then

               echo "Errors found in configuration file."

               return 1

       fi

       echo -n "Reloading HAProxy config: "

       /usr/local/sbin/haproxy -f /etc/haproxy/haproxy.conf -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

       success $"Reloading HAProxy config: "

       echo

}

# Possible parameters

case "$1" in

 start)

       start

       ;;

 stop)

       stop

       ;;

 status)

       rhstatus

       ;;

 restart)

       restart

       ;;

 reload)

       reload

       ;;

 checkconfig)

       check

       ;;

 *)

       echo "Usage: haproxy {start|stop|status|restart|reload|checkconfig}"

       exit 1

esac

exit 0

#----------------------------

[root@haproxy ~]#chmod +x /etc/init.d/haproxy

設(shè)置開(kāi)機(jī)啟動(dòng)haproxy服務(wù)

[root@haproxy ~]#chkconfig --add haproxy

[root@haproxy ~]#chkconfig haproxy on

5.配置haproxy

創(chuàng)建chroot目錄,確保該目錄為空,且其賬號(hào)不可訪問(wèn)。

[root@haproxy ~]#mkdir /var/haproxy

[root@haproxy ~]#chmod o= /var/haproxy

創(chuàng)建haproxy配置文件

[root@haproxy ~]#mkdir /etc/haproxy

[root@haproxy ~]#vi /etc/haproxy/haproxy.conf

global段配置

#全局配置

global

   maxconn 32768 # Max simultaneous connections from an upstream server

   spread-checks 5 # Distribute health checks with some randomness

   chroot /var/haproxy

   daemon

   log 127.0.0.1 local0

   log 127.0.0.1 local1 notice

   #debug # Uncomment for verbose logging

defaults段配置

#默認(rèn)配置,應(yīng)用于所有下邊的服務(wù)

defaults

   log global

   mode http

   balance roundrobin

   retries 3

   option abortonclose # abort request if client closes output channel while waiting

   option httpclose # add "Connection:close" header if it is missing

   option forwardfor # insert x-forwarded-for header so that app servers can see both proxy and client IPs

   option redispatch # any server can handle any session

   option httplog

   option dontlognull

   timeout http-request 5s   #aginst Slowloris attack

   timeout client 60s

   timeout connect 9s

   timeout server 30s

   timeout check 5s

   stats enable

   errorfile 503 /etc/haproxy/errors/503.http

stat監(jiān)控配置

#配置haproxy的狀態(tài)監(jiān)控

listen stats

   bind 192.168.10.132:8888

   stats uri /

   stats realm Haproxy\ Statistics

   stats auth hadmin:yhXV2WAbybXd1euzEXbe

   stats refresh 20

log配置

1.配置rsyslog以接收haproxy日志

[root@haproxy ~]#vi /etc/rsyslog.d/haproxy.conf

# Custom log facilities for haproxy

local0.* -/var/log/haproxy0a.log

local1.* -/var/log/haproxy1a.log

$ModLoad imudp

# load the imudp module for rsyslog

# provides UDP syslog reception

# start UDP server on this port, "*" means all addresses

$UDPServerRun 514

# local IP address (or name) the UDP listens should bind to

$UDPServerAddress 127.0.0.1

[root@haproxy ~]#/etc/init.d/rsyslog restart

注釋:

/var/log/haproxy0a.log前邊的"-"減號(hào)意味著取消日志同步寫(xiě)入。

這可以優(yōu)化一下磁盤寫(xiě)入,尤其是在非常繁忙的系統(tǒng)中。

不過(guò)如果突然斷電,可能會(huì)損失一些未寫(xiě)入硬盤的日志。

2.配置logrotate

[root@haproxy ~]#vi /etc/logrotate.d/haproxy

/var/log/haproxy*.log

{

   daily

    rotate 4

   missingok

   notifempty

   compress

   delaycompress

   sharedscripts

   postrotate

       /etc/init.d/haproxy reload >/dev/null

   endscript

}

注釋:

如果站點(diǎn)數(shù)量較多,可能會(huì)希望將不同站點(diǎn)的日志分開(kāi),可以看看后邊的"參考文檔E"。

http應(yīng)用配置

listen VIP_64.4.2.111

  bind 64.4.2.111:80

  cookie SERVERID insert indirect nocache

  server s31 192.168.10.31:80 check cookie s1

  server s32 192.168.10.32:80 check cookie s2

tcp應(yīng)用配置

listen VIP_64.4.2.118

  bind 64.4.2.118:22186

  mode tcp

  option tcplog

  server s41 192.168.10.41:22186 check

  server s42 192.168.10.42:22186 check

會(huì)話保持配置

#需要做會(huì)話保持的tcp配置,采用源地址hash

listen VIP_64.4.2.109

   bind 64.4.2.109:1235

   balance source

   option tcplog

   hash-type consistent # optional

   server s11 192.168.10.11:1235 check

   server s12 192.168.10.12:1235 check

 #需要做會(huì)話保持的http配置

listen VIP_64.4.2.111

  bind 64.4.2.111:80

  cookie SERVERID insert indirect nocache

  server s31 192.168.10.31:80 check cookie s1

  server s32 192.168.10.32:80 check cookie s2

源地址透?jìng)髋渲?/span>

#需要查看用戶真實(shí)IP的配置

listen VIP_64.4.2.118

  bind 64.4.2.118:22186

  mode tcp

  option tcplog

  source 0.0.0.0 usesrc clientip

  server s41 192.168.10.41:22186 check

  server s42 192.168.10.42:22186 check

為TPROXY設(shè)置iptables規(guī)則

[root@haproxy ~]#/sbin/iptables -t mangle -N DIVERT

[root@haproxy ~]#/sbin/iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT

[root@haproxy ~]#/sbin/iptables -t mangle -A DIVERT -j MARK --set-mark 1

[root@haproxy ~]#/sbin/iptables -t mangle -A DIVERT -j ACCEPT

[root@haproxy ~]#/sbin/ip rule add fwmark 1 lookup 100

[root@haproxy ~]#/sbin/ip route add local 0.0.0.0/0 dev lo table 100

 #給tproxy后端做NAT

[root@haproxy ~]#/sbin/iptables -t nat -A POSTROUTING -s backend's_ip -o eht0 -j MASQUERADE

在后端服務(wù)器上設(shè)置haproxy為默認(rèn)網(wǎng)關(guān)

[root@backend ~]# ip route add default via haproxy_lanip

5.相關(guān)內(nèi)核參數(shù)調(diào)整

[root@haproxy ~]# vi /etc/sysctl.conf

 #允許ip轉(zhuǎn)發(fā)

net.ipv4.ip_forward = 1

#設(shè)置松散逆向路徑過(guò)濾

net.ipv4.conf.default.rp_filter = 2

net.ipv4.conf.all.rp_filter = 2

net.ipv4.conf.eth0.rp_filter = 0

#允許ICMP重定向

net.ipv4.conf.all.send_redirects = 1

net.ipv4.conf.default.send_redirects = 1

#發(fā)送到一個(gè)監(jiān)聽(tīng)的socket上的最大已完成連接隊(duì)列長(zhǎng)度

#三次握手已經(jīng)完成,但還未被應(yīng)用層接收(accept),但也處于ESTABLISHED狀態(tài)

#隊(duì)列長(zhǎng)度由listen的backlog參數(shù)和內(nèi)核的 net.core.somaxconn 參數(shù)共同決定

#當(dāng)這個(gè)隊(duì)列滿了之后,不管未完成連接隊(duì)列是否已滿,是否啟用syncookie,都不在接收新的SYN請(qǐng)求.

net.core.somaxconn = 32768

#允許綁定到非本地地址,用于keepalived

net.ipv4.ip_nonlocal_bind = 1

#增加可用的端口范圍

net.ipv4.ip_local_port_range = 1024 65023

#防攻擊使用,如無(wú)必要一定要設(shè)置成0

net.ipv4.tcp_abort_on_overflow = 0

#如果套接字由本端要求關(guān)閉,這個(gè)參數(shù)決定了它保持在FIN-WAIT-2狀態(tài)的時(shí)間,缺省值是60秒。

#減小這個(gè)值,可以使TCP/IP更快的釋放連接,騰出更多資源給新連接。推薦15-30秒。

net.ipv4.tcp_fin_timeout = 10

#最后一個(gè)數(shù)據(jù)包發(fā)送完成和第一個(gè)keepalive包被檢測(cè)到之間的時(shí)間間隔

#表示當(dāng)keepalive起用的時(shí)候,TCP發(fā)送keepalive消息的頻度,缺省是2小時(shí)。

net.ipv4.tcp_keepalive_time = 300

#系統(tǒng)所能處理不屬于任何進(jìn)程的TCP sockets最大數(shù)量。

#假如超過(guò)這個(gè)數(shù)量,那么不屬于任何進(jìn)程的連接會(huì)被立即reset,并同時(shí)顯示警告信息。

net.ipv4.tcp_max_orphans = 262144

#backlog隊(duì)列是一個(gè)大的內(nèi)存結(jié)構(gòu),用來(lái)處理收到的帶有SYN標(biāo)記的數(shù)據(jù)包,直到三次握手完成。

#這個(gè)參數(shù)控制了同一時(shí)間內(nèi)操作系統(tǒng)可以處理多少個(gè)半開(kāi)連接,當(dāng)連接數(shù)達(dá)到這個(gè)數(shù)值的設(shè)定后,系統(tǒng)會(huì)丟棄隨后的請(qǐng)求。

net.ipv4.tcp_max_syn_backlog = 16384

#表示系統(tǒng)同時(shí)保持TIME_WAIT套接字的最大數(shù)量,如果超過(guò)這個(gè)數(shù)字,TIME_WAIT套接字將立刻被清除并打印警告信息。

net.ipv4.tcp_max_tw_buckets = 262144

#對(duì)于遠(yuǎn)端的連接請(qǐng)求SYN,內(nèi)核會(huì)發(fā)送SYN ＋ ACK數(shù)據(jù)報(bào),以確認(rèn)收到上一個(gè) SYN連接請(qǐng)求包。

#這是所謂的三次握手( threeway handshake)機(jī)制的第二個(gè)步驟。這里決定內(nèi)核在放棄連接之前所送出的 #SYN+ACK數(shù)目。如果你的網(wǎng)站SYN_RECV狀態(tài)確實(shí)挺多,為了避免syn攻擊,那么可以調(diào)節(jié)重發(fā)的次數(shù)。

net.ipv4.tcp_synack_retries = 3

#開(kāi)啟/關(guān)閉SYN Cookies

#當(dāng)啟動(dòng)SYN Cookie時(shí),主機(jī)在發(fā)送 SYN/ACK 確認(rèn)封包前,會(huì)要求 Client 端在短時(shí)間內(nèi)回復(fù)一個(gè)序號(hào)

#這個(gè)序號(hào)包含許多原本 SYN 封包內(nèi)的信息,包括 IP、port 等。

#若 Client 端可以回復(fù)正確的序號(hào),那么主機(jī)就確定該封包為可信的,因此會(huì)發(fā)送 SYN/ACK 封包,否則就不理會(huì)此一封包。

#這個(gè)參數(shù)不會(huì)提高性能,而且違背TCP協(xié)議,如果不是遭到SYN Flood攻擊,不要打開(kāi)。

net.ipv4.tcp_syncookies = 0

#根據(jù)RFC1323,會(huì)向TCP包頭中插入12byte,2.6內(nèi)核的Linux默認(rèn)是打開(kāi)的,某些情況下timestamp數(shù)值有可能溢出造成TCP超時(shí)

#建議關(guān)閉。

net.ipv4.tcp_timestamps = 0

#開(kāi)啟TCP連接中TIME-WAIT sockets的快速回收

net.ipv4.tcp_tw_recycle = 1

#開(kāi)啟重用,允許將TIME-WAIT sockets重新用于新的TCP連接

net.ipv4.tcp_tw_reuse = 1

#如果TCP窗口大小超過(guò)65536,需要此選項(xiàng)打開(kāi)大TCP窗口支持。

net.ipv4.tcp_window_scaling=1

#決定TCP協(xié)議棧如何使用內(nèi)存,單位是內(nèi)存分頁(yè),而不是字節(jié)。每個(gè)內(nèi)存分頁(yè)一般為4K。

#當(dāng)超過(guò)第二個(gè)值時(shí),TCP進(jìn)入pressure模式,此時(shí)TCP嘗試穩(wěn)定其內(nèi)存的使用,

#當(dāng)小于第一個(gè)值時(shí),就退出pressure模式,TCP不會(huì)考慮釋放內(nèi)存。

#當(dāng)內(nèi)存占用超過(guò)第三個(gè)值時(shí),TCP就拒絕分配socket了,查看dmesg,會(huì)打出很多的日志“TCP: too many of orphaned sockets”。

#如果不是非常必要,一般不要?jiǎng)酉到y(tǒng)默認(rèn)的值,默認(rèn)值一般來(lái)說(shuō)夠用了

net.ipv4.tcp_mem = "786432 2097152 3145728"

#TCP流中重排序的數(shù)據(jù)包最大數(shù)量

net.ipv4.tcp_reordering = 3

#系統(tǒng)auto-tuning時(shí),每個(gè)socket使用的內(nèi)存。分別是最小,缺省,最大TCP接收窗口的內(nèi)存大小,單位byte

#如果設(shè)置net.core.rmem_default,則該值會(huì)覆蓋缺省值

#如果設(shè)置net.core.rmem_max,則該值會(huì)覆蓋最大值

net.ipv4.tcp_rmem = "4096 87380 16777216"

6.keepalived配置

安裝keepalived

[root@haproxy ~]#yum install keepalived

配置keepalived

[root@haproxy ~]# vi /etc/keepalived/keepalived.conf

   ! Configuration File for keepalived 

   global_defs {  # global_defs全局配置標(biāo)識(shí),說(shuō)明這個(gè)區(qū)域{}是全局配置

      notification_email {   # 發(fā)送email通知,以及email發(fā)送給哪些郵件地址,郵件地址可以多個(gè),每行一個(gè)。

           admin@demo.com 

      } 

      notification_email_from admin@demo.com  # 發(fā)送通知郵件時(shí)郵件源地址是誰(shuí)

      smtp_connect_timeout 3  # smtp連接超時(shí)時(shí)間

      smtp_server 127.0.0.1   # 發(fā)送email時(shí)使用的smtp服務(wù)器地址

      router_id haproxy_101  # 機(jī)器標(biāo)識(shí),從節(jié)點(diǎn)為haproxy_102

   } 

   vrrp_script chk_haproxy {  # 定義腳本名字

       script "killall -0 haproxy" 

       interval 2   # 腳本執(zhí)行間隔2s

       weight 10  # 腳本結(jié)果導(dǎo)致的優(yōu)先級(jí)變更：10表示優(yōu)先級(jí)+10；-10則表示優(yōu)先級(jí)-10

          fall 2   # require 2 failures for KO
          rise 2  # require 2 successes for OK

   } 

   vrrp_instance VI_1 {  # vrrp實(shí)例名稱

       interface eth1  # 實(shí)例綁定的網(wǎng)卡,因?yàn)樵谂渲锰摂MIP的時(shí)候必須是在已有的網(wǎng)卡上添加的

       state MASTER # 從節(jié)點(diǎn)則此此處為BACKUP ,需要大寫(xiě)這些單詞

       priority 101 # 設(shè)置本節(jié)點(diǎn)的優(yōu)先級(jí),數(shù)值愈大,優(yōu)先級(jí)越高,優(yōu)先級(jí)高的為master 

       virtual_router_id 50 # 主、備機(jī)的virtual_router_id必須相同?。?span style="color: green;"> 

       garp_master_delay 1 # 主從切換時(shí)間,單位為秒。 

       authentication {  # 設(shè)置認(rèn)證,同一vrrp實(shí)例MASTER與BACKUP 使用相同的密碼才能正常通信。

           auth_type PASS  # 認(rèn)證方式,可以是PASS或AH兩種認(rèn)證方式

           auth_pass U5vXgwcveTuDt66MxJa7  # 認(rèn)證密碼

       } 

       virtual_ipaddress {  # 這里設(shè)置的就是VIP，也就是用工作的虛擬IP地址,VIP最多20個(gè)

          64.4.2.110/24 dev eth0

       }

       virtual_ipaddress_excluded {  # 超過(guò)20個(gè)VIP可以添加在virtual_ipaddress_excluded中,這些VIP不需要發(fā)送檢測(cè)包

          64.4.2.111/24 dev eth0

          64.4.2.112/24 dev eth0

          202.113.58.7/24 dev eth1

       }

       track_interface {  # 跟蹤接口,設(shè)置額外的監(jiān)控,里面任意一塊網(wǎng)卡出現(xiàn)問(wèn)題,都會(huì)進(jìn)入故障(FAULT)狀態(tài)

          eth0 

          eth1 

       } 

       track_script {  # 引用vrrp_script,有點(diǎn)類似腳本里面的函數(shù)引用一樣,先定義,后引用函數(shù)名

           chk_haproxy  # 調(diào)用腳本必須放在virtual_ipaddress之后

       } 

       #狀態(tài)通知 

       notify_master /etc/keepalived/scripts/be_master.sh   # 當(dāng)進(jìn)入Master狀態(tài)時(shí)會(huì)呼叫notify_master

       notify_backup /etc/keepalived/scripts/be_backup.sh  # 當(dāng)進(jìn)入Backup狀態(tài)時(shí)會(huì)呼叫notify_backup

       notify_fault /etc/keepalived/scripts/be_fault.sh   # 當(dāng)發(fā)現(xiàn)異常情況時(shí)進(jìn)入Fault狀態(tài)呼叫notify_fault

       notify_stop  /etc/keepalived/scripts/be_stop.sh    # 當(dāng)Keepalived程序終止時(shí)則呼叫notify_stop

   } 

確認(rèn)keepalived工作正常

[root@haproxy ~]# tcpdump -v -i eth0 host 224.0.0.18

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
16:54:01.743275 IP (tos 0x0, ttl 255, id 451, offset 0, flags [none], proto: VRRP (112), length: 96) 10.10.28.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 103, authtype simple, intvl 1s, length 76, addrs(15): 123.12.15.2,123.12.15.3[|vrrp]
16:54:02.744241 IP (tos 0x0, ttl 255, id 452, offset 0, flags [none], proto: VRRP (112), length: 96) 10.10.28.5 > 224.0.0.18: VRRPv2, Advertisement, vrid 51, prio 103, authtype simple, intvl 1s, length 76, addrs(15): 123.12.15.2,123.12.15.3[|vrrp]

六、進(jìn)階應(yīng)用

1.限制單個(gè)IP的并發(fā)連接數(shù)

frontend ft_web

   bind 0.0.0.0:8080

   # Table definition 

   stick-table type ip size 100k expire 30s store conn_cur

   # Allow clean known IPs to bypass the filter

   tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }

   # Shut the new connection as long as the client has already 10 opened

   tcp-request connection reject if { src_conn_cur ge 10 }

   tcp-request connection track-sc1 src

2.限制單個(gè)IP建立連接的頻率

frontend ft_web

   bind 0.0.0.0:8080

   # Table definition 

   stick-table type ip size 100k expire 30s store conn_rate(3s)

   # Allow clean known IPs to bypass the filter

   tcp-request connection accept if { src -f /etc/haproxy/whitelist.lst }

   # Shut the new connection as long as the client has already 10 opened

   tcp-request connection reject if { src_conn_rate ge 10 }

   tcp-request connection track-sc1 src

3.限制HTTP請(qǐng)求的的頻率

frontend ft_web

   bind 0.0.0.0:8080

   # Use General Purpose Couter (gpc) 0 in SC1 as a global abuse counter

   # Monitors the number of request sent by an IP over a period of 10 seconds

   stick-table type ip size 1m expire 10s store gpc0,http_req_rate(10s)

   tcp-request connection track-sc1 src

   tcp-request connection reject if { src_get_gpc0 gt 0 }

backend bk_web

   balance roundrobin

   cookie MYSRV insert indirect nocache

   # If the source IP sent 10 or more http request over the defined period,

   # flag the IP as abuser on the frontend

   acl abuse src_http_req_rate(ft_web) ge 10

   acl flag_abuser src_inc_gpc0(ft_web)

   tcp-request content reject if abuse flag_abuser

   server srv1 192.168.1.2:80 check cookie srv1 maxconn 100

   server srv2 192.168.1.3:80 check cookie srv2 maxconn 100

4.haproxy的監(jiān)控

hatop是一個(gè)用python語(yǔ)言編寫(xiě)的,交互式的ncurses客戶端程序。

它的輸出類似top程序,可以用來(lái)實(shí)時(shí)查看haproxy的狀態(tài),如果允許level admin則還可以enable,disable服務(wù)器。

[root@haproxy ~]# yum install socat

[root@haproxy ~]# wget http://hatop./files/hatop-0.7.7.tar.gz

[root@haproxy ~]# tar xvzf hatop-0.7.7.tar.gz

[root@haproxy ~]# cd hatop-0.7.7

[root@haproxy ~]# install -m 755 bin/hatop /usr/local/bin

[root@haproxy ~]# install -m 644 man/hatop.1 /usr/local/share/man/man1

[root@haproxy ~]# gzip /usr/local/share/man/man1/hatop.1

[root@haproxy ~]# vi /etc/haproxy/haproxy.conf

在global段內(nèi)加入如下:

stats socket /var/run/haproxy.stat mode 0600 level admin

重起haproxy

[root@haproxy ~]# /etc/init.d/haproxy reload

確認(rèn)socket已建立

[root@haproxy ~]# ls -al /var/run/haproxy.stat

srw-------. 1 root root 0 Jan 15 20:53 haproxy.sock

運(yùn)行hatop查看haproxy相關(guān)實(shí)時(shí)信息

[root@haproxy ~]# hatop -s /var/run/haproxy.stat

5.用Zabbix監(jiān)控haproxy[http://www./2010/10/15/script-and-template-to-export-data-from-haproxy-to-zabbix]

6.單網(wǎng)卡多個(gè)不同網(wǎng)段的相關(guān)配置

 [root@localhost examples]# vi /etc/iproute2/rt_tables

文件結(jié)尾追加如下內(nèi)容:

64       CNC64

202     CNC202

211     CNC211

配置多路由表

[root@haproxy ~]# vi /etc/haproxy/haproxy.conf

#!/bin/bash

######

CNC64_IP="64.4.2.0/24"

CNC64_GW="64.4.2.1"

CNC202_IP="202.108.35.0/24"

CNC202_GW="202.108.1"

CNC211_IP="211.113.58.0/24"

CNC211_GW="211.113.58.1"

ip route flush table CNC64

ip route add default via $CNC64_GW dev eth0 table CNC64

ip rule add from $CNC64_IP table CNC64

ip route flush table CNC202

ip route add default via $CNC202_GW dev eth0 table CNC202

ip rule add from $CNC202_IP table CNC202

ip route flush table CNC211

ip route add default via $CNC211_GW dev eth0 table CNC211

ip rule add from $CNC211_IP table CNC211

修改keepalived配置文件

[root@haproxy ~]# vi /etc/haproxy/haproxy.conf

virtual_ipaddress_excluded {  # 超過(guò)20個(gè)VIP可以添加在virtual_ipaddress_excluded中,這些VIP不需要發(fā)送檢測(cè)包

          64.4.2.111/24 dev eth0

          202.108.35.22/24 dev eth0

          211.113.58.7/24 dev eth0

       }

七、SSL offload配置（使用self-signed證書(shū)）

 2017.02.16 補(bǔ)充一個(gè)方便的技巧

haproxy官方提供了針對(duì)vim的語(yǔ)法文件，可以高亮顯示keyword，對(duì)于修改配置文件來(lái)說(shuō)很方便。

方法說(shuō)一下：

1.將haproxy源碼中example目錄中的haproxy.vim復(fù)制到$HOME/.vim/syntax/

2.修改$HOME/.vimrc，加入:  au BufRead,BufNewFile haproxy* set ft=haproxy

八、系統(tǒng)安全加固

[root@haproxy ~]#yum install yum-remove-with-leaves

[root@haproxy ~]#yum remove gcc make

[root@haproxy ~]#vi remove-list

system-config-firewall-base

iptables-ipv6

dhcp-common

pciutils-libs

efibootmgr

dhclient

kernel-firmware

iwl5150-firmware

iwl6050-firmware

iwl6000g2a-firmware

iwl6000-firmware

ql2400-firmware

ql2100-firmware

libertas-usb8388-firmware

ql2500-firmware

zd1211-firmware

rt61pci-firmware

ql2200-firmware

ipw2100-firmware

ipw2200-firmware

iwl5000-firmware

ivtv-firmware

xorg-x11-drv-ati-firmware

atmel-firmware

iwl4965-firmware

iwl3945-firmware

rt73usb-firmware

ql23xx-firmware

bfa-firmware

iwl100-firmware

b43-openfwwf

aic94xx-firmware

iwl1000-firmware

[root@haproxy ~]#for I in `cat remove-list `;do yum -y remove $i;done

八、參考文檔

1-http:///2010/11/04/a-custom-init-d-start-up-script-for-haproxy-start-stop-restart-reload-checkconfig/

2-http://www./haproxy/simple-sysctl-tunings-for-haproxy/

3-https://gist.github.com/4039319

4-http://www./files/linux-kernel/Documentation/networking/tproxy.txt

5-http://blog./2012/09/10/how-to-get-ssl-with-haproxy-getting-rid-of-stunnel-stud-nginx-or-pound/

6-http://www./connect/articles/apache-2-ssltls-step-step-part-2

7-http://www./2008/05/13/load-balancing-qos-with-haproxy/

8-http://h10025.www1.hp.com/ewfrf/wc/document?cc=us&lc=en&dlc=en&tmp_geoLoc=true&docname=c03561757

9-http://www./how-to-log-haproxy-messages-only-once/#more-713

10-https:///blog/2010/08/haproxy-logging

11-http:///blog/2010/08/11/haproxy-logging/

12-https://gist.github.com/1271962

13-http://www./doc/rsyslog_conf_actions.html

14-http://tehlose./2011/10/10/a-log-file-for-each-virtual-host-with-haproxy-and-rsyslog/

15-http://jit./2009/11/haproxy-routing-by-domain-name.html

16-http:///2010/01/16/virtual-hosting-with-haproxy-and-wsgi.html

17-http://blog./post/31927044856/3-ways-to-configure-haproxy-for-websockets

18-http://blog.csdn.net/dog250/article/details/7107537

19-http://www./content/monitoring-processes-kill

20-http:///technology/ha-lamp-with-keepalived-pt2/

21-http://zauc./2010/08/31/keepalived-conf之vrrp-instance部分解讀/

22-http://interu./entry/20081024/1224784798

23-http://bbs./thread-845-1-1.html

24-http:///archives/1942.html

25-http://www.intel.com/content/www/us/en/ethernet-controllers/82575-82576-82598-82599-ethernet-controllers-latency-appl-note.html

26-http://blog.csdn.net/turkeyzhou/article/details/7528182

27-http://www./files/pdf/techpaper/VMW-Tuning-Latency-Sensitive-Workloads.pdf

28-http://www.intel.com/support/cn/network/sb/cs-025829.htm

29-http://kaivanov./2015/02/keepalived-using-unicast-track-and.html

30-http://www./2013/03/setting-up-custom-tcpip-keep-alive.html

31-https:///using-ssl-certificates-with-haproxy

32-https://www./community/tutorials/how-to-create-a-ssl-certificate-on-nginx-for-centos-6

33-http://man./content/manage/vi/doc/syntax.html