MySQL復(fù)制能夠保證數(shù)據(jù)的冗余的同時可以做讀寫分離來分擔系統(tǒng)壓力,如果是主主復(fù)制還可以很好的避免主節(jié)點的單點故障。但是MySQL主主復(fù)制存在一些問題無法滿足我們的實際需要:未提供統(tǒng)一訪問入口來實現(xiàn)負載均衡,如果其中master宕掉的話需要手動切換到另外一個master,而不能自動進行切換。
這篇文章下面要介紹如何通過LVS+Keepalived的方式來是實現(xiàn)MySQL的高可用性,同時解決以上問題。
Keepalived和LVS介紹
Keepalived是一個基于VRRP(虛擬路由冗余協(xié)議)可用來實現(xiàn)服務(wù)高可用性的軟件方案,避免出現(xiàn)單點故障。Keepalived一般用來實現(xiàn)輕量級高可用性,且不需要共享存儲,一般用于兩個節(jié)點之間,常見有LVS+Keepalived、Nginx+Keepalived組合。
LVS(Linux Virtual Server)是一個高可用性虛擬的服務(wù)器集群系統(tǒng)。本項目在1998年5月由章文嵩博士成立,是中國國內(nèi)最早出現(xiàn)的自由軟件項目之一。 LVS主要用于多服務(wù)器的負載均衡,作用于網(wǎng)絡(luò)層。LVS構(gòu)建的服務(wù)器集群系統(tǒng)中,前端的負載均衡層被稱為Director Server;后端提供服務(wù)的服務(wù)器組層被稱為Real Server。通過下圖可以大致了解LVS的基礎(chǔ)架構(gòu)。 abc
LVS有三種工作模式,分別是DR(Direct Routing 直接路由)、TUN(Tunneling IP隧道)、NAT(Network Address Translation 網(wǎng)絡(luò)地址轉(zhuǎn)換)。其中TUN模式能夠支持更多的Real Server,但需要所有服務(wù)器支持IP隧道協(xié)議;DR也可以支持相當?shù)腞eal Server,但需要保證Director Server虛擬網(wǎng)卡與物理網(wǎng)卡在同一網(wǎng)段;NAT擴展性有限,無法支持更多的Real Server,因為所有的請求包和應(yīng)答包都需要Director Server進行解析再生,影響效率。 同時,LVS負載均衡有10中調(diào)度算法,分別是rr、wrr、lc、wlc、lblc、lblcr、dh、sh、sed、nq
詳細的LVS說明請參見 傳送門
本文中將利用LVS實現(xiàn)MySQL的讀寫負載均衡,Keepalived避免節(jié)點出現(xiàn)單點故障。
LVS+Keepalived配置
環(huán)境準備LVS1:192.168.1.2
LVS2:192.168.1.11
MySQL Server1:192.168.1.5
MySQL Server2:192.168.1.6
VIP:192.168.1.100
OS: CentOS 6.4
架構(gòu)圖
Keepalive安裝
keepalived下載地址
需要安裝以下軟件包
# yum install -y kernel-devel openssl openssl-devel
解壓keepalived到/usr/local/并進入目錄執(zhí)行配置編譯
# ./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-431.5.1.el6.x86_64/
Keepalived configuration
------------------------
Keepalived version : 1.2.13
Compiler : gcc
Compiler flags : -g -O2
Extra Lib : -lssl -lcrypto -lcrypt
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
IPVS use libnl : No
fwmark socket support : Yes
Use VRRP Framework : Yes
Use VRRP VMAC : Yes
SNMP support : No
SHA1 support : No
Use Debug flags : No
# make && make install
默認情況下keepalived啟動時會去/etc/keepalived目錄下找配置文件,將需要的配置文件拷貝到指定位置
# cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
# chkconfig mysqld on
# chkconfig keepalived on
LVS安裝
ipvsadm下載地址
需要安裝以下軟件包
# yum install -y libnl* popt*
查看是否加載lvs模塊
# modprobe -l |grep ipvs
解壓安裝
# ln -s /usr/src/kernels/2.6.32-431.5.1.el6.x86_64/ /usr/src/linux
# tar -zxvf ipvsadm-1.26.tar.gz
# make && make install
LVS安裝完成,查看當前LVS集群
# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
LVS+Keepalived配置搭建MySQL主主復(fù)制
這里不再贅述,請參考MySQL復(fù)制
配置Keepalived
下面是LVS1節(jié)點(Keepalived主節(jié)點)上的Keepalived配置,LVS2類似
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id LVS1
}
vrrp_instance VI_1 {
state MASTER #指定instance初始狀態(tài),實際根據(jù)優(yōu)先級決定.backup節(jié)點不一樣
interface eth0 #虛擬IP所在網(wǎng)
virtual_router_id 51 #VRID,相同VRID為一個組,決定多播MAC地址
priority 100 #優(yōu)先級,另一臺改為90.backup節(jié)點不一樣
advert_int 1 #檢查間隔
authentication {
auth_type PASS #認證方式,可以是pass或ha
auth_pass 1111 #認證密碼
}
virtual_ipaddress {
192.168.1.100 #VIP
}
}
virtual_server 192.168.1.100 3306 {
delay_loop 6 #服務(wù)輪詢的時間間隔
lb_algo wrr #加權(quán)輪詢調(diào)度,LVS調(diào)度算法 rr|wrr|lc|wlc|lblc|sh|sh
lb_kind DR #LVS集群模式 NAT|DR|TUN,其中DR模式要求負載均衡器網(wǎng)卡必須有一塊與物理網(wǎng)卡在同一個網(wǎng)段
#nat_mask 255.255.255.0
persistence_timeout 50 #會話保持時間
protocol TCP #健康檢查協(xié)議
## Real Server設(shè)置,3306就是MySQL連接端口
real_server 192.168.1.5 3306 {
weight 3 ##權(quán)重
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 192.168.1.6 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}
配置LVS
編寫LVS啟動腳本/etc/init.d/realserver
#!/bin/sh
VIP=192.168.1.100
. /etc/rc.d/init.d/functions
case "$1" in
# 禁用本地的ARP請求、綁定本地回環(huán)地址
start)
/sbin/ifconfig lo down
/sbin/ifconfig lo up
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
/sbin/sysctl -p >/dev/null 2>&1
/sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up #在回環(huán)地址上綁定VIP,設(shè)定掩碼,與Direct Server(自身)上的IP保持通信
/sbin/route add -host $VIP dev lo:0
echo "LVS-DR real server starts successfully.\n"
;;
stop)
/sbin/ifconfig lo:0 down
/sbin/route del $VIP >/dev/null 2>&1
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "LVS-DR real server stopped.\n"
;;
status)
isLoOn=`/sbin/ifconfig lo:0 | grep "$VIP"`
isRoOn=`/bin/netstat -rn | grep "$VIP"`
if [ "$isLoON" == "" -a "$isRoOn" == "" ]; then
echo "LVS-DR real server has run yet."
else
echo "LVS-DR real server is running."
fi
exit 3
;;
*)
echo "Usage: $0 {start|stop|status}"
exit 1
esac
exit 0
將lvs腳本加入開機自啟動
# chmod +x /etc/init.d/realserver
# echo "/etc/init.d/realserver" >> /etc/rc.d/rc.local
分別啟動LVS和keepalived
# service realserver start
# service keepalived start
注意此時網(wǎng)卡的變化,可以看到虛擬網(wǎng)卡已經(jīng)分配到了realserver上。
此時查看LVS集群狀態(tài),可以看到集群下有兩個Real Server,調(diào)度算法,權(quán)重等信息。ActiveConn代表當前Real Server的活躍連接數(shù)
# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.100:3306 wrr persistent 50
-> 192.168.1.5:3306 Route 3 4 1
-> 192.168.1.6:3306 Route 3 0 2
此時LVS+Keepalived+MySQL主主復(fù)制已經(jīng)搭建完成。
測試驗證功能性驗證
關(guān)閉MySQL Server2
# service mysqld stop
在LVS1查看/var/log/messages中關(guān)于keepalived日志,LVS1檢測到了MySQL Server2宕機,同時LVS集群自動剔除了故障節(jié)點
Sep 9 13:50:53 192.168.1.2 Keepalived_healthcheckers[18797]: TCP connection to [192.168.1.6]:3306 failed !!!
Sep 9 13:50:53 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.6]:3306 from VS [192.168.1.100]:3306
從新啟動MySQL Server2后自動將故障節(jié)點自動加入LVS集群
Sep 9 13:51:41 192.168.1.2 Keepalived_healthcheckers[18797]: TCP connection to [192.168.1.6]:3306 success.
Sep 9 13:51:41 192.168.1.2 Keepalived_healthcheckers[18797]: Adding service [192.168.1.6]:3306 to VS [192.168.1.100]:3306
關(guān)閉LVS1上的Keepalived(模擬宕機操作),查看LVS1上的日志,可以看到Keepalived移出了LVS1上的VIP
Sep 9 14:01:27 192.168.1.2 Keepalived[18796]: Stopping Keepalived v1.2.13 (09/09,2014)
Sep 9 14:01:27 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.5]:3306 from VS [192.168.1.100]:3306
Sep 9 14:01:27 192.168.1.2 Keepalived_healthcheckers[18797]: Removing service [192.168.1.6]:3306 from VS [192.168.1.100]:3306
Sep 9 14:01:27 192.168.1.2 Keepalived_vrrp[18799]: VRRP_Instance(VI_1) sending 0 priority
Sep 9 14:01:27 192.168.1.2 Keepalived_vrrp[18799]: VRRP_Instance(VI_1) removing protocol VIPs.
同時查看LVS2上日志,可以看到LVS2成為了Master,并接管了VIP
Sep 9 14:11:24 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Transition to MASTER STATE
Sep 9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Entering MASTER STATE
Sep 9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) setting protocol VIPs.
Sep 9 14:11:25 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.100
Sep 9 14:11:25 192.168.1.11 Keepalived_healthcheckers[7456]: Netlink reflector reports IP 192.168.1.100 added
Sep 9 14:11:25 192.168.1.11 avahi-daemon[1407]: Registering new address record for 192.168.1.100 on eth0.IPv4.
Sep 9 14:11:30 192.168.1.11 Keepalived_vrrp[7457]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.100
在LVS2上查看LVS集群狀態(tài),一切正常。
# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.1.100:3306 wrr persistent 50
-> 192.168.1.5:3306 Route 3 2 0
-> 192.168.1.6:3306 Route 3 1 0
總結(jié)
- MySQL主主復(fù)制是集群的基礎(chǔ),組成Server Array,其中每個節(jié)點作為Real Server。
- LVS服務(wù)器提供了負載均衡的作用,將用戶請求分發(fā)到Real Server,一臺Real Server故障并不會影響整個集群的。
- Keepalived搭建主備LVS服務(wù)器,避免了LVS服務(wù)器的單點故障,出現(xiàn)故障時可以自動切換到正常的節(jié)點。
|