|
1.節(jié)點規(guī)劃 1.1 mysql數(shù)據(jù)節(jié)點： db169 db172 db173 三個節(jié)點為xtradb cluster節(jié)點。 1.2 keepalived節(jié)點： db162 db163 虛擬ip為192.168.1.201 haproxy節(jié)點（僅為了對比lvs的性能才安裝的）：db169（部署在xtradb cluster的一個節(jié)點上） 1.3 atlas節(jié)點：和xtradb cluster節(jié)點部署在一起，也為三個節(jié)點 注意：atlas和mysql要部署在一個節(jié)點上，如果不在一個節(jié)點上則不能用lvs dr模式負(fù)載均衡 1.4 客戶端測試節(jié)點：db55 ip地址為192.168.1.* ,節(jié)點名為db+ip地址末位 2.安裝lvs及keepavlied（db162、db163上） 2.1安裝依賴包 yum -y install kernel-devel make gcc openssl-devel libnl* 下載并連接linux kernel文件，注意版本要一致（uname -a） [root@db163 ~]# ln -s /usr/src/kernels/2.6.32-358.el6.x86_64/ /usr/src/linux 安裝keepalived、lvs [root@db162 ~]# yum install ipvsadm [root@db162 ~]# yum install keepalived [root@db163 ~]# yum install ipvsadm [root@db163 ~]# yum install keepalived 2.2.配置keepavlied，注意lvs不需要單獨配置，在keepalived里配置就行了 [root@db162 ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { router_id MySQL_LB1 } vrrp_sync_group VSG { group { MySQL_Loadblancing } } vrrp_instance MySQL_Loadblancing { state MASTER interface eth0 virtual_router_id 51 priority 101 advert_int 1 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.1.201 } } virtual_server 192.168.1.201 1234 { delay_loop 6 lb_algo rr lb_kind DR # nat_mask 255.255.255.0 #persistence_timeout 50 protocol TCP real_server 192.168.1.169 1234 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 1234 } } real_server 192.168.1.172 1234 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 1234 } } real_server 192.168.1.173 1234 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 1234 } } } 備機上的keepalived配置 [root@db163 ~]# cat /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { router_id MySQL_LB2 } vrrp_sync_group VSG { group { MySQL_Loadblancing } } vrrp_instance MySQL_Loadblancing { state BACKUP interface eth0 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.1.201 } } virtual_server 192.168.1.201 1234 { delay_loop 6 lb_algo rr lb_kind DR # nat_mask 255.255.255.0 #persistence_timeout 50 protocol TCP real_server 192.168.1.169 1234 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 1234 } } real_server 192.168.1.172 1234 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 1234 } } real_server 192.168.1.173 1234 { weight 3 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 1234 } } } 3.realserver（數(shù)據(jù)節(jié)點）上的配置 分別在三個數(shù)據(jù)節(jié)點db169、db172、db173上安裝如下腳本： [root@db172 ~]# cat /etc/init.d/lvsdr.sh #!/bin/bash VIP=192.168.1.201 . /etc/rc.d/init.d/functions case '$1' in start) /sbin/ifconfig lo down /sbin/ifconfig lo up echo '1' >/proc/sys/net/ipv4/conf/lo/arp_ignore echo '2' >/proc/sys/net/ipv4/conf/lo/arp_announce echo '1' >/proc/sys/net/ipv4/conf/all/arp_ignore echo '2' >/proc/sys/net/ipv4/conf/all/arp_announce /sbin/sysctl -p >/dev/null 2>&1 /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 up /sbin/route add -host $VIP dev lo:0 echo 'LVS-DR real server starts successfully.\n' ;; stop) /sbin/ifconfig lo:0 down /sbin/route del $VIP >/dev/null 2>&1 echo '0' >/proc/sys/net/ipv4/conf/lo/arp_ignore echo '0' >/proc/sys/net/ipv4/conf/lo/arp_announce echo '0' >/proc/sys/net/ipv4/conf/all/arp_ignore echo '0' >/proc/sys/net/ipv4/conf/all/arp_announce echo 'LVS-DR real server stopped.' ;; status) isLoOn=`/sbin/ifconfig lo:0 | grep '$VIP'` isRoOn=`/bin/netstat -rn | grep '$VIP'` if [ '$isLoOn' == '' -a '$isRoOn' == '' ]; then echo 'LVS-DR real server has to run yet.' else echo 'LVS-DR real server is running.' fi exit 3 ;; *) echo 'Usage: $0 {start|stop|status}' exit 1 esac exit 0 增加x權(quán)限：chmod +x /etc/init.d/lvsdr.sh 增加開機自啟動：echo '/etc/init.d/lvsdr.sh start' >> /etc/rc.local 4.分別在三個數(shù)據(jù)節(jié)點db169、db172、db173上安裝atlas 下載atlas，并yum安裝 yum install -y Atlas-2.1.el6.x86_64.rpm 配置atlas [root@db172 ~]# cat /usr/local/mysql-proxy/conf/test.cnf [mysql-proxy] #帶#號的為非必需的配置項目 #管理接口的用戶名 admin-username = admin #管理接口的密碼 admin-password = 123456 #Atlas后端連接的MySQL主庫的IP和端口，可設(shè)置多項，用逗號分隔 proxy-backend-addresses = 192.168.1.173:3306 #Atlas后端連接的MySQL從庫的IP和端口，@后面的數(shù)字代表權(quán)重，用來作負(fù)載均衡，若省略則默認(rèn)為1，可設(shè)置多項，用逗號分隔 proxy-read-only-backend-addresses = 192.168.1.169:3306@1,192.168.1.172:3306@1 #用戶名與其對應(yīng)的加密過的MySQL密碼，密碼使用PREFIX/bin目錄下的加密程序encrypt加密，下行的user1和user2為示例，將其替換為你的MySQL的用戶名和加密密碼！ pwds = usr_test:/iZxz+0GRoA=, usr_test2:/iZxz+0GRoA= ,root:/iZxz+0GRoA= #設(shè)置Atlas的運行方式，設(shè)為true時為守護進程方式，設(shè)為false時為前臺方式，一般開發(fā)調(diào)試時設(shè)為false，線上運行時設(shè)為true daemon = true #設(shè)置Atlas的運行方式，設(shè)為true時Atlas會啟動兩個進程，一個為monitor，一個為worker，monitor在worker意外退出后會自動將其重啟，設(shè)為false時只有worker，沒有monitor，一般開發(fā)調(diào)試時設(shè)為false，線上運行時設(shè)為true keepalive = true #工作線程數(shù)，對Atlas的性能有很大影響，可根據(jù)情況適當(dāng)設(shè)置 event-threads = 10 #日志級別，分為message、warning、critical、error、debug五個級別 log-level = message #日志存放的路徑 log-path = /usr/local/mysql-proxy/log #SQL日志的開關(guān)，可設(shè)置為OFF、ON、REALTIME，OFF代表不記錄SQL日志，ON代表記錄SQL日志，REALTIME代表記錄SQL日志且實時寫入磁盤，默認(rèn)為OFF #sql-log = OFF #實例名稱，用于同一臺機器上多個Atlas實例間的區(qū)分 #instance = test #Atlas監(jiān)聽的工作接口IP和端口 proxy-address = 0.0.0.0:1234 #Atlas監(jiān)聽的管理接口IP和端口 admin-address = 0.0.0.0:2345 #分表設(shè)置，此例中person為庫名，mt為表名，id為分表字段，3為子表數(shù)量，可設(shè)置多項，以逗號分隔，若不分表則不需要設(shè)置該項 #tables = person.mt.id.3 #默認(rèn)字符集，設(shè)置該項后客戶端不再需要執(zhí)行SET NAMES語句 #charset = utf8 #允許連接Atlas的客戶端的IP，可以是精確IP，也可以是IP段，以逗號分隔，若不設(shè)置該項則允許所有IP連接，否則只允許列表中的IP連接 #client-ips = 127.0.0.1, 192.168.1 #Atlas前面掛接的LVS的物理網(wǎng)卡的IP(注意不是虛IP)，若有LVS且設(shè)置了client-ips則此項必須設(shè)置，否則可以不設(shè)置 #lvs-ips = 192.168.1.1 5.啟動數(shù)據(jù)節(jié)點(分別在三個數(shù)據(jù)節(jié)點上db169、db172、db173) 5.1 啟動mysql數(shù)據(jù)庫 5.2 啟動atlas : /usr/local/mysql-proxy/bin/mysql-proxyd test start 5.3 啟動lvs腳本： /etc/init/lvsdr.sh start 6.啟動keepalived（db162、db163上） /etc/init.d/keepalived start 7.驗證： 啟動keepalived后，主節(jié)點為db162，查看vip是不是啟動了： [root@db162 ~]# ip ad 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:1d:7d:a8:40:d9 brd ff:ff:ff:ff:ff:ff inet 192.168.1.162/24 brd 192.168.1.255 scope global eth0 inet 192.168.1.201/32 scope global eth0 inet6 fe80::21d:7dff:fea8:40d9/64 scope link valid_lft forever preferred_lft forever 驗證此節(jié)點沒有1234端口監(jiān)聽： [root@db162 ~]# netstat -anp|grep 1234 此處無輸出 在192.168.1.55（db55）上連接192.168.1.201（注意此節(jié)點沒有1234端口在監(jiān)聽，發(fā)來的連接會被路由到真正的數(shù)據(jù)節(jié)點） [root@db55 ~]# mysql -h 192.168.1.201 -P1234 -uroot -p123456 Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1871354501 Server version: 5.0.81-log Percona XtraDB Cluster binary (GPL) 5.6.19-25.6, Revision 824, wsrep_25.6.r4111 Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | dd | | mcldb | | mysql | | mysqlslap | | performance_schema | | test | +--------------------+ 7 rows in set (0.00 sec) 數(shù)據(jù)是正確的 8.監(jiān)控lvs 使用sysbench壓力測試，然后監(jiān)控線程分布： [root@topdb soft]# sysbench --test=oltp --num-threads=100 --max-requests=100000 --oltp-table-size=1000000 --oltp-test-mode=nontrx --db-driver=mysql --mysql-db=dd --mysql-host=192.168.1.201 --mysql-port=1234 --mysql-user=root --mysql-password=123456 --oltp-nontrx-mode=select --oltp-read-only=on --db-ps-mode=disable run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 100 Doing OLTP test. Running non-transactional test Doing read-only test Using Special distribution (12 iterations, 1 pct of values are returned in 75 pct cases) Using 'BEGIN' for starting transactions Using auto_inc on the id column Maximum number of requests for OLTP test is limited to 100000 Threads started! Done. OLTP test statistics: queries performed: read: 100033 write: 0 other: 0 total: 100033 transactions: 100033 (13416.81 per sec.) deadlocks: 0 (0.00 per sec.) read/write requests: 100033 (13416.81 per sec.) other operations: 0 (0.00 per sec.) Test execution summary: total time: 7.4558s total number of events: 100033 total time taken by event execution: 744.5136 per-request statistics: min: 0.71ms avg: 7.44ms max: 407.23ms approx. 95 percentile: 28.56ms Threads fairness: events (avg/stddev): 1000.3300/831.91 execution time (avg/stddev): 7.4451/0.00 [root@db162 ~]# ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.1.201:1234 rr -> 192.168.1.169:1234 Route 3 0 33 -> 192.168.1.172:1234 Route 3 0 34 -> 192.168.1.173:1234 Route 3 0 34 可以看出負(fù)載均衡在了三個節(jié)點。 另外可以自己關(guān)機測試下keepalived的故障轉(zhuǎn)移。 注意點： 1.atlas和mysql數(shù)據(jù)節(jié)點要放在一臺機器上，如果atlas在別處搭建，則lvs會無法使用。 2.keepavlied最好和數(shù)據(jù)節(jié)點不在同一臺機器上，否則可能會有問題。 3.最好有真機測試，虛擬機上之前一直沒搗鼓好。 4.keepalived配置最好根據(jù)示例文件修改，如果有空格、tab鍵不規(guī)范，可能造成無法爭取讀取配置，并且不給出錯誤提示。
|
|
|
