RHEL 5.4下部署LVS(DR)+keepalived實現(xiàn)高性能高可用負(fù)載均衡
一、簡介
LVS是Linux Virtual Server的簡寫,意即Linux虛擬服務(wù)器,是一個虛擬的服務(wù)器集群系統(tǒng)。本項目在1998年5月由章文嵩博士成立,是中國國內(nèi)最早出現(xiàn)的自由軟件項目之一。
目前有三種IP負(fù)載均衡技術(shù)(VS/NAT、VS/TUN和VS/DR);十種調(diào)度算法(rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq)。
Keepalived在這里主要用作RealServer的健康狀態(tài)檢查以及LoadBalance主機和BackUP主機之間failover的實現(xiàn)。
LVS+Keepalived能實現(xiàn)的功能:利用LVS控制器主備模式避免單點故障以及自動刪除故障WEB服務(wù)器結(jié)點并當(dāng)它恢復(fù)后再自動添加到群集中。
拓?fù)鋱D:
二、系統(tǒng)環(huán)境
系統(tǒng)平臺:RHEL?5.4
LVS版本:ipvsadm-1.24
keepalived版本:keepalived-1.1.15
三、安裝
1、在兩臺LVS Server上安裝lvs、keepalived軟件(LVS_MASTER & LVS_BACKUP)
lvs install -------------
[root@LVS-MASTER ~]# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
[root@LVS-MASTER ~]# ln -s /usr/src/kernels/2.6.18-194.el5-i686//usr/src/linux/
[root@LVS-MASTER ~]# tar zxvf ipvsadm-1.24.tar.gz
[root@LVS-MASTER ~]# cd ipvsadm-1.24
[root@LVS-MASTER ipvsadm-1.24]# make && make install
Keepalived install?
-------------
[root@LVS-MASTER ~]# wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz
[root@LVS-MASTER ~]# tar zxvf keepalived-1.1.15.tar.gz
[root@LVS-MASTER ~]# cd keepalived-1.1.15
[root@LVS-MASTER keepalived-1.1.15]# ./configure && make && make install
#########?將keepalived做成啟動服務(wù),方便管理##########
[root@LVS-MASTER ~]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
[root@LVS-MASTER ~]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
[root@LVS-MASTER ~]# mkdir /etc/keepalived/
[root@LVS-MASTER ~]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
[root@LVS-MASTER ~]# cp /usr/local/sbin/keepalived /usr/sbin/
[root@LVS-MASTER ~]# service keepalived start | stop
2、keepalived的配置
###################? MASTER ###################
! Configuration File for keepalived
global_defs {
notification_email {
root@linux.tang.chao #設(shè)置報警郵件地址,可以設(shè)置多個,每行一個。
mchina_tang@qq.com #需開啟本機的sendmail服務(wù)
}
notification_email_from keepalived@localhost #設(shè)置郵件的發(fā)送地址
smtp_server 127.0.0.1 #設(shè)置smtp server地址
smtp_connect_timeout 30 #設(shè)置連接smtp server的超時時間
router_id LVS_DEVEL #表示運行keepalived服務(wù)器的一個標(biāo)識。發(fā)郵件時顯示在郵件主題的信息
}
vrrp_instance VI_1 {
state MASTER #指定keepalived的角色,MASTER表示此主機是主服務(wù)器,BACKUP表示此主機是備用服務(wù)器
interface eth0 #指定HA監(jiān)測網(wǎng)絡(luò)的接口
virtual_router_id 51 #虛擬路由標(biāo)識,這個標(biāo)識是一個數(shù)字,同一個vrrp實例使用唯一的標(biāo)識。即同一vrrp_instance下,MASTER和BACKUP必須是一致的
priority 100 #定義優(yōu)先級,數(shù)字越大,優(yōu)先級越高,在同一個vrrp_instance下,MASTER的優(yōu)先級必須大于BACKUP的優(yōu)先級
advert_int 1 #設(shè)定MASTER與BACKUP負(fù)載均衡器之間同步檢查的時間間隔,單位是秒
authentication { #設(shè)置驗證類型和密碼
auth_type PASS #設(shè)置驗證類型,主要有PASS和AH兩種
auth_pass 1111 #設(shè)置驗證密碼,在同一個vrrp_instance下,MASTER與BACKUP必須使用相同的密碼才能正常通信
}
virtual_ipaddress { #設(shè)置虛擬IP地址,可以設(shè)置多個虛擬IP地址,每行一個
10.0.0.148
}
}
virtual_server 10.0.0.148 80 { #設(shè)置虛擬服務(wù)器,需要指定虛擬IP地址和服務(wù)端口,IP與端口之間用空格隔開
delay_loop 6 #設(shè)置運行情況檢查時間,單位是秒
lb_algo rr #設(shè)置負(fù)載調(diào)度算法,這里設(shè)置為rr,即輪詢算法
lb_kind DR #設(shè)置LVS實現(xiàn)負(fù)載均衡的機制,有NAT、TUN、DR三個模式可選
persistence_timeout 50 #會話保持時間,單位是秒。這個選項對動態(tài)網(wǎng)頁是非常有用的,為集群系統(tǒng)中的session共享提供了一個很好的解決方案。
#有了這個會話保持功能,用戶的請求會被一直分發(fā)到某個服務(wù)節(jié)點,直到超過這個會話的保持時間。
#需要注意的是,這個會話保持時間是最大無響應(yīng)超時時間,也就是說,用戶在操作動態(tài)頁面時,如果50秒內(nèi)沒有執(zhí)行任何操作,
#那么接下來的操作會被分發(fā)到另外的節(jié)點,但是如果用戶一直在操作動態(tài)頁面,則不受50秒的時間限制
protocol TCP #指定轉(zhuǎn)發(fā)協(xié)議類型,有TCP和UDP兩種
real_server 10.0.0.137 80 { #配置服務(wù)節(jié)點1,需要指定real server的真實IP地址和端口,IP與端口之間用空格隔開
weight 3 #配置服務(wù)節(jié)點的權(quán)值,權(quán)值大小用數(shù)字表示,數(shù)字越大,權(quán)值越高,設(shè)置權(quán)值大小可以為不同性能的服務(wù)器
#分配不同的負(fù)載,可以為性能高的服務(wù)器設(shè)置較高的權(quán)值,而為性能較低的服務(wù)器設(shè)置相對較低的權(quán)值,這樣才能合理地利用和分配系統(tǒng)資源
TCP_CHECK { #realserver的狀態(tài)檢測設(shè)置部分,單位是秒
connect_timeout 10 #表示3秒無響應(yīng)超時
nb_get_retry 3 #表示重試次數(shù)
delay_before_retry 3 #表示重試間隔
connect_port 80
}
}
real_server 10.0.0.139 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
####################?BACKUP ###################
! Configuration File for keepalived
global_defs {
notification_email {
root@linux.tang.chao
mchina_tang@qq.com
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.0.0.148
}
}
virtual_server 10.0.0.148 80 {
delay_loop 6
lb_algo rr
lb_kind DR
persistence_timeout 50
protocol TCP
real_server 10.0.0.137 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 10.0.0.139 80 {
weight 3
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
3、在兩臺Web Server上執(zhí)行realserver.sh腳本,為lo:0綁定VIP地址10.0.0.148、抑制ARP廣播。
[root@web1 ~]# cat realserver.sh
#!/bin/bash #description: Config realserver VIP=10.0.0.148 /etc/rc.d/init.d/functions case "$1" in start) /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP /sbin/route add -host $VIP dev lo:0 echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl -p >/dev/null 2> &1 echo "RealServer Start OK" ;; stop) /sbin/ifconfig lo:0 down /sbin/route del $VIP >/dev/null 2> &1 echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "RealServer Stoped" ;; *) echo "Usage: $0 {start|stop}" exit 1 esac exit 0
[root@web1 ~]# sh realserver.sh start
4、分別在LVS-MASTER、LVS-BACKUP上執(zhí)行service keepalived start啟動keepalived就可實現(xiàn)負(fù)載均衡及高可用集群了;
[root@LVS-MASTER keepalived]# service keepalived start
[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.139:http????????????? Route?? 3????? 0????????? 0??????? ?
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#
查看日志/var/log/messages
[root@LVS-MASTER keepalived]# tail -n 30/var/log/messages
Apr 19 03:59:58 LVS-MASTER avahi-daemon[2845]: Withdrawing address record for10.0.0.148 on eth0.
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Keepalived v1.2.1 (04/17,2012)
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Healthcheck child process,pid=5960
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink reflector
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink command channel
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Configuration is using :12257 Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Using LinkWatch kernelnetlink reflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.137:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.139:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlinkreflector
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlink commandchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering gratutious ARP sharedchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Opening file'/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Configuration is using : 36698Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Using LinkWatch kernel netlinkreflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: VRRP sockpool: [ifindex(2),proto(112), fd(11,12)]
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting VRRP child process, pid=5961
Apr 19 04:00:20 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Transition toMASTER STATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 04:00:21 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:21 LVS-MASTER avahi-daemon[2845]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:26 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
[root@LVS-MASTER keepalived]#
四、測試
接下來做高可用性測試&故障切換測試......
####高可用性測試####
模擬故障,將LVS-MASTER上的keepalived服務(wù)停掉,然后觀察LVS-BACKUP上的日志,信息如下
[root@LVS-BACKUP keepalived]# tail -f /var/log/messages
Apr 19 03:33:04 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Transition to MASTER STATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP avahi-daemon[2825]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 03:33:10 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
從日志中可知,主機出現(xiàn)故障后,備機立刻檢測到,此時備機變?yōu)镸ASTER角色,并且接管了主機的虛擬IP資源,最后將虛擬IP綁定在etho設(shè)備上。
將LVS-MASTER?上的keepalived服務(wù)開啟后,LVS-BACKUP的日志狀態(tài)。
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Entering BACKUP STATE
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP avahi-daemon[2825]: Withdrawing address record for 10.0.0.148 on eth0.
從日志可知,備機在檢測到主機重新恢復(fù)正常后,釋放了虛擬IP資源重新成為BACKUP角色
####故障切換測試####
故障切換是測試當(dāng)某個節(jié)點出現(xiàn)故障后,Keepalived監(jiān)制模塊是否能及時發(fā)現(xiàn)然后屏蔽故障節(jié)點,同時將服務(wù)器轉(zhuǎn)移到正常節(jié)點來執(zhí)行。
將web2節(jié)點服務(wù)停掉,假設(shè)這個節(jié)點出現(xiàn)故障,然后主、備機日志信息如下
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] failed!!!
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:Removing service [10.0.0.139:80] from VS [10.0.0.148:80]
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#
從以上可以看出,Keepalived監(jiān)控模塊檢測到10.0.0.139這臺主機出現(xiàn)故障后,將些web2從集群系統(tǒng)中剔除掉了。?此時訪問http://10.0.0.148只能看到web1了)
重新啟動web2節(jié)點的服務(wù),日志信息如下:
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] success.
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:Adding service [10.0.0.139:80] to VS [10.0.0.148:80]
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.
[root@LVS-MASTERkeepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.139:http????????????? Route?? 3????? 0????????? 0??????? ?
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#
Keepalived監(jiān)控模塊檢測到10.0.0.139這臺主機恢復(fù)正常后,又將此節(jié)點加入集群系統(tǒng)中,再次訪問就可以訪問到web2頁面了)
?
CentOS 6.3下部署LVS(NAT)+keepalived實現(xiàn)高性能高可用負(fù)載均衡
http://www.cnblogs.com/mchina/archive/2012/08/27/2644391.html
更多文章、技術(shù)交流、商務(wù)合作、聯(lián)系博主
微信掃碼或搜索:z360901061

微信掃一掃加我為好友
QQ號聯(lián)系: 360901061
您的支持是博主寫作最大的動力,如果您喜歡我的文章,感覺我的文章對您有幫助,請用微信掃描下面二維碼支持博主2元、5元、10元、20元等您想捐的金額吧,狠狠點擊下面給點支持吧,站長非常感激您!手機微信長按不能支付解決辦法:請將微信支付二維碼保存到相冊,切換到微信,然后點擊微信右上角掃一掃功能,選擇支付二維碼完成支付。
【本文對您有幫助就好】元
