RHEL 5.4下部署LVS(DR)+keepalived實現(xiàn)高性能高可用負(fù)載均衡

一、簡介

LVS是Linux Virtual Server的簡寫，意即Linux虛擬服務(wù)器，是一個虛擬的服務(wù)器集群系統(tǒng)。本項目在1998年5月由章文嵩博士成立，是中國國內(nèi)最早出現(xiàn)的自由軟件項目之一。

目前有三種IP負(fù)載均衡技術(shù)（VS/NAT、VS/TUN和VS/DR）；十種調(diào)度算法（rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）。

Keepalived在這里主要用作RealServer的健康狀態(tài)檢查以及LoadBalance主機和BackUP主機之間failover的實現(xiàn)。

LVS+Keepalived能實現(xiàn)的功能：利用LVS控制器主備模式避免單點故障以及自動刪除故障WEB服務(wù)器結(jié)點并當(dāng)它恢復(fù)后再自動添加到群集中。

拓?fù)鋱D：

RHEL 5.4下部署LVS(DR)+keepalived實現(xiàn)高性能高可用負(fù)載均衡

二、系統(tǒng)環(huán)境

系統(tǒng)平臺：RHEL?5.4

LVS版本：ipvsadm-1.24

keepalived版本：keepalived-1.1.15

三、安裝

1、在兩臺LVS Server上安裝lvs、keepalived軟件（LVS_MASTER & LVS_BACKUP）
lvs install -------------

[root@LVS-MASTER ~]# wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz

[root@LVS-MASTER ~]# ln -s /usr/src/kernels/2.6.18-194.el5-i686//usr/src/linux/

[root@LVS-MASTER ~]# tar zxvf ipvsadm-1.24.tar.gz

[root@LVS-MASTER ~]# cd ipvsadm-1.24

[root@LVS-MASTER ipvsadm-1.24]# make && make install

Keepalived install? -------------

[root@LVS-MASTER ~]# wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz

[root@LVS-MASTER ~]# tar zxvf keepalived-1.1.15.tar.gz

[root@LVS-MASTER ~]# cd keepalived-1.1.15

[root@LVS-MASTER keepalived-1.1.15]# ./configure && make && make install

#########?將keepalived做成啟動服務(wù)，方便管理##########

[root@LVS-MASTER ~]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/

[root@LVS-MASTER ~]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

[root@LVS-MASTER ~]# mkdir /etc/keepalived/

[root@LVS-MASTER ~]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

[root@LVS-MASTER ~]# cp /usr/local/sbin/keepalived /usr/sbin/

[root@LVS-MASTER ~]# service keepalived start | stop

2、keepalived的配置

###################? MASTER ###################

          
             ! Configuration File for keepalived



global_defs {

   notification_email {

     root@linux.tang.chao               #設(shè)置報警郵件地址，可以設(shè)置多個，每行一個。

     mchina_tang@qq.com                 #需開啟本機的sendmail服務(wù)

   }

   notification_email_from keepalived@localhost         #設(shè)置郵件的發(fā)送地址

   smtp_server 127.0.0.1                                #設(shè)置smtp server地址

   smtp_connect_timeout 30                              #設(shè)置連接smtp server的超時時間

   router_id LVS_DEVEL                                  #表示運行keepalived服務(wù)器的一個標(biāo)識。發(fā)郵件時顯示在郵件主題的信息

}



vrrp_instance VI_1 {

    state MASTER              #指定keepalived的角色，MASTER表示此主機是主服務(wù)器，BACKUP表示此主機是備用服務(wù)器

    interface eth0            #指定HA監(jiān)測網(wǎng)絡(luò)的接口

    virtual_router_id 51      #虛擬路由標(biāo)識，這個標(biāo)識是一個數(shù)字，同一個vrrp實例使用唯一的標(biāo)識。即同一vrrp_instance下，MASTER和BACKUP必須是一致的

    priority 100              #定義優(yōu)先級，數(shù)字越大，優(yōu)先級越高，在同一個vrrp_instance下，MASTER的優(yōu)先級必須大于BACKUP的優(yōu)先級

    advert_int 1              #設(shè)定MASTER與BACKUP負(fù)載均衡器之間同步檢查的時間間隔，單位是秒

    authentication {          #設(shè)置驗證類型和密碼

        auth_type PASS        #設(shè)置驗證類型，主要有PASS和AH兩種

        auth_pass 1111        #設(shè)置驗證密碼，在同一個vrrp_instance下，MASTER與BACKUP必須使用相同的密碼才能正常通信

    }

    virtual_ipaddress {       #設(shè)置虛擬IP地址，可以設(shè)置多個虛擬IP地址，每行一個

        10.0.0.148

    }

}



virtual_server 10.0.0.148 80 {      #設(shè)置虛擬服務(wù)器，需要指定虛擬IP地址和服務(wù)端口，IP與端口之間用空格隔開

    delay_loop 6                    #設(shè)置運行情況檢查時間，單位是秒

    lb_algo rr                      #設(shè)置負(fù)載調(diào)度算法，這里設(shè)置為rr，即輪詢算法

    lb_kind DR                      #設(shè)置LVS實現(xiàn)負(fù)載均衡的機制，有NAT、TUN、DR三個模式可選

    persistence_timeout 50          #會話保持時間，單位是秒。這個選項對動態(tài)網(wǎng)頁是非常有用的，為集群系統(tǒng)中的session共享提供了一個很好的解決方案。

                                    #有了這個會話保持功能，用戶的請求會被一直分發(fā)到某個服務(wù)節(jié)點，直到超過這個會話的保持時間。

                                    #需要注意的是，這個會話保持時間是最大無響應(yīng)超時時間，也就是說，用戶在操作動態(tài)頁面時，如果50秒內(nèi)沒有執(zhí)行任何操作，

                                    #那么接下來的操作會被分發(fā)到另外的節(jié)點，但是如果用戶一直在操作動態(tài)頁面，則不受50秒的時間限制

    protocol TCP                    #指定轉(zhuǎn)發(fā)協(xié)議類型，有TCP和UDP兩種



    real_server 10.0.0.137 80 {     #配置服務(wù)節(jié)點1，需要指定real server的真實IP地址和端口，IP與端口之間用空格隔開

        weight 3                    #配置服務(wù)節(jié)點的權(quán)值，權(quán)值大小用數(shù)字表示，數(shù)字越大，權(quán)值越高，設(shè)置權(quán)值大小可以為不同性能的服務(wù)器

                                    #分配不同的負(fù)載，可以為性能高的服務(wù)器設(shè)置較高的權(quán)值，而為性能較低的服務(wù)器設(shè)置相對較低的權(quán)值，這樣才能合理地利用和分配系統(tǒng)資源

        TCP_CHECK {                 #realserver的狀態(tài)檢測設(shè)置部分，單位是秒

            connect_timeout 10      #表示3秒無響應(yīng)超時

            nb_get_retry 3          #表示重試次數(shù)

            delay_before_retry 3    #表示重試間隔

            connect_port 80

        }

    }

    real_server 10.0.0.139 80 {

        weight 3

        TCP_CHECK {

            connect_timeout 10

            nb_get_retry 3

            delay_before_retry 3

            connect_port 80

        }

    }

}

####################?BACKUP ###################

          
            ! Configuration File for keepalived



global_defs {

   notification_email {

     root@linux.tang.chao

     mchina_tang@qq.com

   }

   notification_email_from Alexandre.Cassen@firewall.loc

   smtp_server 127.0.0.1

   smtp_connect_timeout 30

   router_id LVS_DEVEL

}



vrrp_instance VI_1 {

    state BACKUP

    interface eth0

    virtual_router_id 51

    priority 99

    advert_int 1

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    virtual_ipaddress {

        10.0.0.148

    }

}



virtual_server 10.0.0.148 80 {

    delay_loop 6

    lb_algo rr

    lb_kind DR

    persistence_timeout 50

    protocol TCP



    real_server 10.0.0.137 80 {

        weight 3

        TCP_CHECK {

            connect_timeout 10

            nb_get_retry 3

            delay_before_retry 3

            connect_port 80

        }

    }

    real_server 10.0.0.139 80 {

        weight 3

        TCP_CHECK {

            connect_timeout 10

            nb_get_retry 3

            delay_before_retry 3

            connect_port 80

        }

    }

}

3、在兩臺Web Server上執(zhí)行realserver.sh腳本，為lo:0綁定VIP地址10.0.0.148、抑制ARP廣播。

[root@web1 ~]# cat realserver.sh

          
            #!/bin/bash

#description: Config realserver



VIP=10.0.0.148

 

/etc/rc.d/init.d/functions

 

case "$1" in

start)

       /sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP

       /sbin/route add -host $VIP dev lo:0

       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

       sysctl -p >/dev/null 2>
          
          
            &1
          
          
            

       echo "RealServer Start OK"

       ;;

stop)

       /sbin/ifconfig lo:0 down

       /sbin/route del $VIP >/dev/null 2>
          
          
            &1
          
          
            

       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

       echo "RealServer Stoped"

       ;;

*)

       echo "Usage: $0 {start|stop}"

       exit 1

esac

 

exit 0

[root@web1 ~]# sh realserver.sh start

4、分別在LVS-MASTER、LVS-BACKUP上執(zhí)行service keepalived start啟動keepalived就可實現(xiàn)負(fù)載均衡及高可用集群了；

[root@LVS-MASTER keepalived]# service keepalived start

[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.139:http????????????? Route?? 3????? 0????????? 0??????? ?
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#

查看日志/var/log/messages

[root@LVS-MASTER keepalived]# tail -n 30/var/log/messages
Apr 19 03:59:58 LVS-MASTER avahi-daemon[2845]: Withdrawing address record for10.0.0.148 on eth0.
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Keepalived v1.2.1 (04/17,2012)
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting Healthcheck child process,pid=5960
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink reflector
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Registering Kernelnetlink command channel
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Configuration is using :12257 Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Using LinkWatch kernelnetlink reflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.137:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_healthcheckers: Activating healtcheckerfor service [10.0.0.139:80]
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.159 added
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlinkreflector
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering Kernel netlink commandchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Registering gratutious ARP sharedchannel
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Opening file'/etc/keepalived/keepalived.conf'.
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Configuration is using : 36698Bytes
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: Using LinkWatch kernel netlinkreflector...
Apr 19 04:00:19 LVS-MASTER Keepalived_vrrp: VRRP sockpool: [ifindex(2),proto(112), fd(11,12)]
Apr 19 04:00:19 LVS-MASTER Keepalived: Starting VRRP child process, pid=5961
Apr 19 04:00:20 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Transition toMASTER STATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 04:00:21 LVS-MASTER Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.136 added
Apr 19 04:00:21 LVS-MASTER avahi-daemon[2845]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 04:00:21 LVS-MASTER Keepalived_vrrp: Netlink reflector reports IP10.0.0.136 added
Apr 19 04:00:26 LVS-MASTER Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
[root@LVS-MASTER keepalived]#

四、測試

接下來做高可用性測試&故障切換測試......

####高可用性測試####

模擬故障，將LVS-MASTER上的keepalived服務(wù)停掉，然后觀察LVS-BACKUP上的日志，信息如下

[root@LVS-BACKUP keepalived]# tail -f /var/log/messages
Apr 19 03:33:04 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Transition to MASTER STATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Entering MASTERSTATE
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) settingprotocol VIPs.
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148
Apr 19 03:33:05 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reportsIP 10.0.0.157 added
Apr 19 03:33:05 LVS-BACKUP avahi-daemon[2825]: Registering new address recordfor 10.0.0.148 on eth0.
Apr 19 03:33:10 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Sendinggratuitous ARPs on eth0 for 10.0.0.148

從日志中可知，主機出現(xiàn)故障后，備機立刻檢測到，此時備機變?yōu)镸ASTER角色，并且接管了主機的虛擬IP資源，最后將虛擬IP綁定在etho設(shè)備上。

將LVS-MASTER?上的keepalived服務(wù)開啟后，LVS-BACKUP的日志狀態(tài)。

Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) Received higher prio advert
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1)Entering BACKUP STATE
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: VRRP_Instance(VI_1) removing protocol VIPs.
Apr 19 03:34:02 LVS-BACKUP Keepalived_vrrp: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP Keepalived_healthcheckers: Netlink reflector reports IP 10.0.0.157 removed
Apr 19 03:34:02 LVS-BACKUP avahi-daemon[2825]: Withdrawing address record for 10.0.0.148 on eth0.

從日志可知，備機在檢測到主機重新恢復(fù)正常后，釋放了虛擬IP資源重新成為BACKUP角色

####故障切換測試####

故障切換是測試當(dāng)某個節(jié)點出現(xiàn)故障后，Keepalived監(jiān)制模塊是否能及時發(fā)現(xiàn)然后屏蔽故障節(jié)點，同時將服務(wù)器轉(zhuǎn)移到正常節(jié)點來執(zhí)行。

將web2節(jié)點服務(wù)停掉，假設(shè)這個節(jié)點出現(xiàn)故障，然后主、備機日志信息如下

Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] failed!!!
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers:Removing service [10.0.0.139:80] from VS [10.0.0.148:80]
Apr 19 03:35:04 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.

[root@LVS-MASTER keepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#

從以上可以看出，Keepalived監(jiān)控模塊檢測到10.0.0.139這臺主機出現(xiàn)故障后，將些web2從集群系統(tǒng)中剔除掉了。?此時訪問http://10.0.0.148只能看到web1了)

keepalived

重新啟動web2節(jié)點的服務(wù)，日志信息如下：

Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:TCP connection to [10.0.0.139:80] success.
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers:Adding service [10.0.0.139:80] to VS [10.0.0.148:80]
Apr 19 03:38:22 LVS-MASTER Keepalived_healthcheckers: Remote SMTP server [127.0.0.1:25] connected.

[root@LVS-MASTERkeepalived]# ipvsadm -L
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
? -> RemoteAddress:Port?????????? Forward Weight ActiveConn InActConn
TCP? 10.0.0.148:http rr persistent 50
? -> 10.0.0.139:http????????????? Route?? 3????? 0????????? 0??????? ?
? -> 10.0.0.137:http????????????? Route?? 3????? 0????????? 0??????? ?
[root@LVS-MASTER keepalived]#

Keepalived監(jiān)控模塊檢測到10.0.0.139這臺主機恢復(fù)正常后，又將此節(jié)點加入集群系統(tǒng)中，再次訪問就可以訪問到web2頁面了）

lvs

CentOS 6.3下部署LVS(NAT)+keepalived實現(xiàn)高性能高可用負(fù)載均衡

http://www.cnblogs.com/mchina/archive/2012/08/27/2644391.html

RHEL 5.4下部署LVS(DR)+keepalived實現(xiàn)高性能高可用負(fù)載均衡

更多文章、技術(shù)交流、商務(wù)合作、聯(lián)系博主

微信掃碼或搜索：z360901061

微信掃一掃加我為好友

QQ號聯(lián)系： 360901061

您的支持是博主寫作最大的動力，如果您喜歡我的文章，感覺我的文章對您有幫助，請用微信掃描下面二維碼支持博主2元、5元、10元、20元等您想捐的金額吧，狠狠點擊下面給點支持吧，站長非常感激您！手機微信長按不能支付解決辦法：請將微信支付二維碼保存到相冊，切換到微信，然后點擊微信右上角掃一掃功能，選擇支付二維碼完成支付。

【本文對您有幫助就好】元

2元

5元

10元

20元

自定義