Lvs+Keepalived+Bind+web构建高可用负载均衡系统

原创作品，允许转载，转载时请务必以超链接形式标明文章原始出处、作者信息和本声明。否则将追究法律责任。http://hatech.blog.51cto.com/8360868/1417899

——————————-

一、前言

二、环境

三、配置

1.LB-Master及LB-Backup配置

(1)LB-Master及LB-Backup安装keepalived和ipvsadm

(2)LB-Master的keepalived主配置文档

(3)LB-Backup的keepalived主配置文档

2.DNS-Master及DNS-Backup配置

(1)DNS-Master及DNS-Backup网卡设置

(2)DNS-Master及DNS-Backup安装bind及ARP设置

(3)DNS-Master配置

(4)DNS-Backup配置

3.web1及web2配置

四、测试

五、故障模拟

模拟LB-Master失效
模拟DNS-Master失效
模拟web1失效

——————————-

一、前言

这个实验是实现企业高可用负载均衡的web案例，无论Master或Backup宕掉任意一台，业务照常可用。分析拓扑，其中LB-Master负责web的调度，LB-Master负责DNS的调度，实现负载均衡，如果其中一台出现故障，那么web和DNS服务会立即转移到另一台服务器上，DNS分为主辅同步，实现高可用，web在生产环境下页面应该是一致的，为了试验效果，web页面设置不一样，在实际环境下，web后端应架设共享存储，实现web页面的一致性。

二、环境

系统：CentOS6.4 32位 6台

拓扑图：

IP划分：

三、配置

1.LB-Master及LB-Backup配置

(1)LB-Master及LB-Backup安装keepalived和ipvsadm

12345678910111213141516171819202122232425262728293031323334 # yum groupinstall "Additional Development" //安装开发工具# yum groupinstall "Development tools"# tar -zxvf keepalived-1.2.1.tar.gz -C /usr/local/src/# cd /usr/local/src/keepalived-1.2.1# ./cnfigureKeepalived configuration------------------------Keepalived version : 1.2.1Compiler : gccCompiler flags : -g -O2Extra Lib : -lpopt -lssl -lcrypto Use IPVS Framework : No //配置出现错误IPVS sync daemon support : NoUse VRRP Framework : YesUse Debug flags : No解决方法：# yum install kernel-devel ipvsadm# ln -s /usr/src/kernels/2.6.32-358.el6.i686/ /usr/src/linux# ./cnfigure //再次配置环境# make //编译# make install //安装 # cd /usr/local/etc //keepalived默认安装路径# lldrwxr-xr-x. 3 root root 4096 May 24 00:37 keepaliveddrwxr-xr-x. 3 root root 4096 May 24 00:29 rc.ddrwxr-xr-x. 2 root root 4096 May 24 00:29 sysconfig 配置以系统方式service启动# cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/ # cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/# mkdir /etc/keepalived# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/# cp /usr/local/sbin/keepalived /usr/sbin/

(2)LB-Master的keepalived主配置文档

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105 # cat /etc/keepalived/keepalived.conf#guration File for keepalived#global defineglobal_defs { router_id Haweb_1 }vrrp_sync_group VGM { group { VI_WEB }}vrrp_sync_group VGN { group { HA_DNS }} # vvrp_instance define #vrrp_instance VI_WEB { state MASTER interface eth0 lvs_sync_daemon_inteface eth0 virtual_router_id 55 priority 100 advert_int 5 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.2.200/24 dev eth0 }}########## LVS ###########virtual_server 192.168.2.200 80 { delay_loop 6 lb_algo rr lb_kind DR# persistence_timeout 20 protocol TCP real_server 192.168.2.50 80 { weight 100 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.2.60 80 { weight 100 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 80 } }}### HA DNS START ###vrrp_instance HA_DNS { state BACKUP interface eth0 lvs_sync_daemon_inteface eth0 virtual_router_id 56 priority 90 advert_int 5 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.2.100/24 dev eth0 }}########## LVS DNS ###########virtual_server 192.168.2.100 53 { delay_loop 6 lb_algo rr lb_kind DR# persistence_timeout 6 protocol UDP real_server 192.168.2.30 53 { weight 100 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 53 } } real_server 192.168.2.40 53 { weight 100 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 53 } }} ##################################

(3)LB-Backup的keepalived主配置文档

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105 # cat /etc/keepalived/keepalived.conf#guration File for keepalived#global defineglobal_defs { router_id Haweb_1 }vrrp_sync_group VGM { group { VI_WEB }}vrrp_sync_group VGN { group { HA_DNS }} # vvrp_instance define #vrrp_instance VI_WEB { state BACKUP interface eth0 lvs_sync_daemon_inteface eth0 virtual_router_id 55 priority 90 advert_int 5 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.2.200/24 dev eth0 }}########## LVS ###########virtual_server 192.168.2.200 80 { delay_loop 6 lb_algo rr lb_kind DR# persistence_timeout 20 protocol TCP real_server 192.168.2.50 80 { weight 100 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 80 } } real_server 192.168.2.60 80 { weight 100 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 80 } }}### HA DNS START ###vrrp_instance HA_DNS { state MASTER interface eth0 lvs_sync_daemon_inteface eth0 virtual_router_id 56 priority 100 advert_int 5 authentication { auth_type PASS auth_pass 123456 } virtual_ipaddress { 192.168.2.100/24 dev eth0 }}########## LVS DNS ###########virtual_server 192.168.2.100 53 { delay_loop 6 lb_algo rr lb_kind DR# persistence_timeout 6 protocol UDP real_server 192.168.2.30 53 { weight 100 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 53 } } real_server 192.168.2.40 53 { weight 100 TCP_CHECK { connect_timeout 3 nb_get_retry 3 delay_before_retry 3 connect_port 53 } }} ##################################

2.DNS-Master及DNS-Backup配置

(1)DNS-Master及DNS-Backup网卡设置

DNS-Master及DNS-Backup在各自网卡基础上再添加lo:0端口。

(2)DNS-Master及DNS-Backup安装bind及ARP设置

12345 # yum install bind bind-chroot# vim /etc/sysctl.conf //添加以下两行net.ipv4.conf.all.arp_announce = 2 //关闭所有接口的ARP响应和发布net.ipv4.conf.all.arp_ignore = 1# sysctl -p

(3)DNS-Master配置

123456 # service named start Generating /etc/rndc.key: [ OK ]Starting named: [ OK ] # cd /var/named/chroot/etc/# vim named.conf //编辑主配置文档

1	`# vim named.rfc1912.zones //编辑区域声明文件[object Object]`

12345 # scp named.conf named.rfc1912.zones 192.168.2.40:/var/named/chroot/etc/# cd /var/named/chroot/var/named/# cp -p named.localhost abc.com.zone# cp -p named.localhost abc.com.local# vim abc.com.zone

1	`# vim abc.com.local`

(4)DNS-Backup配置

123	`# cd /var/named/chroot/etc/# chgrp named named.conf named.rfc1912.zones# vim named.rfc1912.zones`

123456 # service named start# cd /var/named/chroot/var/named/slaves# ll //区域文件已经同步-rw-r--r--. 1 named named 325 May 27 19:31 abc.com.local-rw-r--r--. 1 named named 340 May 27 19:39 abc.com.zone# vim abc.com.zone

1	`# vim abc.com.local`

3.web1及web2配置

在原有网卡基础上再添加一块lo:0网卡

12345678 # yum install httpd# echo "This is web1.">/var/www/html/index.html //web1配置# echo "This is web2.">/var/www/html/index.html //web2配置# vim /etc/sysctl.conf //添加以下两行net.ipv4.conf.all.arp_announce = 2net.ipv4.conf.all.arp_ignore = 1# sysctl -p# service httpd start

四、测试：

测试客户端IP配置：

客户端浏览器访问http://www.abc.com,并不断刷新，可以看出web1和web2交替不断出现，在实际生产环境下web内容是一样的。

LB-Master（负责web轮询）：

1234567891011121314151617181920212223 # ipvsadmIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.2.200:http rr -> 192.168.2.50:http Route 100 0 4 -> 192.168.2.60:http Route 100 0 4 UDP 192.168.2.100:domain rr -> 192.168.2.30:domain Route 100 0 0 -> 192.168.2.40:domain Route 100 0 0 # ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:0c:29:1c:d4:8d brd ff:ff:ff:ff:ff:ff inet 192.168.2.10/24 brd 192.168.2.255 scope global eth0 inet 192.168.2.200/24 scope global secondary eth0 inet6 fe80::20c:29ff:fe1c:d48d/64 scope link valid_lft forever preferred_lft forever

LB-Backup（负责DNS轮询）：

1234567891011121314151617181920212223 IP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.2.200:http rr -> 192.168.2.50:http Route 100 0 0 -> 192.168.2.60:http Route 100 0 0 UDP 192.168.2.100:domain rr -> 192.168.2.30:domain Route 100 0 6 -> 192.168.2.40:domain Route 100 0 6 # ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:0c:29:22:3d:01 brd ff:ff:ff:ff:ff:ff inet 192.168.2.20/24 brd 192.168.2.255 scope global eth0 inet 192.168.2.100/24 scope global secondary eth0 inet6 fe80::20c:29ff:fe22:3d01/64 scope link valid_lft forever preferred_lft forever

五、故障模拟

模拟LB-Master失效

（1）停掉LB-Master的keepalived服务

1234567891011121314151617 # service keepalived stopStopping keepalived: [ OK ]# ipvsadmIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn# ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:0c:29:1c:d4:8d brd ff:ff:ff:ff:ff:ff inet 192.168.2.10/24 brd 192.168.2.255 scope global eth0 inet6 fe80::20c:29ff:fe1c:d48d/64 scope link valid_lft forever preferred_lft forever

（2）查看LB-Backup状态（不断在测试客户端刷新网页）

1234567891011121314151617181920212223 # ip addr1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000 link/ether 00:0c:29:22:3d:01 brd ff:ff:ff:ff:ff:ff inet 192.168.2.20/24 brd 192.168.2.255 scope global eth0 inet 192.168.2.100/24 scope global secondary eth0 inet 192.168.2.200/24 scope global secondary eth0 inet6 fe80::20c:29ff:fe22:3d01/64 scope link valid_lft forever preferred_lft forever# ipvsadmIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.2.200:http rr -> 192.168.2.50:http Route 100 0 9 -> 192.168.2.60:http Route 100 0 10 UDP 192.168.2.100:domain rr -> 192.168.2.30:domain Route 100 0 6 -> 192.168.2.40:domain Route 100 0 6

2.模拟DNS-Master失效

（1）停掉DNS-Master的named服务，并恢复LB-Master的服务

12	`# service named stopStopping named: . [ OK ]`

（2）查看LB-Master状态（不断在测试客户端刷新网页）

123456789 # ipvsadmIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.2.200:http rr -> 192.168.2.50:http Route 100 0 13 -> 192.168.2.60:http Route 100 0 14 UDP 192.168.2.100:domain rr -> 192.168.2.40:domain Route 100 0 0

（3）查看LB-Backup状态

123456789 # ipvsadmIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.2.200:http rr -> 192.168.2.50:http Route 100 0 0 -> 192.168.2.60:http Route 100 0 0 UDP 192.168.2.100:domain rr -> 192.168.2.40:domain Route 100 0 6