keepalived+nginx实现nginx的高可用
===================================
nginx的高可用
nginx实现后端realserver的负载均衡
===================================
实验环境：OS：Centos 6.4(redhat 6.4)
yum源:
|
1
2
3
4
5
6
7
8
9
10
11
|
`[centos]`
`name=sohu-centos`
`baseurl=http:“//mirrors“.sohu.com“/centos/“$releasever“/os/“$basearch`
`gpgcheck=1`
`enable“=0`
`gpgkey=http:“//mirrors“.sohu.com“/centos/RPM-GPG-KEY-CentOS-6`
`[epel]`
`name=sohu-epel`
`baseurl=http:“//mirrors“.sohu.com“/fedora-epel/“$releasever/$basearch/`
`enable“=1`
`gpgcheck=0`
` `
|
**
拓扑图： **
拓扑图的规划：
| IP地址 | 软件 | |
| Master | 172.16.22.1(VIP：172.16.22.100) | keepalived+nginx |
| Backup | 172.16.22.2(VIP：172.16.22.100) | keepalived+nginx |
| apache1 | 172.16.22.3 | httpd |
| apache2 | 172.16.22.4 | httpd |
**
此架构需考虑的问题**
1)、Master没挂，则Master占有vip且nginx运行在Master上
2)、Master挂了，则backup抢占vip且在backup上运行nginx服务
3)、如果master服务器上的nginx服务挂了，则vip资源转移到backup服务器上
4)、检测后端服务器的健康状态
Master和Backup两边都开启nginx服务，无论Master还是Backup，当其中的一个keepalived服务停止后，vip都会漂移到keepalived服务还在的节点上，
如果要想使nginx服务挂了，vip也漂移到另一个节点，则必须用脚本或者在配置文件里面用shell命令来控制。
首先必须明确后端服务器的健康状态检测keepalived在这种架构上是无法检测的，后端服务器的健康状态检测是有nginx来判断的，但是nginx 的检测机制有一定的缺陷，后端服务器某一个宕机之后，nginx还是会分发请求给它，在一定的时间内后端服务响应不了，nginx则会发给另外一个服务 器，然后当客户的请求来了，nginx会一段时间内不会把请求分发给已经宕机的服务器，但是过一段时间后，nginx还是会把分发请求发给宕机的服务器 上。
一、安装keepalived+nginx
Master：
1、安装keepalived和编译安装nginx
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
`[root@jie1 ~]“# yum -y install keepalived`
`[root@jie1 ~]“#tar xf nginx-1.4.2.tar.gz`
`[root@jie1 ~]“#yum -y groupinstall “Development tools” “Server Platform Development”`
`[root@jie1 ~]“#yum -y install pcre-devel`
`[root@jie1 ~]“# cd nginx-1.4.2`
`[root@jie1 nginx-1.4.2]“# groupadd nginx`
`[root@jie1 nginx-1.4.2]“# useradd -r -g nginx nginx`
`[root@jie1 nginx-1.4.2]“#./configure \`
`–prefix=“/usr“\`
`–sbin-path=“/usr/sbin/nginx“\`
`–conf-path=“/etc/nginx/nginx“.conf \`
`–error-log-path=“/var/log/nginx/error“.log \`
`–http-log-path=“/var/log/nginx/access“.log \`
`–pid-path=“/var/run/nginx/nginx“.pid \`
`–lock-path=“/var/lock/nginx“.lock \`
`–user=nginx \`
`–group=nginx \`
`–with-http_ssl_module \`
`–with-http_flv_module \`
`–with-http_stub_status_module \`
`–with-http_gzip_static_module \`
`–http-client-body-temp-path=“/var/tmp/nginx/client/“\`
`–http-proxy-temp-path=“/var/tmp/nginx/proxy/“\`
`–http-fastcgi-temp-path=“/var/tmp/nginx/fcgi/“\`
`–http-uwsgi-temp-path=“/var/tmp/nginx/uwsgi“\`
`–http-scgi-temp-path=“/var/tmp/nginx/scgi“\`
`–with-pcre`
`[root@jie1 nginx-1.4.2]“# make && make install`
|
**
2、提供nginx的system V服务脚本文件**
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
|
`[root@jie1 nginx-1.4.2]“# vim /etc/rc.d/init.d/nginx`
`#!/bin/sh`
`#`
`# nginx – this script starts and stops the nginx daemon`
`#`
`# chkconfig: – 85 15`
`# description: Nginx is an HTTP(S) server, HTTP(S) reverse \`
`# proxy and IMAP/POP3 proxy server`
`# processname: nginx`
`# config: /etc/nginx/nginx.conf`
`# config: /etc/sysconfig/nginx`
`# pidfile: /var/run/nginx.pid`
`# Source function library.`
`. “/etc/rc“.d“/init“.d“/functions`
`# Source networking configuration.`
`. “/etc/sysconfig/network`
`# Check that networking is up.`
`[ “”$NETWORKING”“= “”no”“] && exit0`
`nginx=“”/usr/sbin/nginx”`
`prog=$(“basename“$nginx)`
`NGINX_CONF_FILE=“”/etc/nginx/nginx.conf”`
`[ -f “/etc/sysconfig/nginx“] && . “/etc/sysconfig/nginx`
`lockfile=“/var/lock/subsys/nginx`
`make_dirs() {`
`# make required directories`
`user=`nginx -V 2>&1 | “grep“”configure arguments:”“| “sed“’s/[^*]*–user=\([^ ]*\).*/\1/g’“-“
`options=`$nginx -V 2>&1 | “grep“’configure arguments:’““
`foropt “in“$options; “do`
`if“[ “`echo“$opt | “grep“’.*-temp-path’“` ]; “then`
`value=“`echo“$opt | “cut“-d “”=”“-f 2“
`if“[ ! -d “”$value”“]; “then`
`# echo “creating” $value`
`mkdir“-p $value && “chown“-R $user $value`
`fi`
`fi`
`done`
`}`
`start() {`
`[ -x $nginx ] || exit5`
`[ -f $NGINX_CONF_FILE ] || exit6`
`make_dirs`
`echo“-n $“”Starting $prog: “`
`daemon $nginx -c $NGINX_CONF_FILE`
`retval=$?`
`echo`
`[ $retval -eq0 ] && “touch“$lockfile`
`return“$retval`
`}`
`stop() {`
`echo“-n $“”Stopping $prog: “`
`killproc $prog -QUIT`
`retval=$?`
`echo`
`[ $retval -eq0 ] && “rm“-f $lockfile`
`return“$retval`
`}`
`restart() {`
`configtest || “return“$?`
`stop`
`sleep1`
`start`
`}`
`reload() {`
`configtest || “return“$?`
`echo“-n $“”Reloading $prog: “`
`killproc $nginx -HUP`
`RETVAL=$?`
`echo`
`}`
`force_reload() {`
`restart`
`}`
`configtest() {`
`$nginx -t -c $NGINX_CONF_FILE`
`}`
`rh_status() {`
`status $prog`
`}`
`rh_status_q() {`
`rh_status >“/dev/null2“>&1`
`}`
`case“”$1″“in`
`start)`
`rh_status_q && exit0`
`$1`
`;;`
`stop)`
`rh_status_q || exit0`
`$1`
`;;`
`restart|configtest)`
`$1`
`;;`
`reload)`
`rh_status_q || exit7`
`$1`
`;;`
`force-reload)`
`force_reload`
`;;`
`status)`
`rh_status`
`;;`
`condrestart|try-restart)`
`rh_status_q || exit0`
`;;`
`*)`
`echo“$“”Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}”`
`exit2`
`esac`
`[root@jie1 nginx-1.4.2]“# chmod +x /etc/rc.d/init.d/nginx`
`[root@jie1 nginx-1.4.2]“# service nginx start`
`Starting nginx: [ OK ]`
`[root@jie1 nginx-1.4.2]“# scp -p /etc/rc.d/init.d/nginx 172.16.22.2:/etc/rc.d/init.d #把nginx的服务脚本复制到backup上，-p是保持原有的权限`
|
**
3、修改配置文件**
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
`[root@jie1 ~]“# cd /etc/keepalived/`
`[root@jie1 keepalived]“# vim keepalived.conf`
`global_defs {`
` “notification_email {`
` “root@localhost`
` “}`
` “notification_email_from admin@localhost`
` “smtp_server 127.0.0.1`
` “smtp_connect_timeout 30`
` “router_id LTT`
`}`
`vrrp_script chk_nginx { “#检测nginx服务是否在运行有很多方式，比如进程，用脚本检测等等`
` “script “”killall -0 nginx”` `#用shell命令检查nginx服务是否存在`
` “interval 1 “#时间间隔为1秒检测一次`
` “weight -2 “#当nginx的服务不存在了，就把当前的权重-2`
` “fall 2 “#测试失败的次数`
` “rise 1 “#测试成功的次数`
`}`
`vrrp_instance IN_1 {`
` “state MASTER`
` “interface eth0`
` “virtual_router_id 22`
` “priority 100`
` “advert_int 1`
` “authentication {`
` “auth_type PASS`
` “auth_pass aaaa`
` “}`
` “virtual_ipaddress {`
` “172.16.22.100`
` “}`
` “track_script {`
` “chk_nginx “#引用上面的vrrp_script定义的脚本名称`
`}`
`}`
`[root@jie1 keepalived]“#scp keepalived.conf 172.16.22.2:/etc/keepalived #把配置文件copy到Backup服务器上，copy之前要保证Backup服务器上面已经安装了keepalived`
|
**
4、开启keepalived和nginx的服务**
|
1
2
3
4
5
6
7
8
|
`[root@jie1 keepalived]“# service keepalived start`
`Starting keepalived: [ OK ]`
`[root@jie1 keepalived]“# chkconfig –add keepalived`
`[root@jie1 keepalived]“# chkconfig keepalived on`
`[root@jie1 ~]“# service nginx start`
`Starting nginx: [ OK ]`
`[root@jie1 ~]“# chkconfig –add nginx`
`[root@jie1 ~]“# chkconfig nginx on`
|
**
Backup：**
1、安装keepalived和编译安装nginx
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
`[root@jie2 ~]“# yum -y install keepalived`
`[root@jie2 ~]“#tar xf nginx-1.4.2.tar.gz`
`[root@jie2 ~]“#yum -y groupinstall “Development tools” “Server Platform Development”`
`[root@jie2 ~]“#yum -y install pcre-devel`
`[root@jie2 ~]“# cd nginx-1.4.2`
`[root@jie2 nginx-1.4.2]“# groupadd nginx`
`[root@jie2 nginx-1.4.2]“# useradd -r -g nginx nginx`
`[root@jie2 nginx-1.4.2]“#./configure \`
` “–prefix=“/usr` `\`
` “–sbin-path=“/usr/sbin/nginx` `\`
` “–conf-path=“/etc/nginx/nginx“.conf \`
` “–error-log-path=“/var/log/nginx/error“.log \`
` “–http-log-path=“/var/log/nginx/access“.log \`
` “–pid-path=“/var/run/nginx/nginx“.pid \`
` “–lock-path=“/var/lock/nginx“.lock \`
` “–user=nginx \`
` “–group=nginx \`
` “–with-http_ssl_module \`
` “–with-http_flv_module \`
` “–with-http_stub_status_module \`
` “–with-http_gzip_static_module \`
` “–http-client-body-temp-path=“/var/tmp/nginx/client/` `\`
` “–http-proxy-temp-path=“/var/tmp/nginx/proxy/` `\`
` “–http-fastcgi-temp-path=“/var/tmp/nginx/fcgi/` `\`
` “–http-uwsgi-temp-path=“/var/tmp/nginx/uwsgi` `\`
` “–http-scgi-temp-path=“/var/tmp/nginx/scgi` `\`
` “–with-pcre`
`[root@jie2 nginx-1.4.2]“# make && make install`
|
**
2、之前 已经从Master复制了nginx的system V服务脚本文件，启动nginx****
服务**
|
1
2
3
4
|
`[root@jie2 ~]“# service nginx start`
`Starting nginx: [ OK ]`
`[root@jie2 ~]“# chkconfig –add nginx`
`[root@jie2 ~]“# chkconfig nginx on`
|
**
3、修改配置文件**
|
1
2
3
4
|
`[root@jie2 ~]“# cd /etc/keepalived/`
`[root@jie2 keepalived]“# vim keepalived.conf #此配置文件是从Master服务器上copy过来，只需小小改动`
`state BACKUP “#把这里原先的MASTER改成BACKUP`
`priority 99 “#把这里原先的100改成99`
|
**
4、开启服务**
|
1
2
3
4
|
`[root@jie2 keepalived]“# service keepalived start`
`Starting keepalived: [ OK ]`
`[root@jie2 keepalived]“# chkconfig –add keepalived`
`[root@jie2 keepalived]“# chkconfig keepalived on`
|
**
apache1：**
1、安装(博主这里用rpm包安装，各位朋友可以用源码编译安装)
|
1
|
`[root@jie3 ~]# yum -y install httpd`
|
**
2、建立测试网页文件**
|
1
2
3
|
`[root@jie3 ~]# cd /“var“/www/html/`
`[root@jie3 html]# cat index.html #建一个测试网页`
`
“this` `is` `apache1` |
**
3、开启服务**
|
1
2
3
4
|
`[root@jie3 html]# service httpd start`
`Starting httpd: [ OK ]`
`[root@jie3 html]# chkconfig –add httpd`
`[root@jie3 html]# chkconfig httpd on`
|
**
apache2：**
1、安装(博主这里用rpm包安装，各位朋友可以用源码编译安装)
|
1
|
`[root@jie4 ~]# yum -y install httpd`
|
**
2、建立测试网页文件**
|
1
2
3
|
`[root@jie4 ~]# cd /“var“/www/html/`
`[root@jie4 html]# cat index.html #建一个测试网页`
`
“this` `is` `apache2` |
**
3、开启服务**
|
1
2
3
4
|
`[root@jie4 html]# service httpd start`
`Starting httpd: [ OK ]`
`[root@jie4 html]# chkconfig –add httpd`
`[root@jie4 html]# chkconfig httpd on`
|
**
此致所有安装已经完成。**
二、nginx实现后端realserver的负载均衡，由于两边的配置文件必须保持一致，所以在Master配置完后直接copy到Backup上**
Master：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
|
`[root@jie1 ~]“# cd /etc/nginx/`
`[root@jie1 nginx]“# grep -v “#” nginx.conf | grep -v “^$”`
`worker_processes 1;`
`events {`
` “worker_connections 1024;`
`}`
`http {`
` “include mime.types;`
` “default_type application“/octet-stream“;`
` “sendfile on;`
` “keepalive_timeout 65;`
` “upstream apacheweb { “#定义负载均衡的模块`
` “server 172.16.22.3:80 max_fails=3 fail_timeout=2s;`
` “server 172.16.22.4:80 max_fails=3 fail_timeout=2s;`
` “}`
` “server {`
` “listen 80;`
` “server_name localhost;`
` “location / {`
` “root html;`
` “index index.html index.htm;`
` “}`
` “error_page 500 502 503 504 “/50x“.html;`
` “location = “/50x“.html {`
` “root html;`
` “}`
` “location ~ \.(html|htm|gif|jpg|jpeg|bmp|png|ico|txt|js|css)$ {`
` “root “/var/www/html“; “#此处定义后端服务器网页存放路径`
` “proxy_pass http:“//apacheweb“;`
` “}`
` “}`
`}`
`[root@jie1 nginx]“# scp nginx.conf 172.16.22.2:/etc/nginx`
|
**
两边分别重启服务**
|
1
2
3
4
5
6
7
8
9
10
|
`[root@jie2 nginx]“# service nginx restart`
`nginx: the configuration “file` `/etc/nginx/nginx“.conf syntax is ok`
`nginx: configuration “file` `/etc/nginx/nginx“.conf “test` `is successful`
`Stopping nginx: [ OK ]`
`Starting nginx: [ OK ]`
`[root@jie2 nginx]“# service nginx restart`
`nginx: the configuration “file` `/etc/nginx/nginx“.conf syntax is ok`
`nginx: configuration “file` `/etc/nginx/nginx“.conf “test` `is successful`
`Stopping nginx: [ OK ]`
`Starting nginx: [ OK ]`
|
**
验证负载均衡：**
本博客只是简单的用nginx做了方向代理和静态页面的负载均衡，keepalived+nginx实现高可用的nginx的动静分离，读写分离，后续会持续更新
版权声明：本文内容由互联网用户自发贡献，该文观点仅代表作者本人。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容， 请发送邮件至 举报，一经查实，本站将立刻删除。
文章由极客之音整理，本文链接：https://www.bmabk.com/index.php/post/160536.html
