Puppet结合Nginx多端口实现负载均衡

发布时间:2019-11-12编辑:脚本学堂
当服务器数量不断增加,一台puppetmaster压力大,解析缓慢,而且时不时出现“time out”之类的报错。在Puppet官网上找寻解决方案,发现puppetmaster可以配置多端口,结合WEB代理(推荐Nginx),这样puppetmaster承受能力至少可以提升数倍以上,相当于在很大程度上优化了puppet的处理能力。

1,遵循前面的环境设定,我们这里的服务器环境及软件版本分别为:
服务器系统:centos5.8 x86_64
Ruby版本:ruby-1.8.5
Puppet版本:puppet-2.7.9
nginx版本:nginx-0.8.46

2,Mongrel安装
要使用puppet多端口配置,需要指定mongrel类型,默认没有安装,需要安装:
 

yum install -y rubygem-mongrel

3,配置puppetmaster
在/etc/sysconfig/puppetmaster文件末尾添加如下两行,分别代表多端口、mongrel类型,内容如下所示:
 

PUPPETMASTER_PORTS=(8141 8142 8143 8144 8145)
PUPPETMASTER_EXTRA_OPTS="--servertype=mongrel --ssl_client_header=HTTP_X_SSL_SUBJECT"

4,安装Nginx服务
安装之前请确保系统已经安装pcre-devel正则库,然后再编译安装Nginx,需要添加SSL模块参数支持,Nginx的安装过程如下所示:
 

复制代码 代码示例:
yum -y install pcre-devel
cd /usr/local/src
wget http://nginx.org/download/nginx-0.8.46.tar.gz
tar zxvf nginx-0.8.46.tar.gz
cd nginx-0.8.46
./configure --prefix=/usr/local/nginx --with-http_ssl_module
make && make install && cd ../
 

添加www用户组及用户,命令如下所示:
 

groupadd www
useradd -g www www

5,我们依据puppet需求来修改配置文件nginx.conf,内容如下所示:
 

复制代码 代码示例:

user www;
worker_processes    8;
events {
        worker_connections    65535;
}

http {
        include             mime.types;
        default_type    application/octet-stream;

        sendfile                on;
        tcp_nopush         on;

        keepalive_timeout    65;
        #定义puppet客户端访问puppet-server端日志格式
        log_format main '$remote_addr - $remote_user [$time_local] "$request" $request_length $request_time $time_local'
                                                '$status $body_bytes_sent $bytes_sent $connection $msec "$http_referer" '
                                                '"$http_user_agent" $http_x_forwarded_for $upstream_response_time $upstream_addr $upstream_status ';
        access_log    /usr/local/nginx/logs/access.log    main;

        upstream puppetmaster {
                server 127.0.0.1:8141;
                server 127.0.0.1:8142;
                server 127.0.0.1:8143;
                server 127.0.0.1:8144;
                server 127.0.0.1:8145;
                }

        server {
        listen 8140;
        root /etc/puppet;
        ssl on;
        ssl_session_timeout 5m;
        #如下为puppetmaster服务器端证书地址   
        ssl_certificate /var/lib/puppet/ssl/certs/server.cn7788.com.pem;
        ssl_certificate_key /var/lib/puppet/ssl/private_keys/server.cn7788.com.pem;
        ssl_client_certificate /var/lib/puppet/ssl/ca/ca_crt.pem;
        ssl_crl /var/lib/puppet/ssl/ca/ca_crl.pem;
        ssl_verify_client optional;
        #File sections
        location /production/file_content/files/ {
        types { }
        default_type application/x-raw;
        #定义puppet推送路径别名
        alias /etc/puppet/files/;
        }
        # Modules files sections
        location ~ /production/file_content/modules/.+/ {
        root /etc/puppet/modules;
        types { }
        default_type application/x-raw;
        rewrite ^/production/file_content/modules/(.+)/(.+)$ /$1/files/$2 break;
        }
        location / {
        ##设置跳转到puppetmaster负载均衡
        proxy_pass http://puppetmaster;
        proxy_redirect off;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Client-Verify $ssl_client_verify;
        proxy_set_header X-SSL-Subject $ssl_client_s_dn;
        proxy_set_header X-SSL-Issuer $ssl_client_i_dn;
        proxy_buffer_size 10m;
        proxy_buffers 1024 10m;
        proxy_busy_buffers_size 10m;
        proxy_temp_file_write_size 10m;
        proxy_read_timeout 120;
     }
}
}

6,修改完nginx.conf文件以后,要启动nginx及puppet-server,这时应该如何操作呢?
1.首先关闭puppetmaster进程,然后先启动nginx,不然nginx是会启动失败的,命令如下所示:
 

/usr/local/nginx/sbin/nginx
 

nginx占用puppetmaster默认的8140端口后,我们可以用如下命令来检查8140端口是否被nginx接管,如下所示:
lsof -i:8140
此命令显示结果表明8140被nginx进程接管,如下所示:
 

COMMAND    PID    USER     FD     TYPE DEVICE SIZE/OFF NODE NAME
nginx     4121    root        6u    IPv4    20668            0t0    TCP *:8140 (LISTEN)
nginx     4122    www    6u    IPv4    20668            0t0    TCP *:8140 (LISTEN)

再启动puppetmaster,命令如下所示:
service puppetmaster start
如果ruby版本为1.8.5的话,等会运行puppetmaster会有如下警告,如下所示:
 

Starting puppetmaster:
Port: 8141** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
                                                                                                                                                                                                                                         [        OK        ]
Port: 8142** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
                                                                                                                                                                                                                                         [        OK        ]
Port: 8143** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
                                                                                                                                                                                                                                         [        OK        ]
Port: 8144** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
                                                                                                                                                                                                                                         [        OK        ]
Port: 8145** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
                                                                                                                                                                                                                                         [        OK        ]
 

这段警告值的意思为:
It's just a warning. Mongrel wants a Ruby version of at least 1.8.6.
But it still runs just fine with previous versions. Just ignore the warning.

中文意思:
Mongrel需要ruby至少是1.8.6以上的版本,但它仍然在当前版本运行,请忽略当前警告,为保证整个puppet运行环境的稳定,这里选择沿用1.8.5版本的ruby。