CentOS 5.9 下nginx proxy_store的使用

相信你会和我一样想过,为什么要用cache呢?为什么不直接把源服务器的数据直接rsync推到其他服务器呢?如果你现在还这么想,推荐你看看《CDN技术详解》。不过如果真要保证所有下载服务器数据完全一致,不一定要用rsync同步的方式,rsync+proxy_store 完全镜像会是一个更好的方式。

一:使用nginx做缓存服务器

1 我的需求还是缓存android的软件包,后缀名是apk。话不多说,直接上配置,供参考:

a-->nginx.conf
user  www www;
worker_processes 8;
error_log  /data/logs/nginx_error.log  crit;
pid        /usr/local/nginx/nginx.pid;
worker_rlimit_nofile 204800;

events
{
  use epoll;
  worker_connections 204800;
}

http
{
  include       mime.types;
  #default_type  application/octet-stream;
  default_type  application/vnd.android.package-archive;

  charset  utf-8;

  server_names_hash_bucket_size 128;
  client_header_buffer_size 2k;
  large_client_header_buffers 4 4k;
  client_max_body_size 8m;

  sendfile on;
  tcp_nopush     on;

  keepalive_timeout 60;
  open_file_cache max=204800 inactive=20s;
  open_file_cache_min_uses 1;
  open_file_cache_valid 30s;


 tcp_nodelay on;
 client_body_buffer_size 512k;
 proxy_connect_timeout 5;
 proxy_read_timeout 60;
 proxy_send_timeout 5;
 proxy_buffer_size 16k;
 proxy_buffers 4 64k;
 proxy_busy_buffers_size 128k;
 proxy_temp_file_write_size 128k;

 gzip on;
 gzip_proxied expired no-cache no-store private auth;
 gzip_min_length  1k;
 gzip_buffers     4 16k;
 gzip_http_version 1.1;
 gzip_comp_level 3;
 gzip_types       text/plain application/x-javascript text/css application/xml;
 gzip_disable  "MSIE [1-6]\.";
 gzip_vary on;

log_format access '$remote_addr - $remote_user [$time_local]  '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent" '
                    '$host $request_time $http_x_forwarded_for';
#access_log /data/logs/http.a.log;
#error_log  /data/logs/http.e.log;

upstream source_site {
 server 192.168.1.163:88 max_fails=2 fail_timeout=30s;
 server 192.168.1.167:88 backup;
}

include vhosts/apk_store.peiqiang.net.conf;
}

b-->apk_store.peiqiang.net.conf
server {
      listen 80;
      server_name  apk.peiqiang.net;
      access_log  /data/logs/apk.a.log;
      error_log   /data/logs/apk.e.log notice;
      rewrite_log off;

      # PHP Scripts is NOT allowed within this site!
      location ~* \.(php|php5|jsp|asp|aspx)$ {
          deny all;
      }

      location /{
          root /data/mumayi/soft;
          error_page 404 = @fetch;
      }

      location @fetch {
          internal;

          proxy_set_header Host $http_host;
          proxy_set_header Accept-Encoding "";
          proxy_pass http://source_site;
          proxy_store on;
          proxy_store_access user:rw group:rw all:r;
          proxy_temp_path /data2/temp;
          root /data/mumayi/soft;
       }

}

c-->/etc/rc.local
#!/bin/sh
#
# This script will be executed *after* all the other init scripts.
# You can put your own initialization stuff in here if you don't
# want to do the full Sys V style init stuff.

touch /var/lock/subsys/local
ulimit -HSn 65535
/usr/local/nginx/sbin/nginx 


说明:
这种完全镜像的方式,在我们线上也有用到,用到的场景如下:
A 源站 server_1
B 中间层(proxy_store) server_2
C cache层 server_3

所有cache回源地址为server_2,server_2回源地址为server_1。这样想当于为源站server_1加个防护层,防止所有cache直接向源站server_1回源,导致源站带宽跑满被打死;同时也相当于对源站的数据做了一个完全备份。当然,为保证A和B数据完全一致,我还写一个rsync同步脚本,每天晚上定时将源站 server_1的数据推到中间层 server_2。

二:是时候结束了

自言自语:

完全镜像源站的方式,仔细想想,相信你一定会有用到他的地方。额,发现头有些晕了,今天这是第三篇博客了,脑细胞有些严重跟不上了,难道已经老了吗...