apache traffic server 日志部分学习小结

前言:

系统环境:
CentOS 6.5 x86_64

ats版本:
[root@mysql ~]# yum --enablerepo=epel install traffic server -y
root@mysql ~]# rpm -qa|grep trafficserver
trafficserver-5.3.0-1.el6.x86_64

一:默认日志格式

(1). 没有自定义格式时,由于ats使用默认的二进制编码记录日志,因此只能使用traffic_logcat来查看日志,如下:

相当于cat,查看所有的日志:
[root@mysql trafficserver]# traffic_logcat /var/log/trafficserver/squid.blog

在/etc/trafficserver/ip_allow.config定义好允许PURGE的IP后
[root@mysql trafficserver]# curl -X PURGE -v http://apke.mumayi.com/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk
* About to connect() to apke.mumayi.com port 80 (#0)
*   Trying 61.135.153.49... connected
* Connected to apke.mumayi.com (61.135.153.49) port 80 (#0)
> PURGE /2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: apke.mumayi.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 04 Mar 2016 06:42:38 GMT
< Connection: keep-alive
< Via: http/1.1 buildhw-08.phx2.fedoraproject.org (ApacheTrafficServer/5.3.0)
< Server: ATS/5.3.0
< Content-Length: 0
<
* Connection #0 to host apke.mumayi.com left intact
* Closing connection #0
[root@mysql trafficserver]# wget -S http://apke.mumayi.com/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk
--2016-03-04 14:42:55--  http://apke.mumayi.com/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk
Resolving apke.mumayi.com... 61.135.153.49
Connecting to apke.mumayi.com|61.135.153.49|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.0 200 OK
  Server: ATS/5.3.0
  Date: Fri, 04 Mar 2016 06:42:54 GMT
  Content-Type: application/vnd.android.package-archive
  Content-Length: 8550063
  Last-Modified: Mon, 29 Feb 2016 12:04:47 GMT
  ETag: "56d433df-8276af"
  Expires: Fri, 11 Mar 2016 06:42:54 GMT
  Cache-Control: max-age=604800
  Accept-Ranges: bytes
  Age: 1
  Connection: keep-alive
  Via: http/1.1 buildhw-08.phx2.fedoraproject.org (ApacheTrafficServer/5.3.0)
Length: 8550063 (8.2M) [application/vnd.android.package-archive]
Saving to: “lanrentingshu_V5.5.1_mumayi_30e3a.apk”

100%[======================================================================================================================================================================================================================>] 8,550,063    816K/s   in 11s

2016-03-04 14:43:06 (748 KB/s) - “lanrentingshu_V5.5.1_mumayi_30e3a.apk” saved [8550063/8550063]

[root@mysql trafficserver]# wget -S http://apke.mumayi.com/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk
--2016-03-04 14:43:07--  http://apke.mumayi.com/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk
Resolving apke.mumayi.com... 61.135.153.49
Connecting to apke.mumayi.com|61.135.153.49|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.0 200 OK
  Server: ATS/5.3.0
  Date: Fri, 04 Mar 2016 06:42:54 GMT
  Content-Type: application/vnd.android.package-archive
  Content-Length: 8550063
  Last-Modified: Mon, 29 Feb 2016 12:04:47 GMT
  ETag: "56d433df-8276af"
  Expires: Fri, 11 Mar 2016 06:42:54 GMT
  Cache-Control: max-age=604800
  Accept-Ranges: bytes
  Age: 13
  Connection: keep-alive
  Via: http/1.1 buildhw-08.phx2.fedoraproject.org (ApacheTrafficServer/5.3.0)
Length: 8550063 (8.2M) [application/vnd.android.package-archive]
Saving to: “lanrentingshu_V5.5.1_mumayi_30e3a.apk.1”

100%[======================================================================================================================================================================================================================>] 8,550,063   --.-K/s   in 0.06s

2016-03-04 14:43:07 (147 MB/s) - “lanrentingshu_V5.5.1_mumayi_30e3a.apk.1” saved [8550063/8550063]

[root@mysql trafficserver]#

相当于tail -f,可以看到http://apke.mumayi.com/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk这个资源,被PURGE后,先miss,接着缓存后,就hit了
[root@mysql trafficserver]# traffic_logcat -f /var/log/trafficserver/squid.blog
1457073722.657 0 183.128.112.85 ERR_CONNECT_FAIL/404 588 GET http://61.135.153.49/31.mp3 - NONE/- text/html
1457073758.531 2 61.135.153.49 TCP_HIT/200 195 PURGE http://120.192.81.163/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk - NONE/- -
1457073786.707 11355 61.135.153.49 TCP_MISS/200 8550491 GET http://120.192.81.163/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk - DIRECT/120.192.81.163 application/vnd.android.package-archive
1457073787.727 73 61.135.153.49 TCP_HIT/200 8550492 GET http://120.192.81.163/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk - NONE/- application/vnd.android.package-archive

(2).更改日志目录,默认日志存放在/var/log/trafficserver:

 CONFIG proxy.config.log.logfile_dir STRING /data0/log/trafficserver

(3).更改默认日志的输出格式,按照ascii码输出日志

CONFIG proxy.config.log.squid_log_is_ascii INT 1
现在可以直接使用cat/tail来查看日志了
[root@mysql trafficserver]# tail -f squid.log
1457320854.684 101 61.135.153.49 TCP_HIT/200 8550496 GET http://120.192.81.163/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk - NONE/- application/vnd.android.package-archive
1457320856.648 74 61.135.153.49 TCP_HIT/200 8550496 GET http://120.192.81.163/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk - NONE/- application/vnd.android.package-archive
1457320857.615 78 61.135.153.49 TCP_HIT/200 8550496 GET http://120.192.81.163/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk - NONE/- application/vnd.android.package-archive
1457320861.788 78 61.135.153.49 TCP_HIT/200 8550496 GET http://120.192.81.163/2016/02/29/3/31274/lanrentingshu_V5.5.1_mumayi_30e3a.apk - NONE/- application/vnd.android.package-archive

(4).关闭squid格式的日志输出

CONFIG proxy.config.log.squid_log_enabled INT 0
再查看日志,发现已经没有输出了
[root@mysql trafficserver]# tail -f squid.log

二:使用自定义格式记录日志

(1).接着上文继续,开启自定义日志格式和设置日志如何存储

修改records.config,开启日志自定义功能
CONFIG proxy.config.log.custom_logs_enabled INT 1
修改logs_xml.config,设置如何保存日志,默认配置里已经给了很多的LogFormat/LogFilter/LogObject的参考例子,我们这里就简单的定义一个LogObject测试下
<LogObject>
    <Format = "squid"/>
    <Filename = "custom"/>
</LogObject>
查看日志输出
[root@mysql trafficserver]# tail -n 3 custom.log
1457328740.853 0 222.134.5.197 ERR_CONNECT_FAIL/404 588 GET http://61.135.153.49/31.mp3 - NONE/- text/html
1457328740.906 0 222.134.5.197 ERR_CONNECT_FAIL/404 588 GET http://61.135.153.49/31.mp3 - NONE/- text/html
1457328742.483 0 222.134.5.197 ERR_CONNECT_FAIL/404 588 GET http://61.135.153.49/31.mp3 - NONE/- text/html

这里简单的解释下logs_xml.config这个文件的组成,主要由LogFormats/LogFilters/LogObject定义块组成,每一块的含义如下:

LogFormat : object defines the content of the log file using printf-style format strings (看到 printf-style,就猜的出来,说的是按照一种什么风格输出日志,例如日志字段间的分隔用的是逗号分隔还是空格分隔了)

LogFilter :object defines a filter so that you include or exclude certain information from the log file (看到Filter也知道,这个部分是做过滤的,确定最后要输出什么样的信息到日志,例如可以只输出某一个域名的日志或者只输出状态码等于200的日志等等)

LogObject :object specifies all the information needed to produce a log file (这是一个总的定义块,相当于c语言最后的main函数,例如你要自定义输出一个日志,首先要定义一个LogObject,然后里面再包括LogFormat/LogFilter等其他部分;LogObject里,至少要包括Format(决定日志如何输出)和Filename(日志输出到哪)这两个部分。)

三:日志轮询

(1).使用traffic server内部的轮询机制来处理,我们线上的配置如下:

CONFIG proxy.config.log.logging_enabled INT 3
CONFIG proxy.config.log.max_secs_per_buffer INT 5
CONFIG proxy.config.log.max_space_mb_for_logs INT 60000
CONFIG proxy.config.log.max_space_mb_for_orphan_logs INT 25
CONFIG proxy.config.log.max_space_mb_headroom INT 5000

CONFIG proxy.config.log.rolling_enabled INT 3
CONFIG proxy.config.log.rolling_interval_sec INT 86400
CONFIG proxy.config.log.rolling_offset_hr INT 0
CONFIG proxy.config.log.rolling_size_mb INT 1024
CONFIG proxy.config.log.auto_delete_rolled_files INT 1

(2).也可以写自定脚本配合logrotate或者crontab来做


参考资料:

traffic server document:https://docs.trafficserver.apache.org/records.config#logging-configuration

traffic server document:https://docs.trafficserver.apache.org/en/4.2.x/admin/working-log-files.en.html

traffic server document:https://trafficserver.readthedocs.org/en/5.3.x/admin/event-logging-formats.en.html

自定义ATS日志输出内容:https://blog.zymlinux.net/index.php/archives/398

ATS 4.2.3自定义日志文件格式的方法:http://blog.csdn.net/tao_627/article/details/45198385

TrafficServer日志系统配置指南:http://jp.51studyit.com/article/details/76369.htm