本文主要使用的是grep,awk,cut等工具来对nginx日志进行统计和分析，具体如下：
1，列出当天访问最多次数的ip地址
cut -d- -f 1 /usr/local/nginx/logs/20160329/access_2016032913.log |uniq -c | sort -rn | head -20
[root@httpservera 20160329]# cut -d- -f 1 /usr/local/nginx/logs/20160329/access_2016032913.log |uniq -c | sort -rn | head -20
69 180.116.214.31
45 180.116.214.31
45 180.116.214.31
36 49.80.54.111
35 183.206.185.204
35 180.116.214.31
32 49.80.54.111
32 49.80.54.111
32 180.116.214.31
31 117.136.45.101
29 180.116.214.31
28 218.205.19.112
28 180.116.214.31
28 180.116.214.31
27 49.80.54.111
27 222.185.248.242
24 49.80.54.111
24 175.0.8.161
23 49.80.54.111
23 49.80.54.111
2，查看某一个页面被访问的次数
[root@httpservera 20160329]#grep "/index.php" log_file | wc -l
3，查看每一个IP访问了多少页面并排序
awk '{++S[$1]} END {for (a in S) print a,S[a]}' access_2016032913.log |uniq|sort -rn|more
[root@httpservera 20160329]# awk '{++S[$1]} END {for (a in S) print a,S[a]}' access_2016032913.log |uniq|sort -rn|more
223.94.229.51 148
223.73.166.191 1
223.68.252.103 156
223.68.167.66 2
223.68.106.138 43
223.67.99.72 7
223.67.153.173 12
223.66.93.152 15
223.66.38.31 103
223.65.191.181 1
223.65.191.135 11
223.65.190.71 13
223.65.141.78 3
223.64.63.71 31
223.64.63.229 7
223.64.62.242 59
223.64.62.23 27
223.64.62.216 1
223.64.62.160 40
223.64.61.136 28
223.64.60.80 13
223.64.60.21 12
223.64.237.37 187
223.64.209.247 2
223.64.158.4 15
其中，sort -rn 按照数字从大到小排序，uniq 将重复行去除。
4，查看某一个ip访问了那些页面：grep ^xx.xx.xx.xx log_file |awk '{print $1,$7}'
[root@httpservera 20160329]# grep ^223.147.39.194 17 access_2016032913.log |awk '{print $1,$7}' grep: 17: No such file or directory
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //thirdpartyapi/appaction/app_action/action_send_batch.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
5,去掉搜索引擎统计当天的页面：awk '{print $12,$1}' access_2016032913.log | grep ^\"Mozilla | awk '{print $2}' |sort | uniq | wc -l
[root@httpservera 20160329]# awk '{print $12,$1}' access_2016032913.log | grep ^\"Mozilla | awk '{print $2}' |sort | uniq | wc -l
35
6，查看一个小时内有多少ip访问：
[root@httpservera 20160329]# awk '{print $4,$1}' access_2016032913.log | grep 29/Mar/2016:13 | awk '{print $2}'| sort | uniq | wc -l
1926
2.访问量统计
1.根据访问IP统计UV
awk '{print $1}' access.log|sort | uniq -c |wc -l
2.统计访问URL统计PV
awk '{print $7}' access.log|wc -l
3.查询访问最频繁的URL
awk '{print $7}' access.log|sort | uniq -c |sort -n -k 1 -r|more
4.查询访问最频繁的IP
awk '{print $1}' access.log|sort | uniq -c |sort -n -k 1 -r|more
5.根据时间段统计查看日志
cat access.log| sed -n '/14\/Mar\/2015:21/,/14\/Mar\/2015:22/p'|more
备注:nginx 日志分割脚本
[root@iZ237lzm354Z logs]# vim /opt/shell/nginx_log.sh
#! /bin/bash
#Power by guojinbao
date=`date +%Y-%m-%d-%H-%M-%S`
logfile="/guojinbao/nginx/logs/access.log"
logdir=/guojinbao/nginx/logs
pid=`cat /usr/local/nginx/logs/nginx.pid`
if [ ! -d $logdir ]; then
mkdir -p $logdir
fi
/bin/mv $logfile $logdir/access_${date}.log
kill -HUP $pid
本文出自 “清风明月” 博客，请务必保留此出处http://liqingbiao.blog.51cto.com/3044896/1758009
