Home Ubuntu Tips Tricks Simple Bash Script to Parse Apache Access log Entries to Find top User Agents, top Urls and top IPs

Simple Bash Script to Parse Apache Access log Entries to Find top User Agents, top Urls and top IPs

0
Simple Bash Script to Parse Apache Access log Entries to Find top User Agents, top Urls and top IPs
Source: fiverr.com

If you have a website running on a LAMP server and receive reasonable amounts of traffic you would run into the need to know some basic statistics like most frequently accessed URL, most frequent visitor IP, most frequent visitor user agent etc. This is especially true if you have have troubles with bots or spammers. The following is a simple bash script to generate the above three pieces of statisics from the apache access log file

#!/bin/bash
# Usage
# ls-httpd type count
# Eg: 
# ls-httpd url 1000
# will find top URLs in the last 1000 access log entries
# ls-httpd ip 1000
# will find top IPs in the last 1000 access log entries
# ls-httpd agent 1000
# will find top user agents in the last 1000 access log entries

type=$1
length=$2

if [ "$3" == "" ]; then
  log_file="/var/log/httpd/example.com-access_log"
else
  log_file="$3"
fi

if [ "$type" = "ip" ]; then
  tail -n $length $log_file | grep -o "[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}" | sort -n | uniq -c | sort -n
elif [ "$type" = "agent" ]; then
  tail -n $length $log_file | awk -F\" '{print $6}'| sort -n | uniq -c | sort -n
elif [ "$type" = "url" ]; then
  tail -n $length $log_file | awk -F\" '{print $2}'| sort -n | uniq -c | sort -n
fi