topic How to Parsing Apache Access Log ? in Getting Data In

How to Parsing Apache Access Log ?

shoautorola — Thu, 24 Jan 2013 14:04:06 GMT

Hi,

I have apache access log with this pattern:

%h %t '%r' '%q' %s %b %D %S %U %v %{User-Agent}i

{text:ip} [{date:Date,dd/MMM/yyyy:HH:mm:ss z}] '{string:HTTP Request Method' '{string:Query String}' {number:Response Code} {string:Bytes Sent} {number:Request Processing Time} {string:User Session ID} {string:Requested URL Path} {string:Local Server Name} {string:User Agent}

i'm struggling to parse the data using regular expression and the overall process how to parse the file.

Here is an example data from the log:
88.117.159.10 [22/Jan/2013:10:57:21 +0100] 'GET /dealers/actions.do HTTP/1.1' '' 200 69068 56 64EB37C2102324BD81E0E0B93243C2232 /dealers/actions.do www.simpledns.com Mozilla/5.0 (Windows NT 6.1; WOW64; rv:18.0) Gecko/20100101 Firefox/18.0

any kind of help will be appreciated.

Re: How to Parsing Apache Access Log ?

sdaniels — Thu, 24 Jan 2013 14:18:11 GMT

Have you tried using the default sourcetypes for access logs? If you assign the sourcetype of access_combined for example Splunk should automatically create the fields for you. Just change the sourcetype in your inputs.conf setting for that file monitor and restart splunk.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Listofpretrainedsourcetypes

In case you run into any issues there are a lot of other postings you can look at here to see if one can help you as well:

http://splunk-base.splunk.com/search/?q=apache&Submit=Search

Re: How to Parsing Apache Access Log ?

shoautorola — Thu, 24 Jan 2013 14:35:51 GMT

it can't parse when i choose existing source type access_combined except the timestamp. It can detect the time stamp only.

Re: How to Parsing Apache Access Log ?

shoautorola — Thu, 24 Jan 2013 14:40:35 GMT

it's much easier to parse custom logs in XpoLog Center using simple wizard. Why not Splunk provide that? I'm evaluation splunk as one of our potential log management system to adopt in our company but seems it lack of some functionality that can make your life easier.

Re: How to Parsing Apache Access Log ?

yannK — Thu, 24 Jan 2013 15:26:14 GMT

Create a specific sourcetype for your data, and define a search time field definition.
see http://docs.splunk.com/Documentation/Splunk/5.0.1/Admin/Transformsconf

example:

in props.conf

[myapache] REPORT-extract_myapache= extract_myapache
and in transforms.conf
[extract_myapache] DELIMS = " " FIELDS = "field1", "field2", "field3"