topic Re: help with apache access searching in Splunk Search

help with apache access searching

splunkmeuser — Mon, 28 Sep 2020 14:30:16 GMT

LogFormat "%h %l %u %t %P \"%r\" %>s %X %b %I %O %D \"%{Referer}i\" \"%{User-Agent}i\" \"%{Host}i\" \"%{X-Forwarded-For}i\" \"%{X-Cluster-Client-IP}i\" \"%{True-Client-IP}i\" \"%{Via}i\" \"%{Akamai-Origin-Hop}i\"" combined

what does the above translate to?

my attempt was (which i'm sure is very wrong):

^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[nspaces:processid]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:connectionstatus]]\s++[[nspaces:bytes_received]]\s++[[nspaces:bytes_sent]]\s++[[nspaces:timeus]]\s++[[qstring:referrer]]\s++[[qstring:useragent]]\s++[[qstring:hservername]]\s++[[qstring:xforwardedfor]]\s++[[qstring:xclusterclientip]]\s++[[qstring:trueclientip]]\s++[[qstring:via]]\s++[[qstring:akamaiorigin]]

Re: help with apache access searching

grijhwani — Mon, 05 Aug 2013 20:07:58 GMT

What are you trying to achieve? With a Splunk search you can simply search on field names as parameters (provided they are appropriately detected at index time, or you have defined a field extractor interactively). I don't really understand what you mean by the use of the phrase "translates to".

A typical search would be:

index=weblogs clientip="75.41.6.*" status!=200 method=GET

Nothing as complex as your regex.

Re: help with apache access searching

splunkmeuser — Mon, 05 Aug 2013 21:40:29 GMT

this is from the field extraction. i need to be able to make splunk recognize the custom format of my apache logs so that i can accurately get values from specific fields. this is needed because i need to be generating reports on the values of those fields. any help will be appreciated!

Re: help with apache access searching

grijhwani — Tue, 06 Aug 2013 01:04:01 GMT

Rather than having to pore through the Apache logformat page to dissect your format string, it would be easier if you were to include a sample log line (suitably obsfuscated if need be provided you leave the general structure intact).

Re: help with apache access searching

splunkmeuser — Mon, 28 Sep 2020 14:30:56 GMT

Here are two lines from my logs:

10.50.1.1 - - [06/Aug/2013:12:20:07 -0400] 19537 "GET /fetch/ext/load.js HTTP/1.1" 200 + 5149 365 5310 4011 "http://hs.garden.com/forum/load/appl/msg116.html" "Mozilla/5.0 (iPad; CPU OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B329 Safari/8536.25" "ss.lototo.com" "-" "-" "-" "-" "-"

10.75.12.9 - - [06/Aug/2013:12:20:07 -0400] 19537 "GET /request/page/xml?path=%2Fcharlie-hunnam%2F1-k-42836&site==entertainment=0&is_xfinity= HTTP/1.1" 200 + 14891 414 15057 97443 "-" "-" "ss.lototo.com" "-" "-" "-" "-" "-"

Please advise.

Re: help with apache access searching

grijhwani — Wed, 07 Aug 2013 03:18:53 GMT

I see 20 fields in your example data and logformat definition, but only 19 in the extractor.

Re: help with apache access searching

splunkmeuser — Wed, 07 Aug 2013 13:11:46 GMT

im pretty sure my extractor (everything i posted in my original post) is not accurate. so i'm hoping you can provide the right regex/extractor that would solve my problem based on the log samples i provided. any ideas?