Splunk Search

help with apache access searching

splunkmeuser
New Member

LogFormat "%h %l %u %t %P \"%r\" %>s %X %b %I %O %D \"%{Referer}i\" \"%{User-Agent}i\" \"%{Host}i\" \"%{X-Forwarded-For}i\" \"%{X-Cluster-Client-IP}i\" \"%{True-Client-IP}i\" \"%{Via}i\" \"%{Akamai-Origin-Hop}i\"" combined

what does the above translate to?

my attempt was (which i'm sure is very wrong):

^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[nspaces:processid]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:connectionstatus]]\s++[[nspaces:bytes_received]]\s++[[nspaces:bytes_sent]]\s++[[nspaces:timeus]]\s++[[qstring:referrer]]\s++[[qstring:useragent]]\s++[[qstring:hservername]]\s++[[qstring:xforwardedfor]]\s++[[qstring:xclusterclientip]]\s++[[qstring:trueclientip]]\s++[[qstring:via]]\s++[[qstring:akamaiorigin]]

Tags (2)
0 Karma

grijhwani
Motivator

What are you trying to achieve? With a Splunk search you can simply search on field names as parameters (provided they are appropriately detected at index time, or you have defined a field extractor interactively). I don't really understand what you mean by the use of the phrase "translates to".

A typical search would be:

index=weblogs clientip="75.41.6.*" status!=200 method=GET

Nothing as complex as your regex.

0 Karma

splunkmeuser
New Member

im pretty sure my extractor (everything i posted in my original post) is not accurate. so i'm hoping you can provide the right regex/extractor that would solve my problem based on the log samples i provided. any ideas?

0 Karma

grijhwani
Motivator

I see 20 fields in your example data and logformat definition, but only 19 in the extractor.

0 Karma

splunkmeuser
New Member

Here are two lines from my logs:

10.50.1.1 - - [06/Aug/2013:12:20:07 -0400] 19537 "GET /fetch/ext/load.js HTTP/1.1" 200 + 5149 365 5310 4011 "http://hs.garden.com/forum/load/appl/msg116.html" "Mozilla/5.0 (iPad; CPU OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B329 Safari/8536.25" "ss.lototo.com" "-" "-" "-" "-" "-"

10.75.12.9 - - [06/Aug/2013:12:20:07 -0400] 19537 "GET /request/page/xml?path=%2Fcharlie-hunnam%2F1-k-42836&site==entertainment=0&is_xfinity= HTTP/1.1" 200 + 14891 414 15057 97443 "-" "-" "ss.lototo.com" "-" "-" "-" "-" "-"

Please advise.

0 Karma

grijhwani
Motivator

Rather than having to pore through the Apache logformat page to dissect your format string, it would be easier if you were to include a sample log line (suitably obsfuscated if need be provided you leave the general structure intact).

0 Karma

splunkmeuser
New Member

this is from the field extraction. i need to be able to make splunk recognize the custom format of my apache logs so that i can accurately get values from specific fields. this is needed because i need to be generating reports on the values of those fields. any help will be appreciated!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...