Splunk Search

help with apache access searching

splunkmeuser
New Member

LogFormat "%h %l %u %t %P \"%r\" %>s %X %b %I %O %D \"%{Referer}i\" \"%{User-Agent}i\" \"%{Host}i\" \"%{X-Forwarded-For}i\" \"%{X-Cluster-Client-IP}i\" \"%{True-Client-IP}i\" \"%{Via}i\" \"%{Akamai-Origin-Hop}i\"" combined

what does the above translate to?

my attempt was (which i'm sure is very wrong):

^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[nspaces:processid]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:connectionstatus]]\s++[[nspaces:bytes_received]]\s++[[nspaces:bytes_sent]]\s++[[nspaces:timeus]]\s++[[qstring:referrer]]\s++[[qstring:useragent]]\s++[[qstring:hservername]]\s++[[qstring:xforwardedfor]]\s++[[qstring:xclusterclientip]]\s++[[qstring:trueclientip]]\s++[[qstring:via]]\s++[[qstring:akamaiorigin]]

Tags (2)
0 Karma

grijhwani
Motivator

What are you trying to achieve? With a Splunk search you can simply search on field names as parameters (provided they are appropriately detected at index time, or you have defined a field extractor interactively). I don't really understand what you mean by the use of the phrase "translates to".

A typical search would be:

index=weblogs clientip="75.41.6.*" status!=200 method=GET

Nothing as complex as your regex.

0 Karma

splunkmeuser
New Member

im pretty sure my extractor (everything i posted in my original post) is not accurate. so i'm hoping you can provide the right regex/extractor that would solve my problem based on the log samples i provided. any ideas?

0 Karma

grijhwani
Motivator

I see 20 fields in your example data and logformat definition, but only 19 in the extractor.

0 Karma

splunkmeuser
New Member

Here are two lines from my logs:

10.50.1.1 - - [06/Aug/2013:12:20:07 -0400] 19537 "GET /fetch/ext/load.js HTTP/1.1" 200 + 5149 365 5310 4011 "http://hs.garden.com/forum/load/appl/msg116.html" "Mozilla/5.0 (iPad; CPU OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B329 Safari/8536.25" "ss.lototo.com" "-" "-" "-" "-" "-"

10.75.12.9 - - [06/Aug/2013:12:20:07 -0400] 19537 "GET /request/page/xml?path=%2Fcharlie-hunnam%2F1-k-42836&site==entertainment=0&is_xfinity= HTTP/1.1" 200 + 14891 414 15057 97443 "-" "-" "ss.lototo.com" "-" "-" "-" "-" "-"

Please advise.

0 Karma

grijhwani
Motivator

Rather than having to pore through the Apache logformat page to dissect your format string, it would be easier if you were to include a sample log line (suitably obsfuscated if need be provided you leave the general structure intact).

0 Karma

splunkmeuser
New Member

this is from the field extraction. i need to be able to make splunk recognize the custom format of my apache logs so that i can accurately get values from specific fields. this is needed because i need to be generating reports on the values of those fields. any help will be appreciated!

0 Karma
Get Updates on the Splunk Community!

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...

Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0

Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...