Splunk Search

help with apache access searching

splunkmeuser
New Member

LogFormat "%h %l %u %t %P \"%r\" %>s %X %b %I %O %D \"%{Referer}i\" \"%{User-Agent}i\" \"%{Host}i\" \"%{X-Forwarded-For}i\" \"%{X-Cluster-Client-IP}i\" \"%{True-Client-IP}i\" \"%{Via}i\" \"%{Akamai-Origin-Hop}i\"" combined

what does the above translate to?

my attempt was (which i'm sure is very wrong):

^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[nspaces:processid]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:connectionstatus]]\s++[[nspaces:bytes_received]]\s++[[nspaces:bytes_sent]]\s++[[nspaces:timeus]]\s++[[qstring:referrer]]\s++[[qstring:useragent]]\s++[[qstring:hservername]]\s++[[qstring:xforwardedfor]]\s++[[qstring:xclusterclientip]]\s++[[qstring:trueclientip]]\s++[[qstring:via]]\s++[[qstring:akamaiorigin]]

Tags (2)
0 Karma

grijhwani
Motivator

What are you trying to achieve? With a Splunk search you can simply search on field names as parameters (provided they are appropriately detected at index time, or you have defined a field extractor interactively). I don't really understand what you mean by the use of the phrase "translates to".

A typical search would be:

index=weblogs clientip="75.41.6.*" status!=200 method=GET

Nothing as complex as your regex.

0 Karma

splunkmeuser
New Member

im pretty sure my extractor (everything i posted in my original post) is not accurate. so i'm hoping you can provide the right regex/extractor that would solve my problem based on the log samples i provided. any ideas?

0 Karma

grijhwani
Motivator

I see 20 fields in your example data and logformat definition, but only 19 in the extractor.

0 Karma

splunkmeuser
New Member

Here are two lines from my logs:

10.50.1.1 - - [06/Aug/2013:12:20:07 -0400] 19537 "GET /fetch/ext/load.js HTTP/1.1" 200 + 5149 365 5310 4011 "http://hs.garden.com/forum/load/appl/msg116.html" "Mozilla/5.0 (iPad; CPU OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B329 Safari/8536.25" "ss.lototo.com" "-" "-" "-" "-" "-"

10.75.12.9 - - [06/Aug/2013:12:20:07 -0400] 19537 "GET /request/page/xml?path=%2Fcharlie-hunnam%2F1-k-42836&site==entertainment=0&is_xfinity= HTTP/1.1" 200 + 14891 414 15057 97443 "-" "-" "ss.lototo.com" "-" "-" "-" "-" "-"

Please advise.

0 Karma

grijhwani
Motivator

Rather than having to pore through the Apache logformat page to dissect your format string, it would be easier if you were to include a sample log line (suitably obsfuscated if need be provided you leave the general structure intact).

0 Karma

splunkmeuser
New Member

this is from the field extraction. i need to be able to make splunk recognize the custom format of my apache logs so that i can accurately get values from specific fields. this is needed because i need to be generating reports on the values of those fields. any help will be appreciated!

0 Karma
Get Updates on the Splunk Community!

Enhance Your Splunk App Development: New Tools & Support

UCC FrameworkAdd-on Builder has been around for quite some time. It helps build Splunk apps faster, but it ...

Prove Your Splunk Prowess at .conf25—No Prereqs Required!

Your Next Big Security Credential: No Prerequisites Needed We know you’ve got the skills, and now, earning the ...

Splunk Observability Cloud's AI Assistant in Action Series: Observability as Code

This is the sixth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...