We just installed Splunk this week (YAY!) and are trying to get our Apache logs digesting to start building dashboards. Our web store uses a modified Apache Access log format that looks like this in our www.conf:
%h %l %u %t %V \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %U %q
I know that none of the extractions provided by access_combined or apache:access (Apache Addon) seem to work because they're looking for the default format, but I'm a little confused on how to help them recognize our format. I see the extractions listed, but they don't look like regular regexes that I've seen. How would I go about changing them to match our formatting?
Appreciate any help!
I can see the default extraction in the TA is:
EXTRACT-apache_access= ^(?<src>[^ ]+)\s+(?<site>.+)\s+(?<ident>\S+)\s+(?<user>\S+)\s+(?<dest_port>\d+)\s+\[(?<timestamp>\d+/\w+/\d+:\d+:\d+:\d+\s+[-+]\d+)[^"\n]*"(?<request>[^"]+)[^ \n]*\s+\"(?<uri_query>[^ ]*)\"\s+(?<status>\d+)(?:[^ \n]* ){2}\"(?<http_referer>[^"]+)\"\s+\"(?<http_user_agent>[^"]+)\"\s+(?<request_bytes>\d+)\s+(?<response_bytes>\d+)\s+(?<response_time_microseconds>\d+)
This matches with your pattern to a certain degree, but NOT all.
(eg src matches to %h ; Unless you put the actual log, we can't be sure which all matches)
So you might need to amend this "EXTRACT-apache_access" in your "local" directory of the app (or your own app) with the www.conf
What is the sourcetype for your logs? Are any of the fields being extracted when you search on the sourcetype?
The sourcetype I've tried apache:access (From the Apache Addon) and access_combined (from the default install), and neither extract data or fields properly.