How can we edit the regular expressions to extract fields from our Apache Access logs?

[romangarth]

We just installed Splunk this week (YAY!) and are trying to get our Apache logs digesting to start building dashboards. Our web store uses a modified Apache Access log format that looks like this in our www.conf:

%h %l %u %t %V \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %U %q

I know that none of the extractions provided by access_combined or apache:access (Apache Addon) seem to work because they're looking for the default format, but I'm a little confused on how to help them recognize our format. I see the extractions listed, but they don't look like regular regexes that I've seen. How would I go about changing them to match our formatting?

Appreciate any help!

[koshyk]

I can see the default extraction in the TA is:

EXTRACT-apache_access= ^(?<src>[^ ]+)\s+(?<site>.+)\s+(?<ident>\S+)\s+(?<user>\S+)\s+(?<dest_port>\d+)\s+\[(?<timestamp>\d+/\w+/\d+:\d+:\d+:\d+\s+[-+]\d+)[^"\n]*"(?<request>[^"]+)[^ \n]*\s+\"(?<uri_query>[^ ]*)\"\s+(?<status>\d+)(?:[^ \n]* ){2}\"(?<http_referer>[^"]+)\"\s+\"(?<http_user_agent>[^"]+)\"\s+(?<request_bytes>\d+)\s+(?<response_bytes>\d+)\s+(?<response_time_microseconds>\d+)

This matches with your pattern to a certain degree, but NOT all.
(eg src matches to %h ; Unless you put the actual log, we can't be sure which all matches)
So you might need to amend this "EXTRACT-apache_access" in your "local" directory of the app (or your own app) with the www.conf

[AnthonyTibaldi]

What is the sourcetype for your logs? Are any of the fields being extracted when you search on the sourcetype?

[romangarth]

The sourcetype I've tried apache:access (From the Apache Addon) and access_combined (from the default install), and neither extract data or fields properly.