Getting Data In

How to get Apache access logs to index with the correct timestamp (strptime)?

Explorer

v5.0.4 indexers

I'm trying to get some Apache access logs to index with the correct timestamp, but no matter what I try, I can't get the date/time to be recognized correctly.

Example log:

www.somesite.com somestuff somemorestuff 192.168.1.1 2014-09-22 08:26:39 CDT 200 200 15416 - HTTP "GET blah" some more stuff

I've applied the following in props.conf to the sourcetype:

[thisparticular:apacheaccess]
MAX_TIMESTAMP_LOOKAHEAD=19
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=(?:\d{1,3}\.){3}\d{1,3}\s

The preview highlights the date and time as being found, but with a bit of a mixed up timestamp:

9/20/01 7:22:39.000 AM

I'd prefer having the timestamp first in the raw log (which is still an option for me), but I want to exhaust efforts in trying to get the above to work before making a change to the log format.

Am I missing something simple here?

0 Karma

Contributor

remove this:

MAX_TIMESTAMP_LOOKAHEAD=19

from your props.conf.

0 Karma

Splunk Employee
Splunk Employee

Make sure you're putting the settings on the right place (indexer vs forwarder): http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

0 Karma

Explorer

Oddly enough, a timezone issue is actually what led me to where I am currently. I was trying to apply a timezone offset to the sourcetype and that's when I realized it wasn't even grabbing the event time from the log - it's using the default indexer time.

As soon as I can get it to grab the time correctly from the log, I should be able to apply the offset as needed.

0 Karma

Contributor

If it's just the timezone, you can specify the timezone in props.conf with

TZ=US/Central

Alternatively Splunk usually does a good job with finding the timestamps on its own. Splunk is typically good about knowing how to parse the Apache logs. See http://docs.splunk.com/Documentation/Splunk/5.0.4/Data/Listofpretrainedsourcetypes

0 Karma

Explorer

No luck jimodonald. In fact, I'm also testing the input on a 6.x platform and get similar results (they don't even offer the "MAX_TIMESTAMP_LOOKAHEAD" option in the 6.x preview).

Here's what it looks like in 6.x with similarly mixed up results:

[thisparticular:apacheaccess]
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=\s(?:\d{1,3}\.){3}\d{1,3}\s

On the above in 6.x, a log with "2014-09-22 08:26:39" yields a timestamp of "9/20/01 6:05:29.000 AM"

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!