Getting Data In

How to get Apache access logs to index with the correct timestamp (strptime)?

cmaier
Explorer

v5.0.4 indexers

I'm trying to get some Apache access logs to index with the correct timestamp, but no matter what I try, I can't get the date/time to be recognized correctly.

Example log:

www.somesite.com somestuff somemorestuff 192.168.1.1 2014-09-22 08:26:39 CDT 200 200 15416 - HTTP "GET blah" some more stuff

I've applied the following in props.conf to the sourcetype:

[thisparticular:apacheaccess]
MAX_TIMESTAMP_LOOKAHEAD=19
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=(?:\d{1,3}\.){3}\d{1,3}\s

The preview highlights the date and time as being found, but with a bit of a mixed up timestamp:

9/20/01 7:22:39.000 AM

I'd prefer having the timestamp first in the raw log (which is still an option for me), but I want to exhaust efforts in trying to get the above to work before making a change to the log format.

Am I missing something simple here?

0 Karma

jimodonald
Contributor

remove this:

MAX_TIMESTAMP_LOOKAHEAD=19

from your props.conf.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Make sure you're putting the settings on the right place (indexer vs forwarder): http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

0 Karma

cmaier
Explorer

Oddly enough, a timezone issue is actually what led me to where I am currently. I was trying to apply a timezone offset to the sourcetype and that's when I realized it wasn't even grabbing the event time from the log - it's using the default indexer time.

As soon as I can get it to grab the time correctly from the log, I should be able to apply the offset as needed.

0 Karma

jimodonald
Contributor

If it's just the timezone, you can specify the timezone in props.conf with

TZ=US/Central

Alternatively Splunk usually does a good job with finding the timestamps on its own. Splunk is typically good about knowing how to parse the Apache logs. See http://docs.splunk.com/Documentation/Splunk/5.0.4/Data/Listofpretrainedsourcetypes

0 Karma

cmaier
Explorer

No luck jimodonald. In fact, I'm also testing the input on a 6.x platform and get similar results (they don't even offer the "MAX_TIMESTAMP_LOOKAHEAD" option in the 6.x preview).

Here's what it looks like in 6.x with similarly mixed up results:

[thisparticular:apacheaccess]
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=\s(?:\d{1,3}\.){3}\d{1,3}\s

On the above in 6.x, a log with "2014-09-22 08:26:39" yields a timestamp of "9/20/01 6:05:29.000 AM"

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...