topic Re: How to get Apache access logs to index with the correct timestamp (strptime)? in Getting Data In

How to get Apache access logs to index with the correct timestamp (strptime)?

cmaier — Mon, 22 Sep 2014 14:49:31 GMT

v5.0.4 indexers

I'm trying to get some Apache access logs to index with the correct timestamp, but no matter what I try, I can't get the date/time to be recognized correctly.

Example log:

www.somesite.com somestuff somemorestuff 192.168.1.1 2014-09-22 08:26:39 CDT 200 200 15416 - HTTP "GET blah" some more stuff

I've applied the following in props.conf to the sourcetype:

[thisparticular:apacheaccess]
MAX_TIMESTAMP_LOOKAHEAD=19
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=(?:\d{1,3}\.){3}\d{1,3}\s

The preview highlights the date and time as being found, but with a bit of a mixed up timestamp:

9/20/01 7:22:39.000 AM

I'd prefer having the timestamp first in the raw log (which is still an option for me), but I want to exhaust efforts in trying to get the above to work before making a change to the log format.

Am I missing something simple here?

Re: How to get Apache access logs to index with the correct timestamp (strptime)?

jimodonald — Mon, 28 Sep 2020 17:39:12 GMT

remove this:

MAX_TIMESTAMP_LOOKAHEAD=19

from your props.conf.

Re: How to get Apache access logs to index with the correct timestamp (strptime)?

cmaier — Mon, 28 Sep 2020 17:39:18 GMT

No luck jimodonald. In fact, I'm also testing the input on a 6.x platform and get similar results (they don't even offer the "MAX_TIMESTAMP_LOOKAHEAD" option in the 6.x preview).

Here's what it looks like in 6.x with similarly mixed up results:

[thisparticular:apacheaccess]
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=\s(?:\d{1,3}\.){3}\d{1,3}\s

On the above in 6.x, a log with "2014-09-22 08:26:39" yields a timestamp of "9/20/01 6:05:29.000 AM"

Re: How to get Apache access logs to index with the correct timestamp (strptime)?

jimodonald — Mon, 22 Sep 2014 15:46:47 GMT

If it's just the timezone, you can specify the timezone in props.conf with

TZ=US/Central

Alternatively Splunk usually does a good job with finding the timestamps on its own. Splunk is typically good about knowing how to parse the Apache logs. See http://docs.splunk.com/Documentation/Splunk/5.0.4/Data/Listofpretrainedsourcetypes

Re: How to get Apache access logs to index with the correct timestamp (strptime)?

cmaier — Mon, 22 Sep 2014 16:04:59 GMT

Oddly enough, a timezone issue is actually what led me to where I am currently. I was trying to apply a timezone offset to the sourcetype and that's when I realized it wasn't even grabbing the event time from the log - it's using the default indexer time.

As soon as I can get it to grab the time correctly from the log, I should be able to apply the offset as needed.

Re: How to get Apache access logs to index with the correct timestamp (strptime)?

gkanapathy — Tue, 23 Sep 2014 01:22:54 GMT

Make sure you're putting the settings on the right place (indexer vs forwarder): http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F