Getting Data In

How to get Apache access logs to index with the correct timestamp (strptime)?

cmaier
Explorer

v5.0.4 indexers

I'm trying to get some Apache access logs to index with the correct timestamp, but no matter what I try, I can't get the date/time to be recognized correctly.

Example log:

www.somesite.com somestuff somemorestuff 192.168.1.1 2014-09-22 08:26:39 CDT 200 200 15416 - HTTP "GET blah" some more stuff

I've applied the following in props.conf to the sourcetype:

[thisparticular:apacheaccess]
MAX_TIMESTAMP_LOOKAHEAD=19
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=(?:\d{1,3}\.){3}\d{1,3}\s

The preview highlights the date and time as being found, but with a bit of a mixed up timestamp:

9/20/01 7:22:39.000 AM

I'd prefer having the timestamp first in the raw log (which is still an option for me), but I want to exhaust efforts in trying to get the above to work before making a change to the log format.

Am I missing something simple here?

0 Karma

jimodonald
Contributor

remove this:

MAX_TIMESTAMP_LOOKAHEAD=19

from your props.conf.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Make sure you're putting the settings on the right place (indexer vs forwarder): http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

0 Karma

cmaier
Explorer

Oddly enough, a timezone issue is actually what led me to where I am currently. I was trying to apply a timezone offset to the sourcetype and that's when I realized it wasn't even grabbing the event time from the log - it's using the default indexer time.

As soon as I can get it to grab the time correctly from the log, I should be able to apply the offset as needed.

0 Karma

jimodonald
Contributor

If it's just the timezone, you can specify the timezone in props.conf with

TZ=US/Central

Alternatively Splunk usually does a good job with finding the timestamps on its own. Splunk is typically good about knowing how to parse the Apache logs. See http://docs.splunk.com/Documentation/Splunk/5.0.4/Data/Listofpretrainedsourcetypes

0 Karma

cmaier
Explorer

No luck jimodonald. In fact, I'm also testing the input on a 6.x platform and get similar results (they don't even offer the "MAX_TIMESTAMP_LOOKAHEAD" option in the 6.x preview).

Here's what it looks like in 6.x with similarly mixed up results:

[thisparticular:apacheaccess]
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT=%Y-%m-%d %H:%M:%S
TIME_PREFIX=\s(?:\d{1,3}\.){3}\d{1,3}\s

On the above in 6.x, a log with "2014-09-22 08:26:39" yields a timestamp of "9/20/01 6:05:29.000 AM"

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...