topic Time stamp stanza in Getting Data In

Time stamp stanza

sbattista09 — Tue, 29 Sep 2020 11:17:57 GMT

I want to make sure i understand this, i have logs that splunk can not find the time stamp on. and some are missing.

for the logs that have the time in them i would juse use this in props.conf on the Heavy forwaders correct?

[source_type]
TIME_PREFIX = \d\d\/\w\w\w\/\d\d\d\d:\d\d:\d\d:\d\d
TIME_FORMAT = %d/%b/%Y%::z

log looks like this:

--ab50cd40-A--
[25/Sep/2016:04:08:52 --0400] 
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

For the logs that do not have a time stamp, how to i set them to use indexed time for the time stamp?

--ab50cd30-A--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

--ac50ad30-H--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

--090e4955-A--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

Re: Time stamp stanza

dmaislin_splunk — Wed, 05 Oct 2016 13:10:19 GMT

Try this:

TIME_PREFIX =  .*?\[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S -%z
MAX_TIMESTAMP_LOOKAHEAD = 28

This is a good blog I put together if you have multiple time formats in the same log file and some events with nothing: http://blogs.splunk.com/2014/04/23/its-that-time-again

For events with no dates at all, just set:

DATETIME_CONFIG = current

Re: Time stamp stanza

sundareshr — Wed, 05 Oct 2016 13:13:34 GMT

For the logs with timestamp, splunk should automatically recognize the timeformat. If it doesn't use this

TIME_FORMAT=%d/%b/%Y:%X
TIME_PREFIX=\[
MAX_TIMESTAMP_LOOKAHEAD=25

For the logs without timestamp, try this

DATETIME_CONFIG=CURRENT

Re: Time stamp stanza

sbattista09 — Tue, 29 Sep 2020 11:18:00 GMT

awesome! so its okay to add all this in one stanza in props.conf?

[sourcetype_name]
TIME_PREFIX = .*?[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S -%z
MAX_TIMESTAMP_LOOKAHEAD = 28
DATETIME_CONFIG = current

Re: Time stamp stanza

tormodbp — Wed, 05 Oct 2016 13:20:34 GMT

I believe that you can do it on the indexer by specifying the following in props.conf

[mysourcetype]
DATETIME_CONFIG = CURRENT

From the props.conf documentation we can see that

"CURRENT" will set the time of the event to the time that the event was
merged from lines, or worded differently, the time it passed through the
aggregator processor.

DATETIME_CONFIG is usually used to specify the file that configures the timestamp extractor, but can also be used to prevent a timestamp extractor or assign the current system time to each event.

More information can be found here:
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Hope this helps

Cheers,

Re: Time stamp stanza

dmaislin_splunk — Tue, 29 Sep 2020 11:17:15 GMT

But DATETIME_CONFIG=current will override the settings for timestamp configurations and will set all timestamps to the current time. I don't know your data so not sure if you need a custom DATETIME_CONFIG file.