Getting Data In

Time stamp stanza

Contributor

I want to make sure i understand this, i have logs that splunk can not find the time stamp on. and some are missing.

for the logs that have the time in them i would juse use this in props.conf on the Heavy forwaders correct?

[sourcetype]
TIME
PREFIX = \d\d\/\w\w\w\/\d\d\d\d:\d\d:\d\d:\d\d
TIME_FORMAT = %d/%b/%Y%::z

log looks like this:

--ab50cd40-A--
[25/Sep/2016:04:08:52 --0400] 
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

For the logs that do not have a time stamp, how to i set them to use indexed time for the time stamp?

--ab50cd30-A--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

--ac50ad30-H--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

--090e4955-A--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
0 Karma
1 Solution

Legend

For the logs with timestamp, splunk should automatically recognize the timeformat. If it doesn't use this

TIME_FORMAT=%d/%b/%Y:%X
TIME_PREFIX=\[
MAX_TIMESTAMP_LOOKAHEAD=25

For the logs without timestamp, try this

DATETIME_CONFIG=CURRENT

View solution in original post

Path Finder

I believe that you can do it on the indexer by specifying the following in props.conf

[mysourcetype]
DATETIME_CONFIG = CURRENT

From the props.conf documentation we can see that

"CURRENT" will set the time of the event to the time that the event was
merged from lines, or worded differently, the time it passed through the
aggregator processor.

DATETIME_CONFIG is usually used to specify the file that configures the timestamp extractor, but can also be used to prevent a timestamp extractor or assign the current system time to each event.

More information can be found here:
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Hope this helps

Cheers,

Legend

For the logs with timestamp, splunk should automatically recognize the timeformat. If it doesn't use this

TIME_FORMAT=%d/%b/%Y:%X
TIME_PREFIX=\[
MAX_TIMESTAMP_LOOKAHEAD=25

For the logs without timestamp, try this

DATETIME_CONFIG=CURRENT

View solution in original post

Splunk Employee
Splunk Employee

Try this:

TIME_PREFIX =  .*?\[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S -%z
MAX_TIMESTAMP_LOOKAHEAD = 28

This is a good blog I put together if you have multiple time formats in the same log file and some events with nothing: http://blogs.splunk.com/2014/04/23/its-that-time-again

For events with no dates at all, just set:

DATETIME_CONFIG = current

Contributor

awesome! so its okay to add all this in one stanza in props.conf?

[sourcetypename]
TIME
PREFIX = .*?[
TIMEFORMAT = %d/%b/%Y:%H:%M:%S -%z
MAX
TIMESTAMPLOOKAHEAD = 28
DATETIME
CONFIG = current

0 Karma

Splunk Employee
Splunk Employee

But DATETIMECONFIG=current will override the settings for timestamp configurations and will set all timestamps to the current time. I don't know your data so not sure if you need a custom DATETIMECONFIG file.

0 Karma