Getting Data In

Time stamp stanza

sbattista09
Contributor

I want to make sure i understand this, i have logs that splunk can not find the time stamp on. and some are missing.

for the logs that have the time in them i would juse use this in props.conf on the Heavy forwaders correct?

[source_type]
TIME_PREFIX = \d\d\/\w\w\w\/\d\d\d\d:\d\d:\d\d:\d\d
TIME_FORMAT = %d/%b/%Y%::z

log looks like this:

--ab50cd40-A--
[25/Sep/2016:04:08:52 --0400] 
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

For the logs that do not have a time stamp, how to i set them to use indexed time for the time stamp?

--ab50cd30-A--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

--ac50ad30-H--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH

--090e4955-A--
BLAH BLAHBLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
BLAH BLAHBLAH BLAHBLAH BLAH
0 Karma
1 Solution

sundareshr
Legend

For the logs with timestamp, splunk should automatically recognize the timeformat. If it doesn't use this

TIME_FORMAT=%d/%b/%Y:%X
TIME_PREFIX=\[
MAX_TIMESTAMP_LOOKAHEAD=25

For the logs without timestamp, try this

DATETIME_CONFIG=CURRENT

View solution in original post

tormodbp
Path Finder

I believe that you can do it on the indexer by specifying the following in props.conf

[mysourcetype]
DATETIME_CONFIG = CURRENT

From the props.conf documentation we can see that

"CURRENT" will set the time of the event to the time that the event was
merged from lines, or worded differently, the time it passed through the
aggregator processor.

DATETIME_CONFIG is usually used to specify the file that configures the timestamp extractor, but can also be used to prevent a timestamp extractor or assign the current system time to each event.

More information can be found here:
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

Hope this helps

Cheers,

sundareshr
Legend

For the logs with timestamp, splunk should automatically recognize the timeformat. If it doesn't use this

TIME_FORMAT=%d/%b/%Y:%X
TIME_PREFIX=\[
MAX_TIMESTAMP_LOOKAHEAD=25

For the logs without timestamp, try this

DATETIME_CONFIG=CURRENT

dmaislin_splunk
Splunk Employee
Splunk Employee

Try this:

TIME_PREFIX =  .*?\[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S -%z
MAX_TIMESTAMP_LOOKAHEAD = 28

This is a good blog I put together if you have multiple time formats in the same log file and some events with nothing: http://blogs.splunk.com/2014/04/23/its-that-time-again

For events with no dates at all, just set:

DATETIME_CONFIG = current

sbattista09
Contributor

awesome! so its okay to add all this in one stanza in props.conf?

[sourcetype_name]
TIME_PREFIX = .*?[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S -%z
MAX_TIMESTAMP_LOOKAHEAD = 28
DATETIME_CONFIG = current

0 Karma

dmaislin_splunk
Splunk Employee
Splunk Employee

But DATETIME_CONFIG=current will override the settings for timestamp configurations and will set all timestamps to the current time. I don't know your data so not sure if you need a custom DATETIME_CONFIG file.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...