Getting Data In

Another timestamp configuration issue

ebeckinger
New Member

I am trying to get my time stamp configured.

My log file has a recognizable date in the title and all my log messages are getting indexed to this date. In each raw log message there is a clearly defined time stamp.

Sample start of log message:
[ WARN] [11.09.21 14:12:05] [95]

I have a props.conf in my local directory (spluts/etc/system/local) and have changed the DATETIME_CONFIG to point to my custom datetime.xml

[default]
DATETIME_CONFIG = \etc\system\local\datetime.xml

My datetime.xml looks like this:

<datetime>
   <define name="_mydatetimeformat" extract="year, month, day, hour, minute, second">
       <text><![CDATA[^(?:\[.*\]\s\[)([0-9]+)\.([0-9]+)\.([0-9]+)\s([0-9]+):([0-9]+):([0-9]+)]]></text>
   </define>
   <timePatterns>
       <use name="_mydatetimeformat"/>
   </timePatterns>
   <datePatterns>
       <use name="_mydatetimeformat"/>
   </datePatterns> 
</datetime>

Perhaps I am not understanding how splunk associates the captures group with the different date fields. But adding this change doesn't seem to affect the dates that are displayed when searching. All log messages are still getting shown as the date time stamp found in the file name.

Any help is appreciated!

0 Karma

kristian_kolb
Ultra Champion

Hi,

I don't think you need to alter the datetime xml file. It's probably easier to add the TIME_PREFIX and TIME_FORMAT for your sourcetype stanza in props.conf (on the indexer).

In your case, that would probably be;

[your_sourcetype]
TIME_PREFIX = ^\[[A-Z]\]\s*\[
TIME_FORMAT = %y\.%m\.%d\s+%H:%M:%S

This is assuming that the severity ([WARN]) is only uppercase letters. You can find more information in the documentation.

http://docs.splunk.com/Documentation/Splunk/4.2.3/Data/Configuretimestamprecognition

Hope this helps,

Kristian

0 Karma

ebeckinger
New Member

I tried the time prefix and time format and it is still using the date in the file name to time stamp all of the log files.

I tried indexing a log file without a time stamp in the file name and it looks like it is parsing the year, month, day correctly but is also using these values for the hour, minute, and second

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...