Another timestamp configuration issue

[ebeckinger]

I am trying to get my time stamp configured.

My log file has a recognizable date in the title and all my log messages are getting indexed to this date. In each raw log message there is a clearly defined time stamp.

Sample start of log message:
[ WARN] [11.09.21 14:12:05] [95]

I have a props.conf in my local directory (spluts/etc/system/local) and have changed the DATETIME_CONFIG to point to my custom datetime.xml

[default]
DATETIME_CONFIG = \etc\system\local\datetime.xml

My datetime.xml looks like this:

<datetime>
   <define name="_mydatetimeformat" extract="year, month, day, hour, minute, second">
       <text><![CDATA[^(?:\[.*\]\s\[)([0-9]+)\.([0-9]+)\.([0-9]+)\s([0-9]+):([0-9]+):([0-9]+)]]></text>
   </define>
   <timePatterns>
       <use name="_mydatetimeformat"/>
   </timePatterns>
   <datePatterns>
       <use name="_mydatetimeformat"/>
   </datePatterns> 
</datetime>

Perhaps I am not understanding how splunk associates the captures group with the different date fields. But adding this change doesn't seem to affect the dates that are displayed when searching. All log messages are still getting shown as the date time stamp found in the file name.

Any help is appreciated!

[kristian_kolb]

Hi,

I don't think you need to alter the datetime xml file. It's probably easier to add the TIME_PREFIX and TIME_FORMAT for your sourcetype stanza in props.conf (on the indexer).

In your case, that would probably be;

[your_sourcetype]
TIME_PREFIX = ^\[[A-Z]\]\s*\[
TIME_FORMAT = %y\.%m\.%d\s+%H:%M:%S

This is assuming that the severity ([WARN]) is only uppercase letters. You can find more information in the documentation.

http://docs.splunk.com/Documentation/Splunk/4.2.3/Data/Configuretimestamprecognition

Hope this helps,

Kristian

[ebeckinger]

I tried the time prefix and time format and it is still using the date in the file name to time stamp all of the log files.

I tried indexing a log file without a time stamp in the file name and it looks like it is parsing the year, month, day correctly but is also using these values for the hour, minute, and second