Getting Data In

Another timestamp configuration issue

ebeckinger
New Member

I am trying to get my time stamp configured.

My log file has a recognizable date in the title and all my log messages are getting indexed to this date. In each raw log message there is a clearly defined time stamp.

Sample start of log message:
[ WARN] [11.09.21 14:12:05] [95]

I have a props.conf in my local directory (spluts/etc/system/local) and have changed the DATETIME_CONFIG to point to my custom datetime.xml

[default]
DATETIME_CONFIG = \etc\system\local\datetime.xml

My datetime.xml looks like this:

<datetime>
   <define name="_mydatetimeformat" extract="year, month, day, hour, minute, second">
       <text><![CDATA[^(?:\[.*\]\s\[)([0-9]+)\.([0-9]+)\.([0-9]+)\s([0-9]+):([0-9]+):([0-9]+)]]></text>
   </define>
   <timePatterns>
       <use name="_mydatetimeformat"/>
   </timePatterns>
   <datePatterns>
       <use name="_mydatetimeformat"/>
   </datePatterns> 
</datetime>

Perhaps I am not understanding how splunk associates the captures group with the different date fields. But adding this change doesn't seem to affect the dates that are displayed when searching. All log messages are still getting shown as the date time stamp found in the file name.

Any help is appreciated!

0 Karma

kristian_kolb
Ultra Champion

Hi,

I don't think you need to alter the datetime xml file. It's probably easier to add the TIME_PREFIX and TIME_FORMAT for your sourcetype stanza in props.conf (on the indexer).

In your case, that would probably be;

[your_sourcetype]
TIME_PREFIX = ^\[[A-Z]\]\s*\[
TIME_FORMAT = %y\.%m\.%d\s+%H:%M:%S

This is assuming that the severity ([WARN]) is only uppercase letters. You can find more information in the documentation.

http://docs.splunk.com/Documentation/Splunk/4.2.3/Data/Configuretimestamprecognition

Hope this helps,

Kristian

0 Karma

ebeckinger
New Member

I tried the time prefix and time format and it is still using the date in the file name to time stamp all of the log files.

I tried indexing a log file without a time stamp in the file name and it looks like it is parsing the year, month, day correctly but is also using these values for the hour, minute, and second

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...