Splunk Search

Extract date / time from source path with custom datetime.xml

emechler_splunk
Splunk Employee
Splunk Employee

I'm hoping someone can help out with something that's been baffling me re: using custom a datetime.xml to extract the date and time from a file's path name. I want to pull out the year, month, day, hour, and minute from the path name for a given set of data. Source would look something like this:

/path/to/data/20140416/1506/2014041615060005/[filename]

Where 2014 = year, 04 = month, 16 = day, 15 = hour, and 06 = minute. I've tried two variants of custom XML and neither are pulling out either the date nor the time (Splunk only seems to use the file's modtime no matter what I try):

<datetime>
  <define name="_customdate" extract="year, month, day">
      <text><![CDATA[(?:source::.*?/)(20\d{2})(\d{2})(\d{2})(?:/)]]></text>
  </define>

  <define name="_customtime" extract="hour, minute">
        <text><![CDATA[(?:source::.*?/)(\d{2})(\d{2})(?:/)]]></text>
  </define>

  <timePatterns>
      <use name="_customtime"/>
  </timePatterns>
  <datePatterns>
      <use name="_customdate"/>
  </datePatterns>
</datetime>

I've even tried doing this with one stanza and that doesn't work either.

  <define name="_masheddate3" extract="year, month, day, hour, minute">
    <text><![CDATA[(?:source:.*?/)(20\d{2})(\d{2})(\d{2})(?:/)(\d{2})(\d{2})(?:/)(\d{16})(?:/)]]></text>
  </define>

Thoughts on what I might be doing wrong here? Thank you!

bosburn_splunk
Splunk Employee
Splunk Employee

Erick,
At this time you cannot extract time from the filename - only the date.

Brian

nitesh218ss
Communicator

Hi in my log event and filename date is not present i want give a fix date to log so what is do ?

0 Karma

emechler_splunk
Splunk Employee
Splunk Employee

Thanks, Brian. If that's the case then I should be able to use the _customdate stanza above just to pull out the date - what do I need to do for the timestamp then? I need something inside timePatterns otherwise I get an error.

0 Karma

jrodman
Splunk Employee
Splunk Employee

Splunk should establish the date from the filename (though it may constrain it to MAX_DAYS_AGO/HENCE so if modtime and the filename differ really sharply, be careful with that). Then it should proceed to attempt to establiash both time and date from the event text, using your rules.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Observability Simplified: Combining User Experience, Application Performance & ...

Tech Talk Observability Simplified: Combining User Experience, Application Performance & Network ...

Event Series May & June: From Network Visibility to Service Intelligence

Unifying the Network: Moving from Alert Noise to Service Intelligence with Splunk ITSI In today’s hybrid ...