Splunk Search

Extract date / time from source path with custom datetime.xml

emechler_splunk
Splunk Employee
Splunk Employee

I'm hoping someone can help out with something that's been baffling me re: using custom a datetime.xml to extract the date and time from a file's path name. I want to pull out the year, month, day, hour, and minute from the path name for a given set of data. Source would look something like this:

/path/to/data/20140416/1506/2014041615060005/[filename]

Where 2014 = year, 04 = month, 16 = day, 15 = hour, and 06 = minute. I've tried two variants of custom XML and neither are pulling out either the date nor the time (Splunk only seems to use the file's modtime no matter what I try):

<datetime>
  <define name="_customdate" extract="year, month, day">
      <text><![CDATA[(?:source::.*?/)(20\d{2})(\d{2})(\d{2})(?:/)]]></text>
  </define>

  <define name="_customtime" extract="hour, minute">
        <text><![CDATA[(?:source::.*?/)(\d{2})(\d{2})(?:/)]]></text>
  </define>

  <timePatterns>
      <use name="_customtime"/>
  </timePatterns>
  <datePatterns>
      <use name="_customdate"/>
  </datePatterns>
</datetime>

I've even tried doing this with one stanza and that doesn't work either.

  <define name="_masheddate3" extract="year, month, day, hour, minute">
    <text><![CDATA[(?:source:.*?/)(20\d{2})(\d{2})(\d{2})(?:/)(\d{2})(\d{2})(?:/)(\d{16})(?:/)]]></text>
  </define>

Thoughts on what I might be doing wrong here? Thank you!

bosburn_splunk
Splunk Employee
Splunk Employee

Erick,
At this time you cannot extract time from the filename - only the date.

Brian

nitesh218ss
Communicator

Hi in my log event and filename date is not present i want give a fix date to log so what is do ?

0 Karma

emechler_splunk
Splunk Employee
Splunk Employee

Thanks, Brian. If that's the case then I should be able to use the _customdate stanza above just to pull out the date - what do I need to do for the timestamp then? I need something inside timePatterns otherwise I get an error.

0 Karma

jrodman
Splunk Employee
Splunk Employee

Splunk should establish the date from the filename (though it may constrain it to MAX_DAYS_AGO/HENCE so if modtime and the filename differ really sharply, be careful with that). Then it should proceed to attempt to establiash both time and date from the event text, using your rules.

0 Karma
Get Updates on the Splunk Community!

Updated Data Type Articles, Anniversary Celebrations, and More on Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

A Prelude to .conf25: Your Guide to Splunk University

Heading to Boston this September for .conf25? Get a jumpstart by arriving a few days early for Splunk ...

4 Ways the Splunk Community Helps You Prepare for .conf25

.conf25 is right around the corner, and whether you’re a first-time attendee or a seasoned Splunker, the ...