Splunk Search

Extract date / time from source path with custom datetime.xml

emechler_splunk
Splunk Employee
Splunk Employee

I'm hoping someone can help out with something that's been baffling me re: using custom a datetime.xml to extract the date and time from a file's path name. I want to pull out the year, month, day, hour, and minute from the path name for a given set of data. Source would look something like this:

/path/to/data/20140416/1506/2014041615060005/[filename]

Where 2014 = year, 04 = month, 16 = day, 15 = hour, and 06 = minute. I've tried two variants of custom XML and neither are pulling out either the date nor the time (Splunk only seems to use the file's modtime no matter what I try):

<datetime>
  <define name="_customdate" extract="year, month, day">
      <text><![CDATA[(?:source::.*?/)(20\d{2})(\d{2})(\d{2})(?:/)]]></text>
  </define>

  <define name="_customtime" extract="hour, minute">
        <text><![CDATA[(?:source::.*?/)(\d{2})(\d{2})(?:/)]]></text>
  </define>

  <timePatterns>
      <use name="_customtime"/>
  </timePatterns>
  <datePatterns>
      <use name="_customdate"/>
  </datePatterns>
</datetime>

I've even tried doing this with one stanza and that doesn't work either.

  <define name="_masheddate3" extract="year, month, day, hour, minute">
    <text><![CDATA[(?:source:.*?/)(20\d{2})(\d{2})(\d{2})(?:/)(\d{2})(\d{2})(?:/)(\d{16})(?:/)]]></text>
  </define>

Thoughts on what I might be doing wrong here? Thank you!

bosburn_splunk
Splunk Employee
Splunk Employee

Erick,
At this time you cannot extract time from the filename - only the date.

Brian

Bosley

nitesh218ss
Communicator

Hi in my log event and filename date is not present i want give a fix date to log so what is do ?

0 Karma

emechler_splunk
Splunk Employee
Splunk Employee

Thanks, Brian. If that's the case then I should be able to use the _customdate stanza above just to pull out the date - what do I need to do for the timestamp then? I need something inside timePatterns otherwise I get an error.

0 Karma

jrodman
Splunk Employee
Splunk Employee

Splunk should establish the date from the filename (though it may constrain it to MAX_DAYS_AGO/HENCE so if modtime and the filename differ really sharply, be careful with that). Then it should proceed to attempt to establiash both time and date from the event text, using your rules.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Break the Build: Inside the KubeDoom Lounge at .conf26

    You step up to the machine. The pixelated corridors of a certain 1993 FPS load in front of you, EMP Pulse ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...