topic extract time from events but date from modification time in Getting Data In

extract time from events but date from modification time

kurt28 — Wed, 28 Sep 2011 11:05:27 GMT

Hello,

I've read Precedence rules for timestamp assignment.

What I want is to extract time)(hour,minute,second,millisecond) from events but date(month,day,year) from modification time, because my log is like this :

[073242:123] xxx xxx xxx ...

which:

07 is for hour

32 is for minute

42 is for second

123 is for millisecond

and I need to get date from modification time because I don't have date info in the events and the log file is created daily.

According to Precedence rules for timestamp assignment,

I try to set

TIME_PREFIX = ^[

TIME_FORMAT = %H%M%S:%Q

in props.conf and I get the right time but wrong date(splunk gets the system date for me)

Besides, theoretically I can let modification time extracted for my event date by setting

DATETIME_CONFIG = NONE in props.conf, but it failed. I still get time that the log be indexed.

And even DATETIME_CONFIG = NONE works for my event date it will fail on

TIME_PREFIX = ^[

TIME_FORMAT = %H%M%S:%Q

because timestamp extraction in event log will be turned off.

any one have idea what I can do for it?

I will be very appreciate for any guidance.

Re: extract time from events but date from modification time

woodcock — Fri, 29 May 2015 21:14:26 GMT

The closest that you can get is to encode the date in the filename as a literal string. Once you do this (you can use a non-splunk custom pre-parser script to do this if you have to), then you can use datetime.xml to pull the date from the filename but the time from each event like this:

<datetime>
<!--EACH FILENAME HAS THE FOLLOWING FORMAT:
"*-YYMMDD-hh_mm" Where
"YY" is a two digit number representing the year.
"MM" is a two digit number representing the month.
"DD" is a two digit number representing the day of the month.
"hh" is a two digit number representing the hour of the day (24 hour).
"mm" is a two digit number representing the number of minutes after the hour.-->
<define name="datefromfile" extract="year, month, day">
   <text><![CDATA[source::.*?-(\d{2})(\d{2})(\d{2}]]>-</text>
</define>
<!--EACH EVENT HAS THE FOLLOWING FORMAT:
#Position|Description
#   1-2  |Time of day-hours
#   3-4  |Time of day-minutes
#   5-6  |Time of day-seconds
<define name="timefromevents" extract="hour, minute,second">
    <text><![CDATA[^(\d{2})(\d{2})(\d{2})]]></text>
</define>
<timePatterns>
   <use name="timefromevents"/>
</timePatterns>
<datePatterns>
   <use name="datefromfile"/>
</datePatterns>
</datetime>