Getting Data In

extract time from events but date from modification time

kurt28
Path Finder

Hello,

I've read Precedence rules for timestamp assignment.

What I want is to extract time)(hour,minute,second,millisecond) from events but date(month,day,year) from modification time, because my log is like this :

[073242:123] xxx xxx xxx ...

which:

07 is for hour

32 is for minute

42 is for second

123 is for millisecond

and I need to get date from modification time because I don't have date info in the events and the log file is created daily.

According to Precedence rules for timestamp assignment,

I try to set

TIME_PREFIX = ^[

TIME_FORMAT = %H%M%S:%Q

in props.conf and I get the right time but wrong date(splunk gets the system date for me)

Besides, theoretically I can let modification time extracted for my event date by setting

DATETIME_CONFIG = NONE in props.conf, but it failed. I still get time that the log be indexed.

And even DATETIME_CONFIG = NONE works for my event date it will fail on

TIME_PREFIX = ^[

TIME_FORMAT = %H%M%S:%Q

because timestamp extraction in event log will be turned off.

any one have idea what I can do for it?

I will be very appreciate for any guidance.

0 Karma

woodcock
Esteemed Legend

The closest that you can get is to encode the date in the filename as a literal string. Once you do this (you can use a non-splunk custom pre-parser script to do this if you have to), then you can use datetime.xml to pull the date from the filename but the time from each event like this:

<datetime>
<!--EACH FILENAME HAS THE FOLLOWING FORMAT:
"*-YYMMDD-hh_mm" Where
"YY" is a two digit number representing the year.
"MM" is a two digit number representing the month.
"DD" is a two digit number representing the day of the month.
"hh" is a two digit number representing the hour of the day (24 hour).
"mm" is a two digit number representing the number of minutes after the hour.-->
<define name="datefromfile" extract="year, month, day">
   <text><![CDATA[source::.*?-(\d{2})(\d{2})(\d{2}]]>-</text>
</define>
<!--EACH EVENT HAS THE FOLLOWING FORMAT:
#Position|Description
#   1-2  |Time of day-hours
#   3-4  |Time of day-minutes
#   5-6  |Time of day-seconds
<define name="timefromevents" extract="hour, minute,second">
    <text><![CDATA[^(\d{2})(\d{2})(\d{2})]]></text>
</define>
<timePatterns>
   <use name="timefromevents"/>
</timePatterns>
<datePatterns>
   <use name="datefromfile"/>
</datePatterns>
</datetime>
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...