topic Re: how to extract date from filename and add it with time from event in the same file in Splunk Search

how to extract date from filename and add it with time from event in the same file

bkumarm — Tue, 25 Jul 2017 13:29:34 GMT

We have log files with names like: " my-file-log1.2017-07-25.name.log"
The events in the log are like this:
060047.342061Z INFO ASDFTestStatusMsg::eval: Passed(123/567892)

The time format in the events is: HHmmss.SSSSSS or HHmmss.SSS

Requirement is to add the date from filename into all the events at index time.
I also need help in converting the time into proper timestamp.

any solutions suggested?

Thanks,
Bharath

Re: how to extract date from filename and add it with time from event in the same file

bkumarm — Tue, 25 Jul 2017 13:52:11 GMT

I see that the http://docs.splunk.com/Documentation/Splunk/6.3.4/Data/HowSplunkextractstimestamps
says it does by default. but it is not working for me

Re: how to extract date from filename and add it with time from event in the same file

niketn — Tue, 25 Jul 2017 15:46:25 GMT

@bkumarm, in your props.conf, have you extracted only the time from your logs? If you have not as it is mentioned in the documentation Splunk will default time to file modified timestamp. If you extract the time properly, Splunk should be able to pull the date from the filename. Proper time format seems to be following:

[<yourSourceTypeName>]
TIME_FORMAT=%H%M%S.%6N

Please try out and confirm.

Re: how to extract date from filename and add it with time from event in the same file

JDukeSplunk — Tue, 29 Sep 2020 15:03:34 GMT

So when you search this data you do not get the fields
date_hour
date_mday
date_minute

etc?

Does adding this to your search add a new field named "indextime" ?

| eval indextime=strftime(_indextime,"%Y-%m-%d %H:%M:%S")

Re: how to extract date from filename and add it with time from event in the same file

bkumarm — Wed, 26 Jul 2017 05:58:09 GMT

Thanks Niket, your clue helped us resolve the issue.
in your props.conf, []
TIME_FORMAT=%H%M%S.%6N

we had also problem in filename, that we fixed.

-Bharath

Re: how to extract date from filename and add it with time from event in the same file

niketn — Wed, 26 Jul 2017 10:53:45 GMT

@bkumarm, glad it worked. Please up vote the comment if it helped.