Splunk Search

how to extract date from filename and add it with time from event in the same file

bkumarm
Contributor

We have log files with names like: " my-file-log1.2017-07-25.name.log"
The events in the log are like this:
060047.342061Z INFO ASDFTestStatusMsg::eval: Passed(123/567892)

The time format in the events is: HHmmss.SSSSSS or HHmmss.SSS

Requirement is to add the date from filename into all the events at index time.
I also need help in converting the time into proper timestamp.

any solutions suggested?

Thanks,
Bharath

1 Solution

bkumarm
Contributor

Thanks Niket, your clue helped us resolve the issue.
in your props.conf, []
TIME_FORMAT=%H%M%S.%6N

we had also problem in filename, that we fixed.

-Bharath

View solution in original post

0 Karma

bkumarm
Contributor

Thanks Niket, your clue helped us resolve the issue.
in your props.conf, []
TIME_FORMAT=%H%M%S.%6N

we had also problem in filename, that we fixed.

-Bharath

0 Karma

niketn
Legend

@bkumarm, glad it worked. Please up vote the comment if it helped.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

bkumarm
Contributor

I see that the http://docs.splunk.com/Documentation/Splunk/6.3.4/Data/HowSplunkextractstimestamps
says it does by default. but it is not working for me

0 Karma

JDukeSplunk
Builder

So when you search this data you do not get the fields
date_hour
date_mday
date_minute

etc?

Does adding this to your search add a new field named "indextime" ?

| eval indextime=strftime(_indextime,"%Y-%m-%d %H:%M:%S")

niketn
Legend

@bkumarm, in your props.conf, have you extracted only the time from your logs? If you have not as it is mentioned in the documentation Splunk will default time to file modified timestamp. If you extract the time properly, Splunk should be able to pull the date from the filename. Proper time format seems to be following:

[<yourSourceTypeName>]
TIME_FORMAT=%H%M%S.%6N

Please try out and confirm.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...