Getting Data In

Extract date from Filename source

ips_mandar
Builder

Hi in my events I am getting time which is extracted correctly by Splunk for _time timestamp and for date extraction I want to use source field which is in format like- D:\\abc\\def\\XYLog09229190601.txt.zip:.\\XYLog09229190601.txt
From above source field I want to extract last 6 digit before .txt to create date like for this example it will be 2019-06-01.
What will be best way to do that..
Is it using datetime.xml? and how I can achieve it since for year only two digits are mentioned.
Thanks.

0 Karma

DavidHourani
Super Champion

Hi @ips_mandar,

Which version are you running ?

If Splunk 7.2+, have a look here :
https://docs.splunk.com/Documentation/Splunk/latest/Data/IngestEval

This is a pretty easy way to get time extracted at index time.

Cheers,
David

ips_mandar
Builder

Thanks For your suggestion @DavidHourani ..Yes I am using Splunk 7.2+
But I thought that it will impact performance so I am trying to avoid it (Ingest Eval).

0 Karma

DavidHourani
Super Champion

Yeah true, there would be an impact... In that case have you tried simply setting TIME_FORMAT in your props.conf ?
This will first apply to the events and then fallback to the filename if it's not found in the event, so it should grab your time from there.

0 Karma

ips_mandar
Builder

Currently I am trying with changing datetime.xml and if that fails then I will try TIME_FORMAT as well. So is there any performance impact if I use separate datetime.xml for app?

0 Karma

DavidHourani
Super Champion

Yeap you can also do it with datetime.xml it's just as good as TIME_FORMAT, I thought you were looking for an easy solution that's why I didn't mention datetime.xml, if you have that working stick to it ^^

0 Karma

ips_mandar
Builder

HI @DavidHourani ,
I want one day before of extracted date as _time How I can achieve this?
like from source I have extracted as 10 June then I want _time as 09 June..

0 Karma

DavidHourani
Super Champion

You can do this at search time with something like this : ...| eval _time=_time-86400
Or save that as a calculated field to have it permanently applied at search time.
Best would be to go back to IngestEval it's the only way to apply an eval at index time..
Or if you can manipulate your filenames, then fix it to day -1 🙂

0 Karma

ips_mandar
Builder

Thanks @DavidHourani
My current problem:-
my event looks like-

31,04:56:47:928,  abc:0xabc,  49.716720, -59.271553,197

where 31 is day of month
and source field looks like - D:\\abc\\def\\XYLog09229190601.txt.zip:.\\XYLog09229190601.txt
So by which way I can get _time as 31 May 2019?

0 Karma

ips_mandar
Builder

and for same source some events looks like

30,04:56:47:928,  abc:0xabc,  49.716720, -59.271553,197

For which I want _time as 30 May 2019

0 Karma

DavidHourani
Super Champion

ouch, so same source, different ways to extract time, is that what you mean ? coz for both events you posted it seems you can get everything from the event itself, except the month, right ?

0 Karma

DavidHourani
Super Champion
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...