topic Set timestamp based on file source path in Getting Data In

Set timestamp based on file source path

gelica — Mon, 28 Sep 2020 14:18:51 GMT

Hi all,

I'm trying to set the timestamp for events from my source. My paths look like this:

C:\Users\angeliga\Filer\336033\gelica_2013-03-06_13-48-45\Server\file_to_index.txt

I have read some answers on this subject here at splunk-base and on some other places.
The suggestions that I've come across are to copy datetime.xml and modify it (from this splunk-base answer), or to do it in transforms.conf (from this splunk-base answer)

But I can't get it to work!

It seems to me that the easiest way would be to use transforms.conf, but I can't figure out how to set the field correctly..

I've also followed the exmples on how to modify datetime.xml, but when it looks like below, I get no events of that my_src_type! To figure out if I did something wrong when editing datetime.xml, I tried to just copy (no editing) it into my local folder and then set DATETIME_CONFIG = /etc/system/local/datetime.xml but it doesn't matter, I still get no events of my_src_type...

[my_src_type]
DATETIME_CONFIG = /etc/system/local/datetime.xml
other sourcetype stuff...

I would also be able to extract the date, but I'm thinking that it would be the same approach?

I hope someone can help me with this, it is very frustrating that I'm not able to make it work.

Re: Set timestamp based on file source path

royimad — Thu, 11 Jul 2013 11:36:52 GMT

Add this line to props.conf file and extract the date from the directory name

EXTRACT-sourcefields = \Users\angeliga\Filer\336033\gelica_(?<the_date>.*)\Server\file_to_index.txt in source

Re: Set timestamp based on file source path

gelica — Thu, 11 Jul 2013 11:58:24 GMT

Thanks for your answer, but I'm looking for a way to do this at index time, and make it the timestamp of the events in order to be able to use timechart and stuff easily.

Re: Set timestamp based on file source path

Ayn — Thu, 11 Jul 2013 12:02:14 GMT

Did you check splunkd.log for any errors related to this time extraction? The timestamp processor is usually pretty good at telling why it's failing for one reason or another. Also I'm assuming you've read this docs page: http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

Re: Set timestamp based on file source path

gelica — Fri, 19 Jul 2013 07:42:41 GMT

In case someone else have this problem, I didn't manage to get it working by using datetime.xml..
Instead I used EVAL in props.conf:

EVAL-_time=strptime(file_name, "%Y-%m-%d_%H-%M-%S")

Probably not the most efficient way to do this, but it works for me for now.

I'm still open to try another way if anyone has any solution.

Re: Set timestamp based on file source path

crt89 — Mon, 28 Sep 2020 16:01:58 GMT

Hi @gelica. I am currently having this same problem. I want the timestamp of the events of my log to be the timestamp on its filename. I see you have managed to do this and I have a question in your config. I tried your config here's mine: EVAL-_time=strptime(file_name, "%m-%d-%Y") and my filename is this: MTYP0-09-26-2013.log. I can't get the timestamp of the file. Hope you can help me on this

Re: Set timestamp based on file source path

gelica — Tue, 04 Mar 2014 12:03:17 GMT

@crt89 I'm not sure, and I'm not able to test since I'm not in that project anymore.
The only thing that comes to my mind is that maybe file_name isn't what you think it is, have you double checked that?
Good luck