Getting Data In
Highlighted

Set timestamp based on file source path

Communicator

Hi all,

I'm trying to set the timestamp for events from my source. My paths look like this:

C:\Users\angeliga\Filer\336033\gelica_2013-03-06_13-48-45\Server\file_to_index.txt

I have read some answers on this subject here at splunk-base and on some other places.
The suggestions that I've come across are to copy datetime.xml and modify it (from this splunk-base answer), or to do it in transforms.conf (from this splunk-base answer)

But I can't get it to work!

It seems to me that the easiest way would be to use transforms.conf, but I can't figure out how to set the field correctly..

I've also followed the exmples on how to modify datetime.xml, but when it looks like below, I get no events of that mysrctype! To figure out if I did something wrong when editing datetime.xml, I tried to just copy (no editing) it into my local folder and then set DATETIMECONFIG = /etc/system/local/datetime.xml but it doesn't matter, I still get no events of mysrc_type...

[my_src_type]
DATETIME_CONFIG = /etc/system/local/datetime.xml
other sourcetype stuff...

I would also be able to extract the date, but I'm thinking that it would be the same approach?

I hope someone can help me with this, it is very frustrating that I'm not able to make it work.

0 Karma
Highlighted

Re: Set timestamp based on file source path

Builder

Add this line to props.conf file and extract the date from the directory name

EXTRACT-sourcefields = \Users\angeliga\Filer\336033\gelica_(?<the_date>.*)\Server\file_to_index.txt in source
0 Karma
Highlighted

Re: Set timestamp based on file source path

Communicator

Thanks for your answer, but I'm looking for a way to do this at index time, and make it the timestamp of the events in order to be able to use timechart and stuff easily.

0 Karma
Highlighted

Re: Set timestamp based on file source path

Legend

Did you check splunkd.log for any errors related to this time extraction? The timestamp processor is usually pretty good at telling why it's failing for one reason or another. Also I'm assuming you've read this docs page: http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

Highlighted

Re: Set timestamp based on file source path

Communicator

In case someone else have this problem, I didn't manage to get it working by using datetime.xml..
Instead I used EVAL in props.conf:

EVAL-_time=strptime(file_name, "%Y-%m-%d_%H-%M-%S")

Probably not the most efficient way to do this, but it works for me for now.

I'm still open to try another way if anyone has any solution.

View solution in original post

Highlighted

Re: Set timestamp based on file source path

Communicator

Hi @gelica. I am currently having this same problem. I want the timestamp of the events of my log to be the timestamp on its filename. I see you have managed to do this and I have a question in your config. I tried your config here's mine: EVAL-time=strptime(filename, "%m-%d-%Y") and my filename is this: MTYP0-09-26-2013.log. I can't get the timestamp of the file. Hope you can help me on this

0 Karma
Highlighted

Re: Set timestamp based on file source path

Communicator

@crt89 I'm not sure, and I'm not able to test since I'm not in that project anymore.
The only thing that comes to my mind is that maybe file_name isn't what you think it is, have you double checked that?
Good luck

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.