topic Re: Timestamp from file name in Getting Data In

Timestamp from file name

hartfoml — Tue, 07 Feb 2012 22:34:15 GMT

I have file names like this "Patch-Data_2-1-2012.csv"

How do I use the date in the file name for the datestamp for all the lines/events in the file

This is a one shot import

Re: Timestamp from file name

hexx — Mon, 28 Sep 2020 10:24:09 GMT

Here's what you need to do :

Write your own regular expression which will extract the month, the day and the year from your event source each in its own capture group.
Include your regex in a custom datetime.xml file. You should use $SPLUNK_HOME/etc/datetime.xml as a reference for the format to use for your own custom datetime.xml
Set up DATETIME_CONFIG in props.conf so that it would use your custom datetime.xml file.

Just to get you started, here's what you'll probably want to have in your own instance of datetime.xml to extract dates from your file source :

<![CDATA[(?:^|source::).*?_(0?[1-9]|1[012])-(0?[1-9]|[12]\d|3[01])-(20\d\d|19\d\d|[901]\d(?!\d))\.csv]]> (...) (...)

The easy way to go about this would be to copy $SPLUNK_HOME/etc/datetime.xml to your custom app and then add the regular expression definition at the end of the existing ones, and reference it under as shown above.

Re: Timestamp from file name

asimagu — Wed, 31 Jul 2013 03:39:38 GMT

hi guys, I have tried replicating this same config and filename on Splunk 5.0.2 but it is not working for me

any known issues on Splunk 5 about this??

Re: Timestamp from file name

gelica — Wed, 31 Jul 2013 05:15:01 GMT

@asimagu
Hi, I've had the same problem, couldn't get it to work at all, Splunk used the modification date instead.

I ended up with overriding _time by using EVAL in props.conf. Not the most elegant solution, but it works for now.

However, it would be nicer to do it at index time as it should, so I really hope you fint a solution that works for me as well 🙂

Re: Timestamp from file name

kiddo258 — Thu, 22 Jan 2015 16:04:59 GMT

It seems a time is necessary at least.

"If no events in a source have a date, Splunk Enterprise tries to find one in the source name or file name. (This requires that the events have a time, even though they don't have a date.)"

Re: Timestamp from file name

duncanzhang — Wed, 30 Sep 2020 01:20:42 GMT

props.conf:::::
[mysourcetype]
TRANSFORMS-change_time= change_time

transforms.conf:::::
[change_time]
INGEST_EVAL = _time = strptime ( replace(source,".+(\d+-\d+-\d+).+", "\1"), "%H-%M-%S")