topic How to configure Splunk to set the event timestamp based on filename for date and events for time? in Getting Data In

How to configure Splunk to set the event timestamp based on filename for date and events for time?

ofaura — Tue, 29 Sep 2020 09:35:16 GMT

Hello,

I don't know if it is possible get this setup. I should load into Splunk a log file with lots of events, but I am not able to set up the timestamp in the right way. In the filename, I can seen the date and in the events the time as following:

Filename: LOG_14-07-09_1100.TST
Events sample:
11000000 RSM2 MC0210 pcs013 ....
11010500 SSM7 MC2020 pkt023 ....
11030500 KSF3 MC4010 pkt313 ....
11100100 TRW71 MC1010 pkt021 ....
11122000 WRM1 MC1020 pkt013 ....
11330200 TWM31 MC0410 pkt118 ....
So, the timestamp should be:
2014/07/09 - 11:00 AM
2014/07/09 - 11:01 AM
2014/07/09 - 11:03 AM
2014/07/09 - 11:10 AM
2014/07/09 - 11:12 AM
2014/07/09 - 11:33 AM

Any idea if this is possible? If so, how?

Thanks in advance,

Re: How to configure Splunk to set the event timestamp based on filename for date and events for time?

somesoni2 — Mon, 02 May 2016 16:58:41 GMT

Below link should give you require details to understand how the timestamp recognition works in Splunk.

http://docs.splunk.com/Documentation/Splunk/6.4.0/Data/HowSplunkextractstimestamps#How_Splunk_Enterprise_assigns_timestamps

Basically you need the processing of point 4 from above link. An untested suggestion would to set the TIME_FORMAT to a value which is not present in the event and let Splunk identify the date from file name and time from event.

Re: How to configure Splunk to set the event timestamp based on filename for date and events for time?

ofaura — Tue, 29 Sep 2020 09:39:35 GMT

Thanks for your answer, but it does not work. When I defined TIME_FORMAT to a value not present then Splunk applies the file mod date and time as the timestamp.

So, I have been working on something like:

TIME_FORMAT = %H%M%S%2N
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = ^\d{8}\s
NO_BINARY_CHECK = true

And this works with the time, Splunk identify the time but not the date, and as you said, the documentations says:

"4. If no events in a source have a date, Splunk Enterprise tries to find a date in the source name or file name. Time of day is not identified in filenames. (This requires that the events have a time, even though they don't have a date.) "

So, this should be working but it does not, any suggestion?

Thanks in advance,

Re: How to configure Splunk to set the event timestamp based on filename for date and events for time?

ofaura — Tue, 29 Sep 2020 09:39:59 GMT

Hello,

I have found a workaround that it´s working fine.

In props.conf:

TIME_FORMAT = %H%M%S%2N
DATETIME_CONFIG =
TZ = Europe/Madrid

And I have rename the file from LOG_14-07-09_1100.TST to LOG_20140709_1100.TST and now, Splunk takes the date from the filename and time from the events.

Oscar