Extract data from filename

[Karthikeyan]

Hi Experts,

I have a a job log file, that gets ingested to Splunk with naming convention "trace_08_19_2021_06_36_03_*********.txt". After trace, the format of the log file is _MM_DD_YYYY_HH_Min_Sec.****. Requirement is to extract the date and time from the file name and show the no. of the logs that get generated in an hour.

I read a blog which mentioned that Splunk extracts the timestamp from filename, but that's not happening with this case.

Kindly help with this case. Thanks in advance for any assistance.

Regards, Karthikeyan.SV

[Karthikeyan]

I ran into an issue with the search command.

index=foo source="trace_*"
| rex field=source "trace_(?<sourceTime>[^\.]+)"
| eval _time=strptime(sourceTime, "%m_%d_%Y_%H_%M_%S")
| timechart span=1h count

When I use the above command, to see the data for 24 hrs, the count for each hour is displayed which is not a correct value. But when I tried to see events for particular hour, directly by accessing the events of an hour, the same search query was used without timechart and this provides increased no. of events which is actually the correct value to be displayed. 

[richgalloway]

Please post a new question for this problem.

[richgalloway]

See https://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps#How_Splunk_soft... for how Splunk assigns timestamps.  This happens at index time.  If it isn't happening then make sure the TIME_FORMAT setting is correct (%m_%d_%Y_%H_%M_%S).

You may be able to extract the times at search time.

index=foo source="trace_*"
| rex field=source "trace_(?<sourceTime>[^\.]+)"
| eval _time=strptime(sourceTime, "%m_%d_%Y_%H_%M_%S")
| timechart span=1h count

[Karthikeyan]

Hi,

The proposed extract string worked. The file naming convention is "trace_mm_dd_yyyy_hh_MM_ss_<ramdom data>. txt, where random data like 1342_30bfb72c_9f70_4e32_ad3a_3066dd187240. The random data shown here is an example. The extract string we used extracts the data from filename after trace_ till the end before ".". Is it possible to extract the data between trace_ till _<random data>.txt ?

Karthikeyan

[richgalloway · Accepted solution]

Try this regex

trace_(?<sourceTime>\d\d_\d\d_\d\d\d\d_\d\d_\d\d_\d\d)