Splunk Search

Extract data from filename

Karthikeyan
Engager

Hi Experts,

I have a a job log file, that gets ingested to Splunk with naming convention "trace_08_19_2021_06_36_03_*********.txt". After trace, the format of the log file is _MM_DD_YYYY_HH_Min_Sec.****. Requirement is to extract the date and time from the file name and show the no. of the logs that get generated in an hour.

I read a blog which mentioned that Splunk extracts the timestamp from filename, but that's not happening with this case.

Kindly help with this case. Thanks in advance for any assistance.

Regards, Karthikeyan.SV

Labels (3)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Try this regex

trace_(?<sourceTime>\d\d_\d\d_\d\d\d\d_\d\d_\d\d_\d\d)
---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

Karthikeyan
Engager

I ran into an issue with the search command.

index=foo source="trace_*"
| rex field=source "trace_(?<sourceTime>[^\.]+)"
| eval _time=strptime(sourceTime, "%m_%d_%Y_%H_%M_%S")
| timechart span=1h count

When I use the above command, to see the data for 24 hrs, the count for each hour is displayed which is not a correct value. But when I tried to see events for particular hour, directly by accessing the events of an hour, the same search query was used without timechart and this provides increased no. of events which is actually the correct value to be displayed. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please post a new question for this problem.

---
If this reply helps you, Karma would be appreciated.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

See https://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps#How_Splunk_soft... for how Splunk assigns timestamps.  This happens at index time.  If it isn't happening then make sure the TIME_FORMAT setting is correct (%m_%d_%Y_%H_%M_%S).

You may be able to extract the times at search time.

 

index=foo source="trace_*"
| rex field=source "trace_(?<sourceTime>[^\.]+)"
| eval _time=strptime(sourceTime, "%m_%d_%Y_%H_%M_%S")
| timechart span=1h count

 

---
If this reply helps you, Karma would be appreciated.
0 Karma

Karthikeyan
Engager

Hi,

The proposed extract string worked. The file naming convention is "trace_mm_dd_yyyy_hh_MM_ss_<ramdom data>. txt, where random data like 1342_30bfb72c_9f70_4e32_ad3a_3066dd187240. The random data shown here is an example. The extract string we used extracts the data from filename after trace_ till the end before ".". Is it possible to extract the data between trace_ till _<random data>.txt ?

Karthikeyan

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this regex

trace_(?<sourceTime>\d\d_\d\d_\d\d\d\d_\d\d_\d\d_\d\d)
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...