Splunk Search

How to make the search dynamic - continuation of "How to use rex and sed to insert '-' and ':' in the result?"

nilotpaldutta
Explorer

I have different environments. In each environment logs are located in different path. e.g.:
C:\Program Files\Splunk...\etc\apps......\abc-20150626123001.log
/u01/splunk/...../etc/apps/...../def-20150626044921.log
/u01/log02/splunk/...../etc/apps/...../mno-20150626071656.log
/u02/splunk/...../etc/apps/...../xyz-20150626044921.log

I am using this query -

index="myindex" | dedup source | sort -source | dedup sourcetype | table sourcetype, source

It gives me the result but contains the complete log path. My requirement is to remove everything and display only the date and time e.g. 2015-06-26 12:30:01

Thanks for answering my previous question [@richgalloway] --> /answers.splunk.com/answers/243218/how-to-use-rex-and-sed-to-insert-and-in-the-result.html
I was able to capture the date and time by using sed and regular expression. e.g.:

index="myindex" | dedup source | sort -source | dedup sourcetype | rex field=source mode=sed "s/[^0-9]*//g" | rex field=source mode=sed "s/(.{4})(.{2})(.{2})/\1-\2-\3 /" | rex field=source mode=sed "s/(.{10})(.{3})(.{2})/\1 \2:\3:/" | table sourcetype, source

This worked only for log files present in "C:\Program Files...."

However it didn't work for other environments. I had to modify each time, e.g. when it's /u01, i used -

index="myindex" | dedup source | sort -source | dedup sourcetype | rex field=source mode=sed "s/01//" | rex field=source mode=sed "s/[^0-9]*//g" | rex field=source mode=sed "s/(.{4})(.{2})(.{2})/\1-\2-\3 /" | rex field=source mode=sed "s/(.{10})(.{3})(.{2})/\1 \2:\3:/" | table sourcetype, source

Basically I just need to capture the YYYYMMDDHHMMSS before .log and remove everything else. The app will be used by various teams. Each team will have their own folder structure. Is there a way to make the query work in any environment irrespective of the path?

Tags (2)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

How 'bout this?

... | rex field=source "(?<date>\d+)\.log$"

That will grab the last bunch of digits before the .log at the end.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I don't know what you're trying to achieve with your whole query, so this is just a guess:

| tstats count where index=myindex by source sourcetype | rex field=source "(?<date>\d+)\.log$
0 Karma

nilotpaldutta
Explorer

Can you please write the whole query?

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...