Splunk Search

How to make the search dynamic - continuation of "How to use rex and sed to insert '-' and ':' in the result?"

nilotpaldutta
Explorer

I have different environments. In each environment logs are located in different path. e.g.:
C:\Program Files\Splunk...\etc\apps......\abc-20150626123001.log
/u01/splunk/...../etc/apps/...../def-20150626044921.log
/u01/log02/splunk/...../etc/apps/...../mno-20150626071656.log
/u02/splunk/...../etc/apps/...../xyz-20150626044921.log

I am using this query -

index="myindex" | dedup source | sort -source | dedup sourcetype | table sourcetype, source

It gives me the result but contains the complete log path. My requirement is to remove everything and display only the date and time e.g. 2015-06-26 12:30:01

Thanks for answering my previous question [@richgalloway] --> /answers.splunk.com/answers/243218/how-to-use-rex-and-sed-to-insert-and-in-the-result.html
I was able to capture the date and time by using sed and regular expression. e.g.:

index="myindex" | dedup source | sort -source | dedup sourcetype | rex field=source mode=sed "s/[^0-9]*//g" | rex field=source mode=sed "s/(.{4})(.{2})(.{2})/\1-\2-\3 /" | rex field=source mode=sed "s/(.{10})(.{3})(.{2})/\1 \2:\3:/" | table sourcetype, source

This worked only for log files present in "C:\Program Files...."

However it didn't work for other environments. I had to modify each time, e.g. when it's /u01, i used -

index="myindex" | dedup source | sort -source | dedup sourcetype | rex field=source mode=sed "s/01//" | rex field=source mode=sed "s/[^0-9]*//g" | rex field=source mode=sed "s/(.{4})(.{2})(.{2})/\1-\2-\3 /" | rex field=source mode=sed "s/(.{10})(.{3})(.{2})/\1 \2:\3:/" | table sourcetype, source

Basically I just need to capture the YYYYMMDDHHMMSS before .log and remove everything else. The app will be used by various teams. Each team will have their own folder structure. Is there a way to make the query work in any environment irrespective of the path?

Tags (2)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

How 'bout this?

... | rex field=source "(?<date>\d+)\.log$"

That will grab the last bunch of digits before the .log at the end.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I don't know what you're trying to achieve with your whole query, so this is just a guess:

| tstats count where index=myindex by source sourcetype | rex field=source "(?<date>\d+)\.log$
0 Karma

nilotpaldutta
Explorer

Can you please write the whole query?

0 Karma
Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...