Splunk Search

How to search information on usage for each search from all the different apps in our Splunk environment?

ECovell
Path Finder

Ladies and Gentlemen,

I have been tasked to write up a search that would give a total usage for each search from all the different apps. I have been digging through the master/deployment head and I have not found anything to work with. Should I be working directly with the search head where all the apps are running? If anyone could help with a good starting point I would be very grateful.

Thanks in advance,
Ernie,

0 Karma
1 Solution

Ravan
Path Finder

If you are looking for search run time, disk usage, no of runs kind of stuff by each app, You can use the searches below and start exploring the data and do further customization.

| rest /servicesNS/-/-/search/jobs | eval diskUsageMB=diskUsage/1024/1024 | rename eai:acl.owner as user,eai:acl.app AS app_name |search user!="splunk-system-user"  |eval search=if((isnull(label) OR label=""),sid,label)| stats sum(diskUsageMB) as totalDiskUsage by user app_name search |sort - totalDiskUsage

index=_audit source=audittrail info=completed search_id=* user=* NOT (user="splunk-system-user" OR user="n/a" OR search_id=rt_*  OR savedsearch_name="") |rex field=_raw "^.*\__{1}(?P.*)\_\_" |stats  count AS No_Runs,avg(total_run_time) AS avg_run_time by savedsearch_name splunk_server user app_name|eval avg_run_time=round(avg_run_time/60,2)|sort - No_Runs avg_run_time

View solution in original post

Ravan
Path Finder

If you are looking for search run time, disk usage, no of runs kind of stuff by each app, You can use the searches below and start exploring the data and do further customization.

| rest /servicesNS/-/-/search/jobs | eval diskUsageMB=diskUsage/1024/1024 | rename eai:acl.owner as user,eai:acl.app AS app_name |search user!="splunk-system-user"  |eval search=if((isnull(label) OR label=""),sid,label)| stats sum(diskUsageMB) as totalDiskUsage by user app_name search |sort - totalDiskUsage

index=_audit source=audittrail info=completed search_id=* user=* NOT (user="splunk-system-user" OR user="n/a" OR search_id=rt_*  OR savedsearch_name="") |rex field=_raw "^.*\__{1}(?P.*)\_\_" |stats  count AS No_Runs,avg(total_run_time) AS avg_run_time by savedsearch_name splunk_server user app_name|eval avg_run_time=round(avg_run_time/60,2)|sort - No_Runs avg_run_time

twinspop
Influencer

Trying to improve on this excellent search. I've got duration and delay stats, and count combined into 1 chart by user, app and saved search name:

((index=_audit source=audittrail) OR index=_internal) savedsearch_name sourcetype!=splunkd_remote_searches  sourcetype!=splunkd_access NOT (splunk_archiver OR "bucket copy trigger")| rex field=_raw "\s(?:(?:search_id)|(?:sid))=[\'\"]?(?<search_id>[^ \'\",]+)"   | stats  max(dispatch_time) as dtime max(scheduled_time) as stime max(total_run_time) AS total_run_time last(app) as app by search_id savedsearch_name host user | eval delay=dtime-stime | stats avg(total_run_time) as avgDur max(total_run_time) as maxDur sum(total_run_time) as totalDur avg(delay) as avgDelay count values(host) as hosts by app savedsearch_name user | sort - count

M2016G0216
Explorer

I'm getting the following error when I run twinspop search:

Error in 'rex' command: Encountered the following error while compiling the regex '\s(?:(?:search_id)|(?:sid))=[\'"]?(?<search_id>[^ \'",]+)': Regex: syntax error in subpattern name (missing terminator)
0 Karma

M2016G0216
Explorer

And, when I run the search suggested by Ravan, I get the following error:

Error in 'rex' command: Encountered the following error while compiling the regex '^.*\__{1}(?P.*)\_\_': Regex: unrecognized character after (?P
0 Karma

ECovell
Path Finder

Thank you very much, the first search gave me a lot of information to work with!!

0 Karma

Ravan
Path Finder

You are welcome.

Also you can try the below app which gives slimier trends for monitoring Splunk users and searches ..etc

https://splunkbase.splunk.com/app/3241/

0 Karma

somesoni2
SplunkTrust
SplunkTrust

By usage do you mean license usage?
If yes then the license usage can be measured by index, host, sourcetype and source. Unless you've a way to map any of these values to an app, I don't think it's possible for measure license usage by app.

0 Karma

ECovell
Path Finder

No sir, I am not looking at license usage, my CSO is looking to clean up the searches and wants to know what searches are used and who is using them.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...