Splunk Search

How to search information on usage for each search from all the different apps in our Splunk environment?

ECovell
Path Finder

Ladies and Gentlemen,

I have been tasked to write up a search that would give a total usage for each search from all the different apps. I have been digging through the master/deployment head and I have not found anything to work with. Should I be working directly with the search head where all the apps are running? If anyone could help with a good starting point I would be very grateful.

Thanks in advance,
Ernie,

0 Karma
1 Solution

Ravan
Path Finder

If you are looking for search run time, disk usage, no of runs kind of stuff by each app, You can use the searches below and start exploring the data and do further customization.

| rest /servicesNS/-/-/search/jobs | eval diskUsageMB=diskUsage/1024/1024 | rename eai:acl.owner as user,eai:acl.app AS app_name |search user!="splunk-system-user"  |eval search=if((isnull(label) OR label=""),sid,label)| stats sum(diskUsageMB) as totalDiskUsage by user app_name search |sort - totalDiskUsage

index=_audit source=audittrail info=completed search_id=* user=* NOT (user="splunk-system-user" OR user="n/a" OR search_id=rt_*  OR savedsearch_name="") |rex field=_raw "^.*\__{1}(?P.*)\_\_" |stats  count AS No_Runs,avg(total_run_time) AS avg_run_time by savedsearch_name splunk_server user app_name|eval avg_run_time=round(avg_run_time/60,2)|sort - No_Runs avg_run_time

View solution in original post

Ravan
Path Finder

If you are looking for search run time, disk usage, no of runs kind of stuff by each app, You can use the searches below and start exploring the data and do further customization.

| rest /servicesNS/-/-/search/jobs | eval diskUsageMB=diskUsage/1024/1024 | rename eai:acl.owner as user,eai:acl.app AS app_name |search user!="splunk-system-user"  |eval search=if((isnull(label) OR label=""),sid,label)| stats sum(diskUsageMB) as totalDiskUsage by user app_name search |sort - totalDiskUsage

index=_audit source=audittrail info=completed search_id=* user=* NOT (user="splunk-system-user" OR user="n/a" OR search_id=rt_*  OR savedsearch_name="") |rex field=_raw "^.*\__{1}(?P.*)\_\_" |stats  count AS No_Runs,avg(total_run_time) AS avg_run_time by savedsearch_name splunk_server user app_name|eval avg_run_time=round(avg_run_time/60,2)|sort - No_Runs avg_run_time

twinspop
Influencer

Trying to improve on this excellent search. I've got duration and delay stats, and count combined into 1 chart by user, app and saved search name:

((index=_audit source=audittrail) OR index=_internal) savedsearch_name sourcetype!=splunkd_remote_searches  sourcetype!=splunkd_access NOT (splunk_archiver OR "bucket copy trigger")| rex field=_raw "\s(?:(?:search_id)|(?:sid))=[\'\"]?(?<search_id>[^ \'\",]+)"   | stats  max(dispatch_time) as dtime max(scheduled_time) as stime max(total_run_time) AS total_run_time last(app) as app by search_id savedsearch_name host user | eval delay=dtime-stime | stats avg(total_run_time) as avgDur max(total_run_time) as maxDur sum(total_run_time) as totalDur avg(delay) as avgDelay count values(host) as hosts by app savedsearch_name user | sort - count

M2016G0216
Explorer

I'm getting the following error when I run twinspop search:

Error in 'rex' command: Encountered the following error while compiling the regex '\s(?:(?:search_id)|(?:sid))=[\'"]?(?<search_id>[^ \'",]+)': Regex: syntax error in subpattern name (missing terminator)
0 Karma

M2016G0216
Explorer

And, when I run the search suggested by Ravan, I get the following error:

Error in 'rex' command: Encountered the following error while compiling the regex '^.*\__{1}(?P.*)\_\_': Regex: unrecognized character after (?P
0 Karma

ECovell
Path Finder

Thank you very much, the first search gave me a lot of information to work with!!

0 Karma

Ravan
Path Finder

You are welcome.

Also you can try the below app which gives slimier trends for monitoring Splunk users and searches ..etc

https://splunkbase.splunk.com/app/3241/

0 Karma

somesoni2
Revered Legend

By usage do you mean license usage?
If yes then the license usage can be measured by index, host, sourcetype and source. Unless you've a way to map any of these values to an app, I don't think it's possible for measure license usage by app.

0 Karma

ECovell
Path Finder

No sir, I am not looking at license usage, my CSO is looking to clean up the searches and wants to know what searches are used and who is using them.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...