How long does a search take

[rmorlen]

Using a Splunk query, how can I tell how long searches are taking? I know I can inspect a search so the information is available. Where can I find it?

[yannK · Accepted solution]

After the search run, you can check the search inspector under the "i" button.
you will have a lot of details about the search speed and results.

Or afterward, check the index=_audit for some search performance metrics on savedsearches names or search id.

[yannK]

The SOS app has some interesting dashboards on it too.

Or I use thinks like

index=_audit total_run_time | convert num(total_run_time) | eval event_per_sec=scan_count/total_run_time | stats count median(event_per_sec) AS median avg(event_per_sec) AS avg perc95(event_per_sec) AS perc95 max(total_run_time) AS maxruntime max(scan_count) AS scancount by search_id


[rmorlen]

Thanks. This helps:

index=_audit earliest=-5m savedsearch_name=* | eval searchStartTime=strptime(apiStartTime, "'%a %B %d %H:%M:%S %Y'") | eval searchEndTime=strptime(apiEndTime, "'%a %B %d %H:%M:%S %Y'") | eval searchExecuteTime=_time | eval deltaFromEnd=searchExecuteTime - searchStartTime | timechart span=1m max(deltaFromEnd) min(deltaFromEnd) avg(deltaFromEnd)

[Damien_Dallimor]

As of Splunk 5 , you can enter the search query :

| history

This will show you the searches that have been run and stats for the searches ie: total_run _time

[Damien_Dallimor]

Splunk 5 is slick, you'll be glad you upgraded when you do.

FYI : the SoS app is also available for prior Splunk versions.Very nice for diagnosing search performance.

[Michael_Schyma1]

Splunk 5 does not seem to have enough documentation yet. We (my company) might wait to upgrade until it is more readily available.

[Damien_Dallimor]

The Splunk on Splunk(SoS) app is your friend 🙂

[rmorlen]

We are not running 5.0.

I am looking more in general. I would like to baseline a search and then also baseline all searches so that we can determine if we are having Splunk performance issues.

So if I come up with a general search (like "index=* earliest=-15s") then determine how long it took to run.

Also do this for all searches so look at the average, median, and max time is takes to do all searches and see if searches are running normal, faster, or slower than something like yesterday or the same time last week.

Thanks.