topic Re: How long does a search take in Splunk Search

How long does a search take

rmorlen — Mon, 12 Nov 2012 22:16:33 GMT

Using a Splunk query, how can I tell how long searches are taking? I know I can inspect a search so the information is available. Where can I find it?

Re: How long does a search take

Damien_Dallimor — Tue, 13 Nov 2012 03:59:30 GMT

As of Splunk 5 , you can enter the search query :

| history

This will show you the searches that have been run and stats for the searches ie: total_run _time

Re: How long does a search take

rmorlen — Tue, 13 Nov 2012 15:31:52 GMT

We are not running 5.0.

I am looking more in general. I would like to baseline a search and then also baseline all searches so that we can determine if we are having Splunk performance issues.

So if I come up with a general search (like "index=* earliest=-15s") then determine how long it took to run.

Also do this for all searches so look at the average, median, and max time is takes to do all searches and see if searches are running normal, faster, or slower than something like yesterday or the same time last week.

Thanks.

Re: How long does a search take

yannK — Tue, 13 Nov 2012 15:36:21 GMT

After the search run, you can check the search inspector under the "i" button.
you will have a lot of details about the search speed and results.

Or afterward, check the index=_audit for some search performance metrics on savedsearches names or search id.

Re: How long does a search take

rmorlen — Mon, 28 Sep 2020 12:47:56 GMT

Thanks. This helps:

index=_audit earliest=-5m savedsearch_name=* | eval searchStartTime=strptime(apiStartTime, "'%a %B %d %H:%M:%S %Y'") | eval searchEndTime=strptime(apiEndTime, "'%a %B %d %H:%M:%S %Y'") | eval searchExecuteTime=_time | eval deltaFromEnd=searchExecuteTime - searchStartTime | timechart span=1m max(deltaFromEnd) min(deltaFromEnd) avg(deltaFromEnd)

Re: How long does a search take

yannK — Tue, 13 Nov 2012 17:45:30 GMT

The SOS app has some interesting dashboards on it too.

Or I use thinks like

index=_audit total_run_time | convert num(total_run_time) | eval event_per_sec=scan_count/total_run_time | stats count median(event_per_sec) AS median avg(event_per_sec) AS avg perc95(event_per_sec) AS perc95 max(total_run_time) AS maxruntime max(scan_count) AS scancount by search_id

Re: How long does a search take

Damien_Dallimor — Tue, 13 Nov 2012 20:31:18 GMT

The Splunk on Splunk(SoS) app is your friend 🙂

Re: How long does a search take

Michael_Schyma1 — Tue, 13 Nov 2012 20:36:30 GMT

Splunk 5 does not seem to have enough documentation yet. We (my company) might wait to upgrade until it is more readily available.

Re: How long does a search take

Damien_Dallimor — Tue, 13 Nov 2012 20:41:31 GMT

Splunk 5 is slick, you'll be glad you upgraded when you do.

FYI : the SoS app is also available for prior Splunk versions.Very nice for diagnosing search performance.