Splunk Search

How long does a search take

rmorlen
Splunk Employee
Splunk Employee

Using a Splunk query, how can I tell how long searches are taking? I know I can inspect a search so the information is available. Where can I find it?

Tags (2)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

After the search run, you can check the search inspector under the "i" button.
you will have a lot of details about the search speed and results.

Or afterward, check the index=_audit for some search performance metrics on savedsearches names or search id.

View solution in original post

yannK
Splunk Employee
Splunk Employee

After the search run, you can check the search inspector under the "i" button.
you will have a lot of details about the search speed and results.

Or afterward, check the index=_audit for some search performance metrics on savedsearches names or search id.

yannK
Splunk Employee
Splunk Employee

The SOS app has some interesting dashboards on it too.

Or I use thinks like

index=_audit total_run_time | convert num(total_run_time) | eval event_per_sec=scan_count/total_run_time | stats count median(event_per_sec) AS median avg(event_per_sec) AS avg perc95(event_per_sec) AS perc95 max(total_run_time) AS maxruntime max(scan_count) AS scancount by search_id

0 Karma

rmorlen
Splunk Employee
Splunk Employee

Thanks. This helps:

index=_audit earliest=-5m savedsearch_name=* | eval searchStartTime=strptime(apiStartTime, "'%a %B %d %H:%M:%S %Y'") | eval searchEndTime=strptime(apiEndTime, "'%a %B %d %H:%M:%S %Y'") | eval searchExecuteTime=_time | eval deltaFromEnd=searchExecuteTime - searchStartTime | timechart span=1m max(deltaFromEnd) min(deltaFromEnd) avg(deltaFromEnd)

0 Karma

Damien_Dallimor
Ultra Champion

As of Splunk 5 , you can enter the search query :

| history

This will show you the searches that have been run and stats for the searches ie: total_run _time

0 Karma

Damien_Dallimor
Ultra Champion

Splunk 5 is slick, you'll be glad you upgraded when you do.

FYI : the SoS app is also available for prior Splunk versions.Very nice for diagnosing search performance.

0 Karma

Michael_Schyma1
Contributor

Splunk 5 does not seem to have enough documentation yet. We (my company) might wait to upgrade until it is more readily available.

0 Karma

Damien_Dallimor
Ultra Champion

The Splunk on Splunk(SoS) app is your friend 🙂

0 Karma

rmorlen
Splunk Employee
Splunk Employee

We are not running 5.0.

I am looking more in general. I would like to baseline a search and then also baseline all searches so that we can determine if we are having Splunk performance issues.

So if I come up with a general search (like "index=* earliest=-15s") then determine how long it took to run.

Also do this for all searches so look at the average, median, and max time is takes to do all searches and see if searches are running normal, faster, or slower than something like yesterday or the same time last week.

Thanks.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...