Getting Data In

How to measure Execution Latency of Ad-hoc Searches?

PowerPacked
Builder

Hi Everyone

How to determine and measure if any Ad hoc Searches are getting queued and by what time on total?
Basically, I would like to have a panel showing the execution latency of Ad-Hoc searches.

Thanks in advance

0 Karma
1 Solution

adonio
Ultra Champion

hello there,

thank you for the question, got me thinking a lot about it.
i am not 100% sure of the solution, but here is what i got:
looks like, splunk generates the values for search_id field based on the time you pressed Enter or hit the Search icon
values are in epoch with milliseconds, with single quotes around example: '1522959775.983'
also, Splunk captures the execution time, in the field exec_time - epoch without milliseconds
i think my search might need some more filtering but hopefully its a good start for you:

index="_audit" action=search  NOT search_id=*scheduler* NOT saved_search=*
| rex field=search_id "'(?<s_id>\d+\.\d{3})'"
| table _time s_id exec_time total_run_time
| eval ad_hoc_latency = exec_time - s_id

see screenshot below:
alt text

hope it helps

View solution in original post

adonio
Ultra Champion

hello there,

thank you for the question, got me thinking a lot about it.
i am not 100% sure of the solution, but here is what i got:
looks like, splunk generates the values for search_id field based on the time you pressed Enter or hit the Search icon
values are in epoch with milliseconds, with single quotes around example: '1522959775.983'
also, Splunk captures the execution time, in the field exec_time - epoch without milliseconds
i think my search might need some more filtering but hopefully its a good start for you:

index="_audit" action=search  NOT search_id=*scheduler* NOT saved_search=*
| rex field=search_id "'(?<s_id>\d+\.\d{3})'"
| table _time s_id exec_time total_run_time
| eval ad_hoc_latency = exec_time - s_id

see screenshot below:
alt text

hope it helps

splunk_arz
Explorer

Thank you very much!
I was looking for such a solution 🙂

I endet up with this one:

index="_audit" action=search  NOT search_id=*scheduler* NOT saved_search=*
| rex field=search_id "'.*_(?<s_id>\d+\.\d+)'"
| eval ad_hoc_latency = round(exec_time - s_id, 3)
| eval ad_hoc_latency = max(ad_hoc_latency,0)
| table _time s_id exec_time total_run_time, ad_hoc_latency
| where ad_hoc_latency>0
| eval Description=case(ad_hoc_latency>0 AND ad_hoc_latency<=0.5,"0-0.5", ad_hoc_latency>0.5 AND ad_hoc_latency<=2,"0.5-2", ad_hoc_latency>2 AND ad_hoc_latency<=5,"2-5", ad_hoc_latency>5 AND ad_hoc_latency<=15,"5-15",ad_hoc_latency>15,">15")
| timechart span=10m count by Description
0 Karma

deepashri_123
Motivator

Hey@PowerPacked,

Have a look at this accepted answer:
https://answers.splunk.com/answers/583285/how-to-list-ad-hocscheduled-searches-in-order-of-c.html
Let me know if this helps!!

0 Karma

PowerPacked
Builder

Thanks for reply @deepashri_123

I know there are couple of searches in DMC about most memory consuming & long running, But i am specifically interested in Execution Latency of Ad - Hoc Searches as i want to monitor the latency.

Even there is a search in DMC about execution latency of Scheduled Searches, but not about ad-hoc.
& am not sure if splunk is writing fields like Search Dispatch time & Search Start time about Ad -hoc Searches.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...