Splunk Cloud Platform

How can I solve for skipped searches?

uagraw01
Builder

Hello Splunkers !!

I have attached below two screenshot related to skip searches. As per the below graph many times we have high number of skip searches. When I validated those I seen that workload_pool are not assigned to many saved searched ( attached in second screenshot ).

My thought here :
Because If so many searches are triggering on the same time and there is no workload_pool setting assigned then it will impact in the search performance and increase the value of skip ratio.

Please let me know I am thinking on a right way ? If not please guide me or suggest me some good workarounds. I know there many blogs available on this. But please do share , if any specific suggestion on this.

uagraw01_0-1670256613772.png

Labels (1)
0 Karma

christhianb
New Member

Hey @uagraw01 

There are different ways to fix it but everything depends on the reason of the skipped search. 

You can run index=_internal sourcetype=scheduler status=skipped | stats values(reason) by savedsearch_name

That should help you out.

Once you identity the reason, make decisions. i.e disable unnecessary alerts, reduce the Time range picker, improve the SPL. This could be a fix for the most common reason " Max Concurrent searches have been reached..." 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @uagraw01,

if you're using an on-premise installation, probably your hardware isn't sufficient to work all the scheduled searches you have.

Which reference hardware are you using? how many scheduled searches?

Ciao.

Giuseppe

0 Karma

uagraw01
Builder

@gcuselloIts Splunk Cloud, and there 40+ saved searches which are showing with no workload_pool

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @uagraw01,

which kind of license are you using: indexed logs or SVC?

if SVC probably you are exceeding your license.

In this case ask to you Splunk partner.

Ciao.

Giuseppe

0 Karma

uagraw01
Builder

@gcuselloCan't we control with putting some new admission rule in workload management ?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @uagraw01,

you could reduce your scheduled searches,

Did you checked license and hardware?

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...