Deployment Architecture

Run saved searches on a single indexer

L_Petch
Explorer

Hello,

 

A client went mad on how many saved searches they require and wont get rid of them. Due to this it is hammering rw on the indexers to the point the indexers can cope and remove themselves from the cluster and then re adds which and more resource strain.
adding more indexers isnt an option

The current setup is 3vm multisite search head cluster and a 4vm multisite indexer cluster.

 

As they only require 3rf and 3sf i am wondering if there is a way to use only 1SH and 1 Indexer for all saved searches to run so that the load doesnt affect the other 3 indexers?

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @L_Petch ,

you can use the Monitoring Console App to have this information.

Ciao.

Giuseppe

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Saved searches are scheduled across the whole search head cluster. (with some additional conditions like a cluster member being in detention). That's what search head is for. Also - limiting searches to just one SH would inevitably lead to delayed/skipped searches. It won't solve the performance issue.

Even if you have multiple indexers holding the same bucket, the indexers holding primary copies respond with results from those primaries - it's by design and lets you distribute the search. Even if you had a possibility to get results from just one indexer, there would be no guarantee that you'd get all events from given time range because with sf=rf=3 and 4 indexers you'd still probably hit (actually would _not_ hit) some buckets which are not present at that chosen indexer.

So your idea is not a very good one.

You can use site affinity to force search heads to use only one site. But again - especially if you already have performance problems - that's counterproductive.

And from experience - it's often not the _number_ of searches but _how_ they're written.

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...