Deployment Architecture

Run saved searches on a single indexer

L_Petch
Path Finder

Hello,

 

A client went mad on how many saved searches they require and wont get rid of them. Due to this it is hammering rw on the indexers to the point the indexers can cope and remove themselves from the cluster and then re adds which and more resource strain.
adding more indexers isnt an option

The current setup is 3vm multisite search head cluster and a 4vm multisite indexer cluster.

 

As they only require 3rf and 3sf i am wondering if there is a way to use only 1SH and 1 Indexer for all saved searches to run so that the load doesnt affect the other 3 indexers?

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @L_Petch ,

you can use the Monitoring Console App to have this information.

Ciao.

Giuseppe

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Saved searches are scheduled across the whole search head cluster. (with some additional conditions like a cluster member being in detention). That's what search head is for. Also - limiting searches to just one SH would inevitably lead to delayed/skipped searches. It won't solve the performance issue.

Even if you have multiple indexers holding the same bucket, the indexers holding primary copies respond with results from those primaries - it's by design and lets you distribute the search. Even if you had a possibility to get results from just one indexer, there would be no guarantee that you'd get all events from given time range because with sf=rf=3 and 4 indexers you'd still probably hit (actually would _not_ hit) some buckets which are not present at that chosen indexer.

So your idea is not a very good one.

You can use site affinity to force search heads to use only one site. But again - especially if you already have performance problems - that's counterproductive.

And from experience - it's often not the _number_ of searches but _how_ they're written.

Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...