Splunk Search

how does search head pooling work with scheduled searches?

tpsplunk
Communicator

i like the idea of search head pooling with respect to ease of managing configs across multiple search heads. but i'm a little confused as to how scheduled searches are divvied between the search heads in the pool. is there any logic behind it? will it work intelligently if i have non-homogeneous HW for my search heads?

1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Whichever search head happens to acquire a lock on a job instance first, they'll run it. So, no, there isn't a lot of intelligence behind it. If a machine is running slower (and assuming clocks are all in sync), then it will be slightly less likely to pick up a job, but that's about it.

View solution in original post

rmorlen
Splunk Employee
Splunk Employee

We have 5 seachheads in our pool. 1 is dedicated to running scheduled searches. The others do NOT run scheduled searches. On the searchheads that you DON'T want to run scheduled searches, in .../splunk/etc/system/local in default-mode.conf add:
[pipeline:scheduler]
disabled = true

We had to do this locally on each searchhead because "disabled=false" doesn't work.

Basically you are setting up a single server as a Splunk jobs server.

gkanapathy
Splunk Employee
Splunk Employee

Whichever search head happens to acquire a lock on a job instance first, they'll run it. So, no, there isn't a lot of intelligence behind it. If a machine is running slower (and assuming clocks are all in sync), then it will be slightly less likely to pick up a job, but that's about it.

tpsplunk
Communicator

are there any plans on the roadmap to set or influence which search head(s) in the search head pool run scheduled searches? I'd like to do this so i can have search head dedicated for ad-hoc searches and others for scheduled searches and still get the ease of maintenance you get with pooling search heads.

Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...