Splunk Search

how does search head pooling work with scheduled searches?

tpsplunk
Communicator

i like the idea of search head pooling with respect to ease of managing configs across multiple search heads. but i'm a little confused as to how scheduled searches are divvied between the search heads in the pool. is there any logic behind it? will it work intelligently if i have non-homogeneous HW for my search heads?

1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Whichever search head happens to acquire a lock on a job instance first, they'll run it. So, no, there isn't a lot of intelligence behind it. If a machine is running slower (and assuming clocks are all in sync), then it will be slightly less likely to pick up a job, but that's about it.

View solution in original post

rmorlen
Splunk Employee
Splunk Employee

We have 5 seachheads in our pool. 1 is dedicated to running scheduled searches. The others do NOT run scheduled searches. On the searchheads that you DON'T want to run scheduled searches, in .../splunk/etc/system/local in default-mode.conf add:
[pipeline:scheduler]
disabled = true

We had to do this locally on each searchhead because "disabled=false" doesn't work.

Basically you are setting up a single server as a Splunk jobs server.

gkanapathy
Splunk Employee
Splunk Employee

Whichever search head happens to acquire a lock on a job instance first, they'll run it. So, no, there isn't a lot of intelligence behind it. If a machine is running slower (and assuming clocks are all in sync), then it will be slightly less likely to pick up a job, but that's about it.

tpsplunk
Communicator

are there any plans on the roadmap to set or influence which search head(s) in the search head pool run scheduled searches? I'd like to do this so i can have search head dedicated for ad-hoc searches and others for scheduled searches and still get the ease of maintenance you get with pooling search heads.

Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...