Deployment Architecture

Search Head Clustering: How to test if work load and user sessions are being distributed across servers?

Champion

Hi,

I've setup SHC across a few servers. How do I test the following in Splunk:

1) User sessions are being distributed across the servers?
2) Work load is being distributed?

0 Karma

SplunkTrust
SplunkTrust

1) Start a user session, maybe run a search... then kill the machine you were on. Get switched to another machine by your LB proxy, verify those search results, newly created dashboard, or whatever you were doing is there.

2) Schedule a bunch of expensive, frequently-run searches on one machine. See that all machines get busy, each search gets executed exactly once per intended run, that each alert action was triggered exactly once per intended triggering, and so on.

SplunkTrust
SplunkTrust

You could run this:

| tstats count where index=_internal AND sourcetype=scheduler by _time host prestats=t | timechart count by host

That's not an accurate count of scheduled executions because it also counts "continued" events, but it's a good Q'n'D approximation that tells you if load is sent to multiple SHs or just one.

0 Karma

Champion

Where can I check that each server is getting busy? I don't want to login to each server individually.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!