Search Head Clustering: How to test if work load and user sessions are being distributed across servers?

[a212830]

Hi,

I've setup SHC across a few servers. How do I test the following in Splunk:

1) User sessions are being distributed across the servers?
2) Work load is being distributed?

[martin_mueller]

1) Start a user session, maybe run a search... then kill the machine you were on. Get switched to another machine by your LB proxy, verify those search results, newly created dashboard, or whatever you were doing is there.

2) Schedule a bunch of expensive, frequently-run searches on one machine. See that all machines get busy, each search gets executed exactly once per intended run, that each alert action was triggered exactly once per intended triggering, and so on.

[martin_mueller]

You could run this:

| tstats count where index=_internal AND sourcetype=scheduler by _time host prestats=t | timechart count by host

That's not an accurate count of scheduled executions because it also counts "continued" events, but it's a good Q'n'D approximation that tells you if load is sent to multiple SHs or just one.

[a212830]

Where can I check that each server is getting busy? I don't want to login to each server individually.