All Apps and Add-ons

SOS - How Do I Configure for a Distributed Environment?

gbowden_pheaa
Path Finder

I have multiple search heads and multiple indexers in a cluster. I've deployed SOS to the cluster master (also the deployment server and license manager), and to all search heads. I've deployed the TA-SOS to all indexers, enabled scripting on all servers, and restarted the entire environment.

All _internal data is being sent to the cluster.

On my master node, SOS shows only itself and the search peers. This is the same for the rest of the search heads.

I want to be able to see everything from my master node. Am I hoping for too much, or have a missed something in the configurations?

Thanks...

1 Solution

gbowden_pheaa
Path Finder

I've made some headway -

I edited the /$SPLUNK_APP/etc/apps/sos/lookups/splunk_servers_cache.csv file by adding the additional search heads into the list. SOS does not report the server hardware and OS statistics, but I can now see the search statistics from my master node.

That's mostly want I wanted.

View solution in original post

gbowden_pheaa
Path Finder

I've made some headway -

I edited the /$SPLUNK_APP/etc/apps/sos/lookups/splunk_servers_cache.csv file by adding the additional search heads into the list. SOS does not report the server hardware and OS statistics, but I can now see the search statistics from my master node.

That's mostly want I wanted.

hexx
Splunk Employee
Splunk Employee

This is indeed how one should proceed. Instance auto-discovery in S.o.S piggy-backs on distributed search, which is why search-heads cannot discover each other but only their own search peers.

The lack of instance details is expected as well for those instances as again, we rely on distributed search to collect those.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...

span_metrics: The OpenTelemetry-Idiomatic Way to See Inside Your Services

You open a trace in Splunk Observability Cloud and everything looks fine. One root span, order-pipeline, with ...