S.o.S - No details for some instances in Splunk Topology

[xzjc3q]

I've just installed Splunk on Splunk version 3.1.0 - build 182161 on Splunk 5.0.7.

I have a distributed environment consisting of:

• 2 search heads
• 2 indexers / search peers
• 8 Heavy forwarders
• 1 Deployment Server/License Master

S.o.S is installed on the search heads and TA-sos on the remaining instances.

When looking in S.o.S on one of the search heads at the Splunk Topology view, why are the instance details not listed for all instances displayed (I get details only for the search head and search peers, although resource usage works)?

[thinrope]

I think I am in the same situation (just number of hosts is different). I can see detailed CPU usage for each of the hosts (since _internal is forwarded to the IDX cluster), but I don't see some of the details inside "A glimpse of your Splunk Enterprise instances" panel. I tried to understand how it pulls those, but am giving up 😐

The SH that has the app and all IDX in the cluster show all details, but other SH, LM, DS, HF, etc. show ONLY Version and Platform. I'd like to see $SPLUNK_HOME and number of cores, etc for all instances.

Where is this data taken from, is it based on (forwarded) logs? If so can someone show a search with no macros?
Or is something pulled via the REST API?

Splunk-6.2.2 / S.o.S-3.2.1 here

[hexx]

The data in that panel relies on the ability to run a search directly against the instance selected. As such, this panel is not expected to work for instances that are note search peers of the S.o.S search-head.

[thinrope]

I see..

Well, then that is a RFE:

Include a script that runs say every 24h on each host and collects those parameters and use standard log forwarding techniques. None of them is expected to change much anyway, so direct query does not make sense, IMHO.

[hexx · Accepted solution]

The S.o.S app is only able to fetch details for other instances which are accessible by distributed search. This is not the case for forwarders, and typically not the case for your Deployment Server/License Master instance, as well as for your other search-head.

Regarding forwarders, we intend to display alternative details (Splunk version, forwarder type, platform) in the future.

For non-forwarder instances, you have a few options:

• Live with the lack of details for these instances
• Install a copy of the S.o.S app on these instances, let it populate the "splunk_instances_info" and "splunk_servers_cache" lookups with the appropriate entries and then manually merge those to the instances of these lookup files on your search-head(s)
• Install a copy of the S.o.S app on these instances and consult details / other views with a local instance scope that way

The second method is the one we recommend, along with configuring those standalone instances to forward their events to your indexers.

You can find more details on how to manage S.o.S' asset tables manually in a distributed environment in the "Learn More" panel of the Deployment Topology view.

[xzjc3q]

I will have a look at implementing the second option. Thank you.