All Apps and Add-ons

S.o.S - No details for some instances in Splunk Topology

xzjc3q
Explorer

I've just installed Splunk on Splunk version 3.1.0 - build 182161 on Splunk 5.0.7.

I have a distributed environment consisting of:

  • 2 search heads
  • 2 indexers / search peers
  • 8 Heavy forwarders
  • 1 Deployment Server/License Master

S.o.S is installed on the search heads and TA-sos on the remaining instances.

When looking in S.o.S on one of the search heads at the Splunk Topology view, why are the instance details not listed for all instances displayed (I get details only for the search head and search peers, although resource usage works)?

1 Solution

hexx
Splunk Employee
Splunk Employee

The S.o.S app is only able to fetch details for other instances which are accessible by distributed search. This is not the case for forwarders, and typically not the case for your Deployment Server/License Master instance, as well as for your other search-head.

Regarding forwarders, we intend to display alternative details (Splunk version, forwarder type, platform) in the future.

For non-forwarder instances, you have a few options:

  • Live with the lack of details for these instances
  • Install a copy of the S.o.S app on these instances, let it populate the "splunk_instances_info" and "splunk_servers_cache" lookups with the appropriate entries and then manually merge those to the instances of these lookup files on your search-head(s)
  • Install a copy of the S.o.S app on these instances and consult details / other views with a local instance scope that way

The second method is the one we recommend, along with configuring those standalone instances to forward their events to your indexers.

You can find more details on how to manage S.o.S' asset tables manually in a distributed environment in the "Learn More" panel of the Deployment Topology view.

View solution in original post

thinrope
Engager

I think I am in the same situation (just number of hosts is different). I can see detailed CPU usage for each of the hosts (since _internal is forwarded to the IDX cluster), but I don't see some of the details inside "A glimpse of your Splunk Enterprise instances" panel. I tried to understand how it pulls those, but am giving up 😐

The SH that has the app and all IDX in the cluster show all details, but other SH, LM, DS, HF, etc. show ONLY Version and Platform. I'd like to see $SPLUNK_HOME and number of cores, etc for all instances.

Where is this data taken from, is it based on (forwarded) logs? If so can someone show a search with no macros?
Or is something pulled via the REST API?

Splunk-6.2.2 / S.o.S-3.2.1 here

0 Karma

hexx
Splunk Employee
Splunk Employee

The data in that panel relies on the ability to run a search directly against the instance selected. As such, this panel is not expected to work for instances that are note search peers of the S.o.S search-head.

0 Karma

thinrope
Engager

I see..

Well, then that is a RFE:

Include a script that runs say every 24h on each host and collects those parameters and use standard log forwarding techniques. None of them is expected to change much anyway, so direct query does not make sense, IMHO.

0 Karma

hexx
Splunk Employee
Splunk Employee

The S.o.S app is only able to fetch details for other instances which are accessible by distributed search. This is not the case for forwarders, and typically not the case for your Deployment Server/License Master instance, as well as for your other search-head.

Regarding forwarders, we intend to display alternative details (Splunk version, forwarder type, platform) in the future.

For non-forwarder instances, you have a few options:

  • Live with the lack of details for these instances
  • Install a copy of the S.o.S app on these instances, let it populate the "splunk_instances_info" and "splunk_servers_cache" lookups with the appropriate entries and then manually merge those to the instances of these lookup files on your search-head(s)
  • Install a copy of the S.o.S app on these instances and consult details / other views with a local instance scope that way

The second method is the one we recommend, along with configuring those standalone instances to forward their events to your indexers.

You can find more details on how to manage S.o.S' asset tables manually in a distributed environment in the "Learn More" panel of the Deployment Topology view.

xzjc3q
Explorer

I will have a look at implementing the second option. Thank you.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...