Getting Data In
Highlighted

Getting all forwarder names via a REST call

Ultra Champion

We created a monitoring dashboard (outside of Splunk) which relies on rest /services/deployment/server/clients to get the list of the forwarders in the system. I would expect this call to return the same list as the list within the serverclass.conf file but apparently these two are not compatible. Is there a way to make them compatible? meaning that this REST call would return exactly the list which serverclass.conf holds.

0 Karma
Highlighted

Re: Getting all forwarder names via a REST call

SplunkTrust
SplunkTrust

The REST Api endpoint will give list of client which are phoning home to deployment server. The serverclass.conf file can have many clients which are not connecting to deployment server and/or have wildcarded server names so there many not be one-to-one mapping available. What is you concern on using just the output of REST endpoint?

Highlighted

Re: Getting all forwarder names via a REST call

Ultra Champion

The thing is that we reach situations where forwarders are down for some time and the rest call doesn't list these servers, so they can end up being down for a couple of weeks. We also make a good effort to maintain our serverclass.conf file. So, it is the single source of truth for us about the host inventory. It's interesting as we don't use wildcards when we specify the whitelists. Should I create a lookup table based on the serverclass.conf file and check each entry by itself to see whether it's phoning home?

0 Karma
Highlighted

Re: Getting all forwarder names via a REST call

SplunkTrust
SplunkTrust

You should be able to get list of whitelist servers using following query.

| rest /services/deployment/server/serverclasses | table title whitelist.* | untable title whitelist hostname | stats count by hostname | table hostname

You can put it to in a lookup file or just use the rest query itself and compare it against deployment/server/clients to know which clients that are configured in serverclass.conf but not sending phonehome. Something like this

| rest /services/deployment/server/serverclasses | table title whitelist.* | untable title whitelist hostname | stats count by hostname | table hostname | eval state="configured" | append [| rest /services/deployment/server/clients | table title | rename title as hostname | eval state="phonehome" ] | stats values(state) as state by hostname | where mvcount(state)=1 AND state="configured"

View solution in original post

Highlighted

Re: Getting all forwarder names via a REST call

Ultra Champion

You are truly amazing - can you please convert it to an answer?

0 Karma
Highlighted

Re: Getting all forwarder names via a REST call

Motivator

Well, reading this allowed me to be to develop a REST search that can lookup the serverClasses associated with a particular host, which is handy-dandy when you get a decommissioned server notice. Here is the base search:

| rest /services/deployment/server/serverclasses splunk_server=local
| table title whitelist.*
| untable title whitelist hostname
| stats values(title) AS serverClasses count by hostname

The base search gives you a list of all hosts and the serverClasses associated with them. To look for a particular host, add this final line:

| search hostname=FQDNofSomeHost

I wanted to create a dashboard out of this, and so the search looks like this:

 <searchTemplate>
| rest /services/deployment/server/serverclasses splunk_server=local
| table title whitelist.*
| untable title whitelist hostname
| stats values(title) AS serverClasses count by hostname
| search hostname = $HOSTNAME$
</searchTemplate>

With a fieldset clause next:

<input type="text" token="HOSTNAME" searchWhenChanged="true"></input>
<input type="time" searchWhenChanged="true">
  <default>Today</default>
</input>

You can use * as a wildcard to search for all hosts, or as the end of a partial host name.

0 Karma