Monitor Splunk

[taka]

I want to monitor Splunk Enterprise in a cluster environment. I monitor the Splunk infrastructure with Newrelic, and I also want to use the DMC health check item.

Where can I get the health check item other than by updating it? Also, please let me know if there are any other ways to monitor Splunk.

[taka]

Hello deepakc

Thank you for your immediate reply!
Do you have any prerequisites or concerns when implementing monitoring of that API endpoint?

[deepakc]

These would come to mind first - there's plenty more, you can explore the others and use them as you as you see fit. 

1. Check the overall health

/services/cluster/manager/health

2. Check Cluster Status of the peers (Indexers

/services/cluster/manager/peers

3. Check the indexing status

/services/cluster/manager/indexes

4. Check the Replication and Search Factor status

/services/cluster/manager/status

You can also check the CM's resources (CPU/MEM etc)
5. Check Resource Utilisation on the CM

/services/server/status/resource-usage/hostwide

[deepakc · Accepted solution]

You can start by having a look at the many API's endpoints available for the aspects of Splunk, and yes we mainly use the monitoring console for cluster -checks and health + others - which uses API's for checks

Have a look at the below API endpoints, as a starting point. 

#Cluster Specific 
https://docs.splunk.com/Documentation/Splunk/9.0.2/RESTREF/RESTcluster#cluster.2Fmanager.2Fhealth 

 #Other API endpoints 

https://docs.splunk.com/Documentation/Splunk/9.0.2/RESTREF/RESTlist#MLOC 

[deepakc]

I forgot to mention in terms of pre-reqs: 

1. Newrelic should have some way of using API calls, you can use Splunk Tokens for API use and as a way of authentication  - see below link for info 

https://docs.splunk.com/Documentation/Splunk/9.2.1/Security/CreateAuthTokens