Reg. running a daily check aside from the MC do you have a check list you run to make sure all components are healthy?

[SamHTexas]

Aside from the MC in distributed mode checks do you have a comprehensive check list you run making sure all counters & components are healthy in the Ent. & the ES. I know a lot of us love the MC. Just wondering what else the champs do in their environment please? Thank u & happy 2022.

[tscroggins]

@SamHTexas 

You might want to look over my response in May: https://community.splunk.com/t5/Monitoring-Splunk/Please-share-a-short-Splunk-preventative-tasks-lis...

The should cover KV store issues. ES is reasonably good at posting messages for its own health checks, and that's generally a good way to monitor problems from the monitoring console instance as a scheduled search:

| rest splunk_server=*/servicesNS/-/-/messages

and filter as needed.

Likewise, you can search deployment and splunkd health for colors other than green:

| rest splunk_server=* /services/server/health/deployment/details

| rest splunk_server=* /services/server/health/splunkd/details

The latter search should be transposed to show health by feature.

[isoutamo]

Here is some thoughts from last conf https://conf.splunk.com/files/2021/slides/PLA1410C.pdf

r. Ismo