Monitoring Splunk

Getting splunk health metrics to prometheus/grafana with grok_exporter

NOCSSMS
Explorer

Hello everyone!

I'm new to splunk, but I'd like to monitory my Splunk Enterprise instance with prometheus and grafana. I'd like to get system statistics about how many messages are ingested by the splunk process (with the sources as labels), what the internal health of the system is (various message queues), processing time for messages, lookup times for various external sources, etc.

I figured that one way of collecting such data would be by scraping various logs with grok_exporter and exporting the relevant metrics to a prometheus instance. However, I'm new to splunk and I don't know what the relevant logs are. Could you guys point me in the right direction?

I'd also like to scrape various warnings/errors generated by splunk (and also license usage messages) so that I can forward them to Alert Manager.

In my searches I did find this project https://github.com/ehershey/splunk-exporter, but it exports only the most basic data (splunk status as up or down).

 

Any advice is welcome!

Regards,

Adrian

Labels (1)
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...