Monitoring Splunk

IOPS reported by bonnie++ and Splunk Monitoring console

koshyk
Super Champion

One of our client have 10K HDD in RAID10 and as per Bonnie++ Random Seeks (IOPS) comes to approx 1500 IOPS and wanted to build a dashboard for IOPS and disk usage. I was thinking to re-use the Monitoring console searches

But when I look into the Monitoring Console or DMC, the results show some Indexers of 6000 IOPS !! which is Not possible. Is this a problem with the Splunk api or does this involve RAM assistance?

the query used in DMC is:

| rest splunk_server_group=* splunk_server_group="*" /services/server/status/resource-usage/iostats   | eval iops = round(reads_ps + writes_ps)
0 Karma

nnmiller
Contributor

You can't sum these as you have in your query, since they are IOPS per disk. From the /services/server/status/resource-usage/iostats docs page:

Access the most recent disk I/O statistics for each disk. This endpoint is currently supported for Linux, Windows, and Solaris. By default this endpoint is updated every 60s seconds.

Running:

splunk cmd splunkd instrument-resource-usage --debug > some_log_file.log 2>&1

shows that these stats come from /proc/diskstats

(HT: blachance_splunk)

Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...