All Apps and Add-ons

Example of how to monitor storage I/O latency?

sloshburch
Ultra Champion

Does anyone have examples of how to use Splunk to monitor storage I/O latency?

1 Solution

sloshburch
Ultra Champion

The Splunk Product Best Practices team helped produce this response. Read more about use case examples Splunk® Platform Use Cases on Splunk Docs.

This use case enables system administrators to identify underlying hardware issues when systems experience storage latency.

This use case is from the Splunk Essentials for Infrastructure Troubleshooting and Monitoring app. For more examples, see the Splunk Essentials for Infrastructure Troubleshooting and Monitoring on Splunkbase.

Measuring Storage IO Latency

Load data

How to implement: Ingest operating system logs and metrics into Splunk Enterprise. Install the Splunk Add-on for Windows or *nix, and enable iostat monitoring for your entire infrastructure. Enable the iostat.sh scripted input in the Splunk Add-on for Windows or *nix. Find the Splunk Add-on for Windows and Splunk Add-on for Unix and Linux on Splunkbase.

Data check: This use case depends on operating system logs and operating system metrics.

Get insights

Track the latency of storage IO using the Splunk Add-on for Windows or *nix to identify storage and disks that are experiencing higher than normal latency. Build dashboards and alerts to take timely action.

Use the following search:

index=* tag=oshost tag=performance tag=storage
| stats avg(latency) AS average_wait_ms BY host

Best practice: In searches, replace the asterisk in index=* with the name of the index that contains the data. By default, Splunk stores data in the main index. Therefore, index=* becomes index=main. Use the OR operator to specify one or multiple indexes to search. For example, index=main OR index=security. See About managing indexes and How indexing works in Splunk docs for details.

Help

If no results appear, deploy the Add-ons to the search heads to access the knowledge objects necessary for simple searching. See About installing Splunk add-ons on Splunk Docs for assistance.

For more support, post a question to the Splunk Answers community.

View solution in original post

sloshburch
Ultra Champion

The Splunk Product Best Practices team helped produce this response. Read more about use case examples Splunk® Platform Use Cases on Splunk Docs.

This use case enables system administrators to identify underlying hardware issues when systems experience storage latency.

This use case is from the Splunk Essentials for Infrastructure Troubleshooting and Monitoring app. For more examples, see the Splunk Essentials for Infrastructure Troubleshooting and Monitoring on Splunkbase.

Measuring Storage IO Latency

Load data

How to implement: Ingest operating system logs and metrics into Splunk Enterprise. Install the Splunk Add-on for Windows or *nix, and enable iostat monitoring for your entire infrastructure. Enable the iostat.sh scripted input in the Splunk Add-on for Windows or *nix. Find the Splunk Add-on for Windows and Splunk Add-on for Unix and Linux on Splunkbase.

Data check: This use case depends on operating system logs and operating system metrics.

Get insights

Track the latency of storage IO using the Splunk Add-on for Windows or *nix to identify storage and disks that are experiencing higher than normal latency. Build dashboards and alerts to take timely action.

Use the following search:

index=* tag=oshost tag=performance tag=storage
| stats avg(latency) AS average_wait_ms BY host

Best practice: In searches, replace the asterisk in index=* with the name of the index that contains the data. By default, Splunk stores data in the main index. Therefore, index=* becomes index=main. Use the OR operator to specify one or multiple indexes to search. For example, index=main OR index=security. See About managing indexes and How indexing works in Splunk docs for details.

Help

If no results appear, deploy the Add-ons to the search heads to access the knowledge objects necessary for simple searching. See About installing Splunk add-ons on Splunk Docs for assistance.

For more support, post a question to the Splunk Answers community.

View solution in original post

sloshburch
Ultra Champion

Update: I added a related video.

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!