Splunk Search

Splunk perfmon disk io search

tkerr1357
Path Finder

Hello all,

 

Looking for some help with a perfmon search. 

index=perfmon host=myhost01s* sourcetype="PerfmonMk:LogicalDisk"  instance=_total

| timechart sum(Disk_Transfers/sec) span=90s

 

This gives me all the IO of the hosts, but it only seems accurate if I chart it to 90s span, which is how often it queries. If I change that then the sum obviously just puts all the values together.

What I want to do is show longer periods of time, but still have it counting the sum of each reading (like a bucket) over time and not adding all numbers over the whole period (span) together. I want to force it to always use the time as the bucket. Any help is much appreciated.

Labels (1)
0 Karma
1 Solution

tscroggins
Builder

@tkerr1357 

If your perfmon input interval is 90 seconds, then Disk_Transfers = 90 * 'Disk_Transfers/sec'. You can displayed corrected Disk_Transfers/sec and Disk_Transfers values for any time span using this:

 

index=perfmon host=myhost01s* sourcetype="PerfmonMk:LogicalDisk" instance=_total
| timechart fixedrange=f minspan=90s per_second(eval(90 * 'Disk_Transfers/sec')) as Disk_Transfers/sec sum(eval(90 * 'Disk_Transfers/sec')) as Disk_Transfers by host

If your interval is unknown, you can estimate it from data:

index=perfmon host=myhost01s* sourcetype="PerfmonMk:LogicalDisk" instance=_total
| streamstats current=f last(_time) as last_time by host
| eval interval=_time-last_time
| where NOT isnull(interval)
| timechart fixedrange=f per_second(eval(interval * 'Disk_Transfers/sec')) as Disk_Transfers/sec sum(eval(interval * 'Disk_Transfers/sec')) as Disk_Transfers by host

Note that your tails may not cover an entire span. E.g. If timechart chooses span=1h and your search time range is not snapped to the nearest hour with @h, then your first and last values will be skewed by missing data.

 

View solution in original post

0 Karma

tscroggins
Builder

@tkerr1357 

If your perfmon input interval is 90 seconds, then Disk_Transfers = 90 * 'Disk_Transfers/sec'. You can displayed corrected Disk_Transfers/sec and Disk_Transfers values for any time span using this:

 

index=perfmon host=myhost01s* sourcetype="PerfmonMk:LogicalDisk" instance=_total
| timechart fixedrange=f minspan=90s per_second(eval(90 * 'Disk_Transfers/sec')) as Disk_Transfers/sec sum(eval(90 * 'Disk_Transfers/sec')) as Disk_Transfers by host

If your interval is unknown, you can estimate it from data:

index=perfmon host=myhost01s* sourcetype="PerfmonMk:LogicalDisk" instance=_total
| streamstats current=f last(_time) as last_time by host
| eval interval=_time-last_time
| where NOT isnull(interval)
| timechart fixedrange=f per_second(eval(interval * 'Disk_Transfers/sec')) as Disk_Transfers/sec sum(eval(interval * 'Disk_Transfers/sec')) as Disk_Transfers by host

Note that your tails may not cover an entire span. E.g. If timechart chooses span=1h and your search time range is not snapped to the nearest hour with @h, then your first and last values will be skewed by missing data.

 

View solution in original post

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.