Perfmon iis stats aggregation

[twinspop]

I've got the Universal Forwarder installed on all our web servers. Every 5 minutes they are posting "Total Bytes Sent" from the "Web Service" object. These are simple counters, so in order to get bandwidth information, I need to use the delta command. That complicates things very quickly tho.

We have 96 web servers broken into 24 farms of 4 each. I'd want to report on bandwidth stats per farm. I'm at a loss for how to mix and match delta, stats, timechart, bucket and friends into aggregating the deltas while separating them into the appropriate farms.

Is it possible, or should I simply be running 24 different searches feeding into a summary index?

[gkanapathy · Accepted solution]

You should use streamstats instead of delta. delta is a very simple command, but for the reasons you've discovered, it's unsuitable for larger scales of data sets.

An example of using streamstats with incrementing counters is:

...
| streamstats current=f global=f window=1
    last(Total_Bytes_Sent) as prev_Total_Bytes_Sent
    last(_time) as prev_time
  by host
| eval dt=_time-prev_time
| eval d_Bytes_Sent=Total_Bytes_Sent-prev_Total_Bytes_Sent
| eval bps=d_Bytes_Sent/dt
| ...

You can pipe this to more search commands (e.g. | timechart avg(bps) by host, stats max(bps) by host) by host and use d_Bytes_Sent for other analysis.

[twinspop]

I think this is getting me down the correct path, but as posted above it does not work. The prev_* values are identical to those in the current record. Setting current=f seems to be the fix. Now to omit the rolled over counters and other cruft.