topic Re: calculating average in Splunk Search

calculating average

pmr — Wed, 18 May 2011 03:20:00 GMT

Need some help on search string to calculate 10 minute average and generate alert.
say for example Virtual memory usage 10 minute average > 10000 generate an alert. How can we calculate 10 minute average ?
this would be from a table event so need to use multikv. sample event below :

USER               PID   PSR   pctCPU       CPUTIME  pctMEM     RSZ_KB     VSZ_KB   TTY      S       ELAPSED  COMMAND             ARGS
root                 1     1      0.0      00:00:16     0.0        788      10384   ?        S   19-08:11:30  init                [3]
root                 2     1      0.0      00:00:00     0.0          0          0   ?        S   19-08:11:30  [kthreadd]          <noArgs>
root                 3     0      0.0      00:00:00     0.0          0          0   ?        S   19-08:11:30  [migration/0]       <noArgs>
root                 4     0      0.0      00:00:00     0.0          0          0   ?        S   19-08:11:30  [ksoftirqd/0]       <noArgs>
root                 5     0      0.0      00:00:26     0.0          0          0   ?        S   19-08:11:30  [events/0]          <noArgs>
root                 6     0      0.0      00:00:00     0.0          0          0   ?        S   19-08:11:30  [cpuset]            <noArgs

thanks
pmr

Re: calculating average

pmr — Thu, 19 May 2011 00:14:58 GMT

Folks,
Any help on this ?

-pmr

Re: calculating average

Masa — Thu, 19 May 2011 06:23:45 GMT

The sample is ps output. Assuming you get the ps output every minutes or so. Virtual memory usage is VSZ_KB per PID in the event. You can run the following search every 10 minutes and set the alert when there is events more than one;

sourcetype=ps earliest=-11m@m latest=-1m@m
   | multikv fields VSZ_KB PID
   | chart avg(VSZ_KB) AS AvgSizeKB by PID 
   | where AvgSizeKB > 10000 
   | sort - AvgSizeKB

Re: calculating average

pmr — Fri, 20 May 2011 07:03:20 GMT

thank you.. works perfect.

-pmr