Splunk Search

streamstats confusion

Path Finder

Having some trouble with streamstats.

I need to be alerted, once, at the time when a logical drive becomes less that 10% available. So, I have a script that writes a log file on a 5min interval, monitored by Splunk as sourcetype 'drivetracker'. I need the query to one-time alert me when the freePercent falls below the 10% mark. The source data looks correct.

Problem is, the Splunk query is not giving me the lastFreePercent figure based on the freePercent figure from the previous log file?

Here is my query:

sourcetype="drivetracker"  devId="*" 
| streamstats current=f window=1 last(freePercent) as lastFreePercent
| where freePercent<10 AND lastFreePercent>9
| table _time, server, devId, freePercent, lastFreePercent

and here are the results:

server  devId   totSpace    usedSpace   frSpace freePercent lastFreePercent
server1 C:  100 96.76   3.24    3   **98**
server1 E:  800 782.75  17.25   2   **76**
server1 F:  800 768.09  31.91   4   **24**

Apologies for the poor formatting. According to this, the three drives on server1 suddenly dropped a whole lot of space in under 5 minutes, which simply isn't the case.

Any help is appreciated!

Thanks!

Tags (1)
0 Karma

SplunkTrust
SplunkTrust

Can you give this a try...

Updated answer:

sourcetype="drivetracker"  devId="*"  | sort -_time
| streamstats current=f window=1 last(freePercent) as lastFreePercent by devId,server
| where freePercent<10 AND lastFreePercent>9
| table _time, server, devId, freePercent, lastFreePercent

Give this a try as well

sourcetype="drivetracker" devId="*" | sort server,devId,-_time | eval devId_server = devId . server | streamstats current=f window=2 last(freePercent) as lastFreePercent by devId_server | where freePercent<10 AND lastFreePercent>9 | table _time, server, devId, devId_server, freePercent, lastFreePercent

Path Finder

The second query is pretty close, however the lastFreePercent shows the updated figure before the freePercent. Despite this being sorted by server or _time. Looking at http://answers.splunk.com/answers/105733/streamstats-is-reversed to see if that helps me. Thanks for all your help so far!

0 Karma

SplunkTrust
SplunkTrust

Give the new answer a try...

0 Karma

Path Finder

By the way, this seems to be pretty close, when I aggregate the server & devId and streamstats by that, but only when I specify the a server & devId:

sourcetype="drivetracker" devId="*" server="server1" devId="C:" | sort -_time | eval devId_server = devId . server | streamstats current=f window=2 last(freePercent) as lastFreePercent by devId_server | where freePercent<10 AND lastFreePercent>9 | table _time, server, devId, devId_server, freePercent, lastFreePercent

Removing [server="server1" devId="C:"] causes a zero result set. Weird!

0 Karma

Path Finder

Interesting - when I ran the query you updated the lastFreePercent field was returned null values (returning zero records based on the filter). Removing the lastFreePercent<x from the filter yielded results, with the current freePercent figure as accurate, it seems.

The test showed that the source data appears to be correct.

0 Karma

SplunkTrust
SplunkTrust

Also try the updated answer, just now seen that the grouping was missing from streamstats.

0 Karma

SplunkTrust
SplunkTrust

Can you validate if the source data looks correct by executing this..

sourcetype="drivetracker" server="server1" devId="C:" OR devId="E:" OR devId="F:" | streamstats count by devId | where count ❤️

This should give you last 2 records for devId C:, E: and F: for server 1. Look at the value (if possible, provide it in the post) and see if really the 2nd last record says freepercent so high.

0 Karma

Path Finder

Thanks for the quick(!) response. Unfortunately the problem persists.

0 Karma