Splunk Search

streamstats confusion

a_splunk_user
Path Finder

Having some trouble with streamstats.

I need to be alerted, once, at the time when a logical drive becomes less that 10% available. So, I have a script that writes a log file on a 5min interval, monitored by Splunk as sourcetype 'drivetracker'. I need the query to one-time alert me when the freePercent falls below the 10% mark. The source data looks correct.

Problem is, the Splunk query is not giving me the lastFreePercent figure based on the freePercent figure from the previous log file?

Here is my query:

sourcetype="drivetracker"  devId="*" 
| streamstats current=f window=1 last(freePercent) as lastFreePercent
| where freePercent<10 AND lastFreePercent>9
| table _time, server, devId, freePercent, lastFreePercent

and here are the results:

server  devId   totSpace    usedSpace   frSpace freePercent lastFreePercent
server1 C:  100 96.76   3.24    3   **98**
server1 E:  800 782.75  17.25   2   **76**
server1 F:  800 768.09  31.91   4   **24**

Apologies for the poor formatting. According to this, the three drives on server1 suddenly dropped a whole lot of space in under 5 minutes, which simply isn't the case.

Any help is appreciated!

Thanks!

Tags (1)
0 Karma

somesoni2
Revered Legend

Can you give this a try...

Updated answer:

sourcetype="drivetracker"  devId="*"  | sort -_time
| streamstats current=f window=1 last(freePercent) as lastFreePercent by devId,server
| where freePercent<10 AND lastFreePercent>9
| table _time, server, devId, freePercent, lastFreePercent

Give this a try as well

sourcetype="drivetracker" devId="*" | sort server,devId,-_time | eval devId_server = devId . server | streamstats current=f window=2 last(freePercent) as lastFreePercent by devId_server | where freePercent<10 AND lastFreePercent>9 | table _time, server, devId, devId_server, freePercent, lastFreePercent

a_splunk_user
Path Finder

The second query is pretty close, however the lastFreePercent shows the updated figure before the freePercent. Despite this being sorted by server or _time. Looking at http://answers.splunk.com/answers/105733/streamstats-is-reversed to see if that helps me. Thanks for all your help so far!

0 Karma

somesoni2
Revered Legend

Give the new answer a try...

0 Karma

a_splunk_user
Path Finder

By the way, this seems to be pretty close, when I aggregate the server & devId and streamstats by that, but only when I specify the a server & devId:

sourcetype="drivetracker" devId="*" server="server1" devId="C:" | sort -_time | eval devId_server = devId . server | streamstats current=f window=2 last(freePercent) as lastFreePercent by devId_server | where freePercent<10 AND lastFreePercent>9 | table _time, server, devId, devId_server, freePercent, lastFreePercent

Removing [server="server1" devId="C:"] causes a zero result set. Weird!

0 Karma

a_splunk_user
Path Finder

Interesting - when I ran the query you updated the lastFreePercent field was returned null values (returning zero records based on the filter). Removing the lastFreePercent<x from the filter yielded results, with the current freePercent figure as accurate, it seems.

The test showed that the source data appears to be correct.

0 Karma

somesoni2
Revered Legend

Also try the updated answer, just now seen that the grouping was missing from streamstats.

0 Karma

somesoni2
Revered Legend

Can you validate if the source data looks correct by executing this..

sourcetype="drivetracker" server="server1" devId="C:" OR devId="E:" OR devId="F:" | streamstats count by devId | where count ❤️

This should give you last 2 records for devId C:, E: and F: for server 1. Look at the value (if possible, provide it in the post) and see if really the 2nd last record says freepercent so high.

0 Karma

a_splunk_user
Path Finder

Thanks for the quick(!) response. Unfortunately the problem persists.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...