Splunk Search

Streamstats vs autoregress

PickleRick
SplunkTrust
SplunkTrust

Hello.

I've noticed that in many solutions when there is a need for a value from previous row, streamstats with window=1 is used. For example - https://community.splunk.com/t5/Splunk-Search/Unable-to-subtract-one-days-hours-from-previous-days-t...

In similar cases I tended to use autoregress which behaves more or less the same.

The question is - what are pros/cons of each of those commands? Do they have some non-obvious limitations? Is any "better" than the other?

Labels (1)
0 Karma

bowesmana
SplunkTrust
SplunkTrust

@PickleRick 

I've found that autoregress has far less applicability that the equivalent streamstats. The main 'limitation' autoregress has in my view is that is does not cope with any split by clause and in almost every case I have wanted to calculate moving averages, I have needed a split by clause.

Also, autoregress will just copy values from previous rows to the current row and if you still want to calculate ranges between those values, you must run a subsequent calculation to work out the difference, e.g. compare

index=_audit
| streamstats window=2 range(_time) as r
| autoregress _time as t p=1 
| eval d=t-_time
| table _time r t d

@ITWhisperer example in your link showing streamstats with the window of 1 and then the eval, can be done with the window of 2 and the range() clause, as above.

As you can see from this split by clause, autoregress cannot handle the calculation of d when looking at the user.

index=_audit
| streamstats global=f window=2 range(_time) as r by user
| autoregress _time as t p=1 
| eval d=t-_time
| table _time user r t d

As for performance, I've not tested comparisons for either, but both are centralized streaming commands, meaning both will run on the search head, so should always work on the smallest dataset possible.

autoregress is naturally an easy command to use, whereas streamstats with split by and all the flags, (current, global, reset*) make it powerful, but somewhat less intuitive.

 

PickleRick
SplunkTrust
SplunkTrust

Thank you for an exhaustive response. I always found autoregress more straightforward and obvious but it seems I sill have to learn a bit more about streamstats. 🙂

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Automated Threat Analysis: Available in ES Premier

Automated Threat Analysis: Centralize and Accelerate Phishing Investigations in Splunk Enterprise ...

What’s New in Splunk AI: Volume 02

Welcome to the second edition of “What’s New in Splunk AI” where we look at the latest and greatest updates, ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...