Splunk Search

Streamstats vs autoregress

PickleRick
SplunkTrust
SplunkTrust

Hello.

I've noticed that in many solutions when there is a need for a value from previous row, streamstats with window=1 is used. For example - https://community.splunk.com/t5/Splunk-Search/Unable-to-subtract-one-days-hours-from-previous-days-t...

In similar cases I tended to use autoregress which behaves more or less the same.

The question is - what are pros/cons of each of those commands? Do they have some non-obvious limitations? Is any "better" than the other?

Labels (1)
0 Karma

bowesmana
SplunkTrust
SplunkTrust

@PickleRick 

I've found that autoregress has far less applicability that the equivalent streamstats. The main 'limitation' autoregress has in my view is that is does not cope with any split by clause and in almost every case I have wanted to calculate moving averages, I have needed a split by clause.

Also, autoregress will just copy values from previous rows to the current row and if you still want to calculate ranges between those values, you must run a subsequent calculation to work out the difference, e.g. compare

index=_audit
| streamstats window=2 range(_time) as r
| autoregress _time as t p=1 
| eval d=t-_time
| table _time r t d

@ITWhisperer example in your link showing streamstats with the window of 1 and then the eval, can be done with the window of 2 and the range() clause, as above.

As you can see from this split by clause, autoregress cannot handle the calculation of d when looking at the user.

index=_audit
| streamstats global=f window=2 range(_time) as r by user
| autoregress _time as t p=1 
| eval d=t-_time
| table _time user r t d

As for performance, I've not tested comparisons for either, but both are centralized streaming commands, meaning both will run on the search head, so should always work on the smallest dataset possible.

autoregress is naturally an easy command to use, whereas streamstats with split by and all the flags, (current, global, reset*) make it powerful, but somewhat less intuitive.

 

PickleRick
SplunkTrust
SplunkTrust

Thank you for an exhaustive response. I always found autoregress more straightforward and obvious but it seems I sill have to learn a bit more about streamstats. 🙂

0 Karma
Get Updates on the Splunk Community!

Get Inspired! We’ve Got Validation that Your Hard Work is Paying Off

We love our Splunk Community and want you to feel inspired by all your hard work! Eric Fusilero, our VP of ...

What's New in Splunk Enterprise 9.4: Features to Power Your Digital Resilience

Hey Splunky People! We are excited to share the latest updates in Splunk Enterprise 9.4. In this release we ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...