Knowledge Management

Looking for the Forecasting Code/Macro from Mike Fisher's Splunk Conf 2016 Presentation

bandit
Motivator

Ok, I found this great post from Conf 2016 by Mike Fisher about using Splunk for forecasting:
https://conf.splunk.com/files/2016/slides/building-a-crystal-ball-forecasting-future-values-for-mult...

I've been looking for the macros in a Splunk App or post, however, I haven't found them and the version in the PDF isn't copying correctly for me. So I've retyped the macro and a static version, however, I'm having trouble getting the search to produce results.

I've posted my copy of the macro below and also a hard coded version of the search.

Macro version:

eval w=case
    (
    ( _time > relative_time ( now(), "$reltime$@d-5w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-5w+$days$d+30m" ) ), 5,
    ( _time > relative_time ( now(), "$reltime$@d-4w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-4w+$days$d+30m" ) ), 4,
    ( _time > relative_time ( now(), "$reltime$@d-3w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-3w+$days$d+30m" ) ), 3,
    ( _time > relative_time ( now(), "$reltime$@d-2w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-2w+$days$d+30m" ) ), 2,
    ( _time > relative_time ( now(), "$reltime$@d-1w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-1w+$days$d+30m" ) ), 1
    ) 
| eval shift=case( isnotnull(w),"+"+w+"w-30m+"+w+"w-20m+"+w+"w-10m+"+w+"w-0m+"+w+"w+10m+"+w+"w+20m+"+w+"w+30m" ) 
| where isnotnull(shift) 
| makemv shift 
| mvexpand shift 
| eval time=relative_time(_time, shift) 
| eventstats avg($val$) as pred by time 
| eval upper=if($val$ > pred, $val$, pred) 
| eval lower=if($val$ < pred, $val$, pred) 
| stats avg($val$) as pred, stdev(upper) as ustdev, stdev(lower) as lstdev by time 
| eval low=pred-lstdev*(sqrt(1/(1-$confidence$/100))) 
| eval low=if(low<0, 0, low) 
| eval high=pred+ustdev*(sqrt(1/(1-$confidence$/100))) 
| eval _time=time 
| timechart span=10m min(pred) as pred, min(low) as low, min(high) as high 
| where _time > relative_time( now(), "$reltime$@d" ) AND _time <= relative_time( now(), "$reltime$+$days$d@d" )

Hard Coded version:

index=summary_trend source=orders_10min earliest=-5w 
| timechart span=10m sum(OrderCount) as actual 
| eval w=case
    (
    ( _time > relative_time ( now(), "+1d@d-5w-30m" ) AND _time <= relative_time ( now(), "+1d@d-5w+3d+30m" ) ), 5,
    ( _time > relative_time ( now(), "+1d@d-4w-30m" ) AND _time <= relative_time ( now(), "+1d@d-4w+3d+30m" ) ), 4,
    ( _time > relative_time ( now(), "+1d@d-3w-30m" ) AND _time <= relative_time ( now(), "+1d@d-3w+3d+30m" ) ), 3,
    ( _time > relative_time ( now(), "+1d@d-2w-30m" ) AND _time <= relative_time ( now(), "+1d@d-2w+3d+30m" ) ), 2,
    ( _time > relative_time ( now(), "+1d@d-1w-30m" ) AND _time <= relative_time ( now(), "+1d@d-1w+3d+30m" ) ), 1
    ) 
| eval shift=case( isnotnull(w),"+"+w+"w-30m+"+w+"w-20m+"+w+"w-10m+"+w+"w-0m+"+w+"w+10m+"+w+"w+20m+"+w+"w+30m" )
| where isnotnull(shift) 
| makemv shift 
| mvexpand shift 
| eval time=relative_time(_time, shift) 
| eventstats avg(actual) as pred by time 
| eval upper=if(actual > pred, actual, pred) 
| eval lower=if(actual < pred, actual, pred) 
| stats avg(actual) as pred, stdev(upper) as ustdev, stdev(lower) as lstdev by time 
| eval low=pred-lstdev*(sqrt(1/(1-90.0/100))) 
| eval low=if(low<0, 0, low) 
| eval high=pred+ustdev*(sqrt(1/(1-90.0/100))) 
| eval _time=time 
| timechart span=10m min(pred) as pred, min(low) as low, min(high) as high 
| where _time > relative_time( now(), "+1d@d" ) AND _time <= relative_time( now(), "+1d+3d@d" )

Thanks for your help,

Rob

skoelpin
SplunkTrust
SplunkTrust

What's your timerange set to? It should be atleast 5 weeks back to return data

I got it working on my end.

0 Karma

mlf
Path Finder

Good to hear.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

I just now realized this thread was 1 years old LOL

0 Karma

mlf
Path Finder

Sorry for being a little late to the tread. What did you're timechart output look like prior to the macro?

0 Karma

Lucas_K
Motivator

Which part errors for your data in the macro?

What parameters were you passing through to forecast5w(val,confidence,rel@me,days)?

Comparing your source and his presentation i'd guess the macro should look like this?

forcast5w(OrderCount,90,1,7)

With your full search looking something like this :

index=summary_trend source=orders_10min earliest=-5w  | `forcast5w(OrderCount,90,1,7)`
0 Karma

bandit
Motivator

Hi, Lucas, I'm not receiving an error, just no results. I do receive results from the base search, just not any results once I add the addtional macro functions. Ultimately, I was trying to avoid reverse-engineering line by line, but that's what I may have to do to get it working. Unless someone knows Mike Fisher???. If you are able to get either the hard coded or macro version to work, that would be great. I will also update this post if I find a solution before then.

I suspect that I have some typos in the the macro code that I tried to recreate from the PDF linked in the question above. I pasted both the macro version with the tokens and the hard coded version to allow for easier debugging.

When using the macro version, my query looked like this

 index=summary_trend source=orders_10min earliest=-5w 
 | timechart span=10m sum(OrderCount) as actual |`forecast5w(actual,90.0,+1d,3)`

Thanks,

Rob

0 Karma

bandit
Motivator

ok, I've found I'm getting results if I remove the last two lines with the timechart and where clause so I'm debugging those now.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...