Archive
Highlighted

Forecast Time series

New Member

Hi Ninjas,

I have a query that looks like this:

sourcetype="x" index=y source="z" host="S" 
| bin _time span=10m
| stats dc(CN) as Actual by _time | lookup CN_Forecast_S.csv _time OUTPUT lowerBound pred upperBound 
| eval isOutlierLow=if(Actual < lowerBound , abs(Actual-lowerBound)/lowerBound, 0) 
| eval isOutlierHigh=if(Actual > upperBound, abs(Actual-upperBound)/upperBound, 0)  
| eval isOutlier=if(Actual < lowerBound OR Actual > upperBound, abs(Actual)/abs(upperBound-lowerBound), 0) 
| fields _time, Actual, lowerBound, pred, upperBound, isOutlier, isOutlierLow, isOutlierHigh

The CNForecastS.csv is a lookup file generated by a savedsearch that predict +2days of data.
The problem is my query display data until "now" only and I would like to show data for the rest +xdays that I already have predicted in the same graph. I tried to specify lastest=+2d@d , but that didn't work
That's the result of my query
alt text
Thank you in advance.

0 Karma
Highlighted

Re: Forecast Time series

SplunkTrust
SplunkTrust

Why are you pulling boundaries from a lookup table? Why not calculate it dynamically or push it into a summary index? Your current method must be pretty slow

This wouldn't really qualify as machine learning as your not dynamically populating boundaries based on previous trends. You should let the machine do the work and calculate the boundaries based off historical values rather than storing them in a lookup table. You're going to get lots of Type I and Type II errors with this method

0 Karma
Highlighted

Re: Forecast Time series

New Member

I'm populating boundaries dynamically based on previous trends and I'm putting them into a local file via outputlookup (because I don't have the permission to use the Collect command so I can't store the data into a summary index).
The question is how to continue to display the boundaries in the same chart from "now" to +2days. Thank you 🙂

0 Karma
Highlighted

Re: Forecast Time series

SplunkTrust
SplunkTrust

This is not an ideal way, you should either get access to the collect command or enable scheudled reports to populate the summary index.

If you want to push the boundaries into the future, you should use the timeshift technique

| eval w=case( (_time>relative_time(now(), "+1d@d-5w-30m") AND _time<=relative_time(now(), "+1d@d-5w+1d+30m")), 5, (_time>relative_time(now(), "+1d@d-4w-30m") AND _time<=relative_time(now(), "+1d@d-4w+1d+30m")), 4, (_time>relative_time(now(), "+1d@d-3w-30m") AND _time<=relative_time(now(), "+1d@d-3w+1d+30m")), 3, (_time>relative_time(now(), "+1d@d-2w-30m") AND _time<=relative_time(now(), "+1d@d-2w+1d+30m")), 2, (_time>relative_time(now(), "+1d@d-1w-30m") AND _time<=relative_time(now(), "+1d@d-1w+1d+30m")), 1) 
| eval shift=case(isnotnull(w),"+"+w+"w-30m,+"+w+"w-20m,+"+w+"w-10m,+"+w+"w-0m,+"+w+"w+10m,+"+w+"w+20m,+"+w+"w+30m,") 
| where isnotnull(shift) 
| makemv delim="," shift 
| mvexpand shift 
| eval time=relative_time(_time,shift) 
0 Karma
Highlighted

Re: Forecast Time series

New Member

Yes I already doing this:
sourcetype="X" index=Y source="Z" host="S"
| bin time span=10m
| stats dc(CN) as Actual by host, _time
| Forecast4w(Actual,90.0,+1d,1) | outputlookup CN
Forecast_S.csv append=True

The Forcast4w is a macro that contain the timeshift technique, and I scheduled this to run every day

0 Karma
Highlighted

Re: Forecast Time series

SplunkTrust
SplunkTrust

So you should be pushing it 1 day into the future already then..

0 Karma
Highlighted

Re: Forecast Time series

New Member

It's a display issue: my current query show the "pred" just until now and I would like that continue to display the rest of the data generated by the Forcast4w macro. I didn't get the result desired by specifying the Latest time +1d@d. Sorry if i wasn't clear enough!

0 Karma
Highlighted

Re: Forecast Time series

SplunkTrust
SplunkTrust

So your issue is solved then? Can you accept the answer to close out the question?

0 Karma
Highlighted

Re: Forecast Time series

New Member

Not yet, still have this issue!

0 Karma
Highlighted

Re: Forecast Time series

SplunkTrust
SplunkTrust

I'm not clear on your problem. What is not working properly?

0 Karma