Forecast Time series

[BenImen]

Hi Ninjas,

I have a query that looks like this:

sourcetype="x" index=y source="z" host="S" 
| bin _time span=10m
| stats dc(CN) as Actual by _time | lookup CN_Forecast_S.csv _time OUTPUT lowerBound pred upperBound 
| eval isOutlierLow=if(Actual < lowerBound , abs(Actual-lowerBound)/lowerBound, 0) 
| eval isOutlierHigh=if(Actual > upperBound, abs(Actual-upperBound)/upperBound, 0)  
| eval isOutlier=if(Actual < lowerBound OR Actual > upperBound, abs(Actual)/abs(upperBound-lowerBound), 0) 
| fields _time, Actual, lowerBound, pred, upperBound, isOutlier, isOutlierLow, isOutlierHigh

The CN_Forecast_S.csv is a lookup file generated by a savedsearch that predict +2days of data.
The problem is my query display data until "now" only and I would like to show data for the rest +xdays that I already have predicted in the same graph. I tried to specify lastest=+2d@d , but that didn't work
That's the result of my query

Thank you in advance.

[skoelpin]

Why are you pulling boundaries from a lookup table? Why not calculate it dynamically or push it into a summary index? Your current method must be pretty slow

This wouldn't really qualify as machine learning as your not dynamically populating boundaries based on previous trends. You should let the machine do the work and calculate the boundaries based off historical values rather than storing them in a lookup table. You're going to get lots of Type I and Type II errors with this method

[BenImen]

I'm populating boundaries dynamically based on previous trends and I'm putting them into a local file via outputlookup (because I don't have the permission to use the Collect command so I can't store the data into a summary index).
The question is how to continue to display the boundaries in the same chart from "now" to +2days. Thank you 🙂

[skoelpin]

This is not an ideal way, you should either get access to the collect command or enable scheudled reports to populate the summary index.

If you want to push the boundaries into the future, you should use the timeshift technique

| eval w=case( (_time>relative_time(now(), "+1d@d-5w-30m") AND _time<=relative_time(now(), "+1d@d-5w+1d+30m")), 5, (_time>relative_time(now(), "+1d@d-4w-30m") AND _time<=relative_time(now(), "+1d@d-4w+1d+30m")), 4, (_time>relative_time(now(), "+1d@d-3w-30m") AND _time<=relative_time(now(), "+1d@d-3w+1d+30m")), 3, (_time>relative_time(now(), "+1d@d-2w-30m") AND _time<=relative_time(now(), "+1d@d-2w+1d+30m")), 2, (_time>relative_time(now(), "+1d@d-1w-30m") AND _time<=relative_time(now(), "+1d@d-1w+1d+30m")), 1) 
| eval shift=case(isnotnull(w),"+"+w+"w-30m,+"+w+"w-20m,+"+w+"w-10m,+"+w+"w-0m,+"+w+"w+10m,+"+w+"w+20m,+"+w+"w+30m,") 
| where isnotnull(shift) 
| makemv delim="," shift 
| mvexpand shift 
| eval time=relative_time(_time,shift) 

[BenImen]

Yes I already doing this:
sourcetype="X" index=Y source="Z" host="S"
| bin _time span=10m
| stats dc(CN) as Actual by host, _time
| Forecast4w(Actual,90.0,+1d,1) | outputlookup CN_Forecast_S.csv append=True
The Forcast4w is a macro that contain the timeshift technique, and I scheduled this to run every day

[skoelpin]

So you should be pushing it 1 day into the future already then..

[BenImen]

It's a display issue: my current query show the "pred" just until now and I would like that continue to display the rest of the data generated by the Forcast4w macro. I didn't get the result desired by specifying the Latest time +1d@d. Sorry if i wasn't clear enough!

[skoelpin]

So your issue is solved then? Can you accept the answer to close out the question?

[BenImen]

Not yet, still have this issue!

[skoelpin]

I'm not clear on your problem. What is not working properly?