Splunk Search

Forecast Time series

BenImen
New Member

Hi Ninjas,

I have a query that looks like this:

sourcetype="x" index=y source="z" host="S" 
| bin _time span=10m
| stats dc(CN) as Actual by _time | lookup CN_Forecast_S.csv _time OUTPUT lowerBound pred upperBound 
| eval isOutlierLow=if(Actual < lowerBound , abs(Actual-lowerBound)/lowerBound, 0) 
| eval isOutlierHigh=if(Actual > upperBound, abs(Actual-upperBound)/upperBound, 0)  
| eval isOutlier=if(Actual < lowerBound OR Actual > upperBound, abs(Actual)/abs(upperBound-lowerBound), 0) 
| fields _time, Actual, lowerBound, pred, upperBound, isOutlier, isOutlierLow, isOutlierHigh

The CN_Forecast_S.csv is a lookup file generated by a savedsearch that predict +2days of data.
The problem is my query display data until "now" only and I would like to show data for the rest +xdays that I already have predicted in the same graph. I tried to specify lastest=+2d@d , but that didn't work
That's the result of my query
alt text
Thank you in advance.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Why are you pulling boundaries from a lookup table? Why not calculate it dynamically or push it into a summary index? Your current method must be pretty slow

This wouldn't really qualify as machine learning as your not dynamically populating boundaries based on previous trends. You should let the machine do the work and calculate the boundaries based off historical values rather than storing them in a lookup table. You're going to get lots of Type I and Type II errors with this method

0 Karma

BenImen
New Member

I'm populating boundaries dynamically based on previous trends and I'm putting them into a local file via outputlookup (because I don't have the permission to use the Collect command so I can't store the data into a summary index).
The question is how to continue to display the boundaries in the same chart from "now" to +2days. Thank you 🙂

0 Karma

skoelpin
SplunkTrust
SplunkTrust

This is not an ideal way, you should either get access to the collect command or enable scheudled reports to populate the summary index.

If you want to push the boundaries into the future, you should use the timeshift technique

| eval w=case( (_time>relative_time(now(), "+1d@d-5w-30m") AND _time<=relative_time(now(), "+1d@d-5w+1d+30m")), 5, (_time>relative_time(now(), "+1d@d-4w-30m") AND _time<=relative_time(now(), "+1d@d-4w+1d+30m")), 4, (_time>relative_time(now(), "+1d@d-3w-30m") AND _time<=relative_time(now(), "+1d@d-3w+1d+30m")), 3, (_time>relative_time(now(), "+1d@d-2w-30m") AND _time<=relative_time(now(), "+1d@d-2w+1d+30m")), 2, (_time>relative_time(now(), "+1d@d-1w-30m") AND _time<=relative_time(now(), "+1d@d-1w+1d+30m")), 1) 
| eval shift=case(isnotnull(w),"+"+w+"w-30m,+"+w+"w-20m,+"+w+"w-10m,+"+w+"w-0m,+"+w+"w+10m,+"+w+"w+20m,+"+w+"w+30m,") 
| where isnotnull(shift) 
| makemv delim="," shift 
| mvexpand shift 
| eval time=relative_time(_time,shift) 
0 Karma

BenImen
New Member

Yes I already doing this:
sourcetype="X" index=Y source="Z" host="S"
| bin _time span=10m
| stats dc(CN) as Actual by host, _time
| Forecast4w(Actual,90.0,+1d,1) | outputlookup CN_Forecast_S.csv append=True

The Forcast4w is a macro that contain the timeshift technique, and I scheduled this to run every day

0 Karma

skoelpin
SplunkTrust
SplunkTrust

So you should be pushing it 1 day into the future already then..

0 Karma

BenImen
New Member

It's a display issue: my current query show the "pred" just until now and I would like that continue to display the rest of the data generated by the Forcast4w macro. I didn't get the result desired by specifying the Latest time +1d@d. Sorry if i wasn't clear enough!

0 Karma

skoelpin
SplunkTrust
SplunkTrust

So your issue is solved then? Can you accept the answer to close out the question?

0 Karma

BenImen
New Member

Not yet, still have this issue!

0 Karma

skoelpin
SplunkTrust
SplunkTrust

I'm not clear on your problem. What is not working properly?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...