Splunk Search

Can I not show the previous predict values in my chart?

aohls
Contributor

When using the predict command the time chart shows the calculated time chart value but also has the prediction line following it. For example I have:

 

 

<search>....

| timechart count as Volume 
| predict Volume

 

 

What happens with this is that my historical values have both the Volume and prediction(Volume) lines. What I would like is for Volume to show until the last calculated point, then switch over and show the prediction(Volume). Is there a way anyone has achieved this? 

Labels (1)
Tags (1)
0 Karma
1 Solution

Richfez
SplunkTrust
SplunkTrust

Well, no, not really.  It's how predict works.

But then again, what's stopping us?  It's just spitting data out that we can see and manipulate.

So, let's run an eval after the predict that sets the fields back to null() on all the "predicted" fields where Volume is actually valid.

...
| timechart count as Volume 
| predict  Volume 
| eval "lower95(prediction(Volume))"=if(isnull(Volume), 'lower95(prediction(Volume))', null()) 
| eval "upper95(prediction(Volume))"=if(isnull(Volume), 'upper95(prediction(Volume))', null()) 
| eval "prediction(Volume)"=if(isnull(Volume), 'prediction(Volume)', null())

And there's an interesting mix needed of single and double-quotes in there.  Double quotes around the values on the left side of the = sign, because otherwise Splunk gets really confused about trying to run functions like "lower95" inside an eval, because it's just interpreting it incorrectly.  The single quotes on those things on the right sides of the = sign is because we want to take the value of the field, and again since it has like weird parens and stuff, Splunk would get confused. 

Anyway!

That works, but there's a gap between the last Volume and the first prediction.  Which makes sense, because Splunk can't start drawing that predict line until you've given it a number. 

There's a variety of ways that maybe we can fix that.

All right, here's something to play with!

...
| timechart count as Volume 
| predict  Volume future_timespan=5
| streamstats count as tracker
| eventstats max(tracker) as overall_values
| eval trim_after = overall_values - 5
| eval "lower95(prediction(Volume))"=if('tracker'>='trim_after', 'lower95(prediction(Volume))', null())
| eval "upper95(prediction(Volume))"=if('tracker'>='trim_after', 'upper95(prediction(Volume))', null())
| eval "prediction(Volume)"=if('tracker'>='trim_after', 'prediction(Volume)', null())
|  fields - tracker, trim_after, overall_values

paste that into your search, and now...

OK, so the changes. 

I added "future_timespan=5" to your predict. It's the default, but I'd rather not rely on defaults because we depend on that later.

Now comes the fun.

Streamstats builds a count (1... ) to the max number of resulting events you end up with. 

Evenstats finds the largest of those and records it into each event so we can compare.

We eval our trim_after field to being max of the tracker value (overall_values) - future_timespan which is 5.

Then we have similar evals as above, only now we check if tracker is bigger than or equal to trim_after, we use the prediction value.  If it's not, we chop that sucker right out of there.

Lastly, that fields just cleans it up. Otherwise the graph is a TOTAL MESS.  lol.

(you can remove that last | fields if you want to see all the "work" it does - just don't look at the graph in that case!)

 

Happy Splunking!

-Rich

View solution in original post

Richfez
SplunkTrust
SplunkTrust

Well, no, not really.  It's how predict works.

But then again, what's stopping us?  It's just spitting data out that we can see and manipulate.

So, let's run an eval after the predict that sets the fields back to null() on all the "predicted" fields where Volume is actually valid.

...
| timechart count as Volume 
| predict  Volume 
| eval "lower95(prediction(Volume))"=if(isnull(Volume), 'lower95(prediction(Volume))', null()) 
| eval "upper95(prediction(Volume))"=if(isnull(Volume), 'upper95(prediction(Volume))', null()) 
| eval "prediction(Volume)"=if(isnull(Volume), 'prediction(Volume)', null())

And there's an interesting mix needed of single and double-quotes in there.  Double quotes around the values on the left side of the = sign, because otherwise Splunk gets really confused about trying to run functions like "lower95" inside an eval, because it's just interpreting it incorrectly.  The single quotes on those things on the right sides of the = sign is because we want to take the value of the field, and again since it has like weird parens and stuff, Splunk would get confused. 

Anyway!

That works, but there's a gap between the last Volume and the first prediction.  Which makes sense, because Splunk can't start drawing that predict line until you've given it a number. 

There's a variety of ways that maybe we can fix that.

All right, here's something to play with!

...
| timechart count as Volume 
| predict  Volume future_timespan=5
| streamstats count as tracker
| eventstats max(tracker) as overall_values
| eval trim_after = overall_values - 5
| eval "lower95(prediction(Volume))"=if('tracker'>='trim_after', 'lower95(prediction(Volume))', null())
| eval "upper95(prediction(Volume))"=if('tracker'>='trim_after', 'upper95(prediction(Volume))', null())
| eval "prediction(Volume)"=if('tracker'>='trim_after', 'prediction(Volume)', null())
|  fields - tracker, trim_after, overall_values

paste that into your search, and now...

OK, so the changes. 

I added "future_timespan=5" to your predict. It's the default, but I'd rather not rely on defaults because we depend on that later.

Now comes the fun.

Streamstats builds a count (1... ) to the max number of resulting events you end up with. 

Evenstats finds the largest of those and records it into each event so we can compare.

We eval our trim_after field to being max of the tracker value (overall_values) - future_timespan which is 5.

Then we have similar evals as above, only now we check if tracker is bigger than or equal to trim_after, we use the prediction value.  If it's not, we chop that sucker right out of there.

Lastly, that fields just cleans it up. Otherwise the graph is a TOTAL MESS.  lol.

(you can remove that last | fields if you want to see all the "work" it does - just don't look at the graph in that case!)

 

Happy Splunking!

-Rich

aohls
Contributor

That is perfect, thank you. I did add an eval for the Volume, when the graph shows the current interval it does not look great as traffic builds and changes a lot. I chopped the active time frame off and aligned the prediction that way.

0 Karma

Richfez
SplunkTrust
SplunkTrust

proof:

Richfez_0-1605134477039.png

Also note, in THAT case the prediction and the old Volume line up.  But normally they're not that lined-up-ed, those predictions are all over at that point.  🙂

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...