Splunk Search

Predict: show past events and future predictions

nplamondon
SplunkTrust
SplunkTrust

I'm using predict, and seeing good results, but I would like to clean up my visualization.

What I would like is to see past data and future predicted data with no overlap.

Using eval predicted=if(isnull(foo), predicted, null()), I am able to show predicted data only where actual data points don't exist, but I still see the upper95 and lower95 overlaid on top of actual data. Trying an equivalent eval for "lower95(predicted)" doesn't work.

Is there a way I can show only actual data where it exists, and only predicted data (with probability ranges) where it doesn't?

0 Karma
1 Solution

nplamondon
SplunkTrust
SplunkTrust

Good news, everybody!

This was entirely a PEBKAC issue. Working on this more today, I found the "Null values" option under formatting. No idea how I had missed that previously. Setting "Null values" to "connect" made the leading zeroed line disappear.

Thanks, again, for your help, Rich and Clara!

View solution in original post

0 Karma

nplamondon
SplunkTrust
SplunkTrust

Good news, everybody!

This was entirely a PEBKAC issue. Working on this more today, I found the "Null values" option under formatting. No idea how I had missed that previously. Setting "Null values" to "connect" made the leading zeroed line disappear.

Thanks, again, for your help, Rich and Clara!

0 Karma

rflouquet
Explorer

Hi @nplamondon !

Sorry but I don't understand how you fired it out... It realy would help me !

Thanks

0 Karma

nplamondon
SplunkTrust
SplunkTrust
  • Open your dashboard, click Edit.
  • On the panel, click the paint brush
  • Change Null Values to "Connect"
  • Close the edit dialog
  • Click Save at the top of the dashboard
0 Karma

Richfez
SplunkTrust
SplunkTrust

You are welcome for what tiny glimmers of help I gave, oh "Everyone's Nate".

0 Karma

Richfez
SplunkTrust
SplunkTrust

Run anywhere:

* 
| timechart count as foo 
| predict foo AS predicted_foo 
| eval upper95(predicted_foo)=if(_time<=now(), predicted_foo, 'upper95(predicted_foo)' )
| eval lower95(predicted_foo)=if(_time<=now(), predicted_foo, 'lower95(predicted_foo)' )

Of course change the time frame to something reasonable (last hour, last day, whatever). NOTE that the upper95(predicted_foo) in the eval has to be inside single quotes, otherwise it'll think it's a function or something, so be careful about that syntax.

That gives, on my data, something like
Predictions only

If you have any problems applying that to your own data, let us know because I'm sure we can help!

Happy Splunking!

nplamondon
SplunkTrust
SplunkTrust

Thanks, Rich. That gets me much closer.

The only issue remaining is that I can't seem to get rid of the lower95. Your solution eliminates the upper95 just fine, but lower95 goes to 0. I'm wondering if I stumbled onto a bug...

| eval predicted=if(isnull(credits), round(predicted, 0), null())
| eval upper95(predicted)=if(_time<=now(), null(), 'upper95(predicted)' )
| eval lower95(predicted)=if(_time<=now(), null(), 'lower95(predicted)' )

prediction with zeroed lower95

0 Karma

Richfez
SplunkTrust
SplunkTrust

That's interesting.

Surely there's a typo? A special character embedded in there or something?

Try
1) Changing the order of the two items around.
2) Get rid of the lower95 part, copy and paste the upper95 again and manually change the "upper" to "lower" in each spot needed.

Number one could be interesting - will lower work and upper not now? Inquiring minds want to know!
Number two should really settle things though. I'm SURE there's just something goofy going on with some space or something.

0 Karma

nplamondon
SplunkTrust
SplunkTrust

I just noticed that my screencaps haven't been posting. Trying again...

I also tried adding | eval lower95(predicted)=if('lower95(predicted)'=0, null(), 'lower95(predicted)') for s&g's, but it was still set to 0.

Looking at the Statistics tab, there are no values where I expect lower95 to be null, which is correct, implying this is a display bug.

prediction with zeroed lower95

0 Karma

Richfez
SplunkTrust
SplunkTrust

Agreed it seems like a UI bug given that last information. I can try to recreate this again, make sure it wasn't a fluke it worked for my screenshot a few days ago.

What version of Splunk is the SH you are using?

My screenshot is from a 6.5.2 on Windows (on my laptop as a test box) using Firefox.

Let me recreate something here on 6.5.2 on Linux using various browser, see if I can make it break or not.

Richfez
SplunkTrust
SplunkTrust

Tested and confirmed I do NOT have this problem.

index=network earliest=-5m latest=-15s 
| bin _time span=15s 
| stats count AS foo by _time 
| predict foo AS predicted_foo
| eval upper95(predicted_foo)=if(_time<=now(), predicted_foo, 'upper95(predicted_foo)' )
| eval lower95(predicted_foo)=if(_time<=now(), predicted_foo, 'lower95(predicted_foo)' )

This has predicted_foo all the way across the Line Chart (as expected, I do nothing with it) with no upper/lower bounds where foo exists. Around where foo stops the upper/lower bounds of predicted_foo appear.

That's Firefox (latestish) on Windows running against a 6.5.2 Search Head.
It also works on IE in the same way.
And on Chrome.

Weird. Well anyway, I think there's a bug somewhere. Just not sure where it is because I can't find it. 🙂

When you find a proper answer (bug, fixed in 6.5.3, whatever) update us and mark something as Accepted!

nplamondon
SplunkTrust
SplunkTrust

Just for completeness (and my sanity), would you please run your query setting upper/lower to null(), rather than predicted_foo? It seems to me the problem appears when nulling lower95.

0 Karma

Richfez
SplunkTrust
SplunkTrust

Sorry, I hadn't noticed this reply.

So, interesting. There is a UI bug in here, or at least a odd behavior. There are several permutations of this:

Using null() for neither and sticking with just making each match our prediction, using null() on just upper95, using null() for lower95, or using null() for both.

Using null() for neither "correction" is our default, works fine.

Using null() for lower is the interesting one, seems that no matter what I do it shades from zero to the value for all locations where the value is null. You'd think that the "format" option would be able to fix this (on how to handle nulls), but it can't. THAT I believe is certainly a bug.

Using null() for upper is not a problem. A slightly different behavior at the point where they become non-predict_foo, because they're not following the predict_foo up until then they don't "Expand out" from predict_foo but instead just "appear".

Using null() for BOTH is also not a problem - Same behavior as above, but they both work.

It's only when ONLY lower95 is replaced with null. So, for sure a bug of some sort.

0 Karma

Richfez
SplunkTrust
SplunkTrust

After that, I saw your answer. LOL.

I even just went through and came up with screenshots of all 4 - I get odd behavior of the lower95 replacement (when used alone) regardless of settings of the formatting of null in the visualization.

I may open a ticket and see what happens with mine, then.

0 Karma

nplamondon
SplunkTrust
SplunkTrust

This is Splunk 6.5.2 running on RHEL 6.8.

Since this reeks of display bug, I'll submit one. Thanks for helping me poke at this!

0 Karma

nplamondon
SplunkTrust
SplunkTrust

Good ideas.

1) Swapping the upper/lower lines did not change the result.
2) I had already done this, thinking I had made a typo. Just for completeness, I tried it again, just now. No change.

0 Karma

cmerriman
Super Champion

can you just do a |fields - lower95 - upper96 or am I misunderstanding something else?

0 Karma

nplamondon
SplunkTrust
SplunkTrust

That would work for eliminating those fields totally, but I'd like to see them for the predicted portion.

0 Karma

cmerriman
Super Champion

what if you did the same thing for lower and upper that you do for predicted?

|eval upper95=if(isnull(foo), upper95, null())|eval lower95=if(isnull(foo), lower95, null())
0 Karma

nplamondon
SplunkTrust
SplunkTrust

`index=xxxxx sourcetype=duo:info_summary earliest=-7d

| timechart span=1h min(telephony_credits_remaining) as credits

| predict future_timespan=168 credits as predicted

| eval predicted=if(isnull(credits), round(predicted, 0), null())`

The fields I'm concerned with are named "upper95(predicted)" and "lower95(predicted)". If I try

| eval predicted=if(isnull(credits), round(predicted, 0), null()), "lower95(predicted)"=if(isnull(credits), "lower95(predicted)", null()), "upper95(predicted)"=if(isnull(credits), "upper95(predicted)", null())

those fields are 0 through the entire graph. If I don't include the quotes, I get an error calling the upper95 function (which isn't a function).

0 Karma

somesoni2
SplunkTrust
SplunkTrust

We need to see you full query to understand what all fields are available. It would be easier then to correlate to your description.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...