Splunk Search

project trendlines into future

ddholstadz
Explorer

Is there a way to make trendline project moving averages into the future?

Tags (2)
1 Solution

Paolo_Prigione
Builder

There actually is no easy way, I fear. You'd need to:

  1. compute the trendline equation to do that (y = m * _time + b) see http://www.tutorvista.com/content/math/geometry/straightlines/two-point-form.php
  2. extend the time field into future
  3. compute the new y over time (the easy part...just an eval)

But.... which is the best window to compute your trendline upon? 5, 20, 30, 1000 events? That totally depends on the case...

Ok, let's move on...here's my approach, in bullet point (I'll use _time as x axis, y as y axis):

  1. You need to compute the best trendline you see fit your data and produce a field "y"
  2. To compute the equation of a line you need 2 (x,y) couples, which you can produce by moving the previous event's y and _time values to the current event. I'll use autoregress and name the two points as (curr_time,curr_y) (prev_time,prev_y)
  3. You do the math and compute slope (m) and y-intercept (b) -> here's your equation!
  4. Now, you said you want the future...so you don't have data for it. You'll have to "gentimes", and then put your slope and intercept into each event.
  5. You compute the predicted value of y
  6. You chart y over time

Here's my try.

| gentimes start=01/01/11 end=02/28/11 increment=6h 
| eval jf=1 
| join jf [

Get a time span and prepare to join the m and b values to all the results:

search <you search and computation of y here>
| autoregress y as prev_y  
| autoregress _time as prev_time 
| rename y as curr_y 
| eval curr_time=_time 
| head 1

Head 1 gets the latest event only, which now has data for the 2 points the prediction line will pass through. Now I'll do the math

| eval m=(curr_y - prev_y)/(curr_time - prev_time) 
| eval b=(prev_y * curr_time - curr_y * prev_time) / (curr_time - prev_time) 
| eval jf=1 
| fields + m b jf
] 

I now have a single result with three fields only, jf (join field) is just for the join operation.

| eval y= m*starttime + b
| eval _time=starttime
| chart values(y) over _time

Your predicted y value for the future.

View solution in original post

Paolo_Prigione
Builder

There actually is no easy way, I fear. You'd need to:

  1. compute the trendline equation to do that (y = m * _time + b) see http://www.tutorvista.com/content/math/geometry/straightlines/two-point-form.php
  2. extend the time field into future
  3. compute the new y over time (the easy part...just an eval)

But.... which is the best window to compute your trendline upon? 5, 20, 30, 1000 events? That totally depends on the case...

Ok, let's move on...here's my approach, in bullet point (I'll use _time as x axis, y as y axis):

  1. You need to compute the best trendline you see fit your data and produce a field "y"
  2. To compute the equation of a line you need 2 (x,y) couples, which you can produce by moving the previous event's y and _time values to the current event. I'll use autoregress and name the two points as (curr_time,curr_y) (prev_time,prev_y)
  3. You do the math and compute slope (m) and y-intercept (b) -> here's your equation!
  4. Now, you said you want the future...so you don't have data for it. You'll have to "gentimes", and then put your slope and intercept into each event.
  5. You compute the predicted value of y
  6. You chart y over time

Here's my try.

| gentimes start=01/01/11 end=02/28/11 increment=6h 
| eval jf=1 
| join jf [

Get a time span and prepare to join the m and b values to all the results:

search <you search and computation of y here>
| autoregress y as prev_y  
| autoregress _time as prev_time 
| rename y as curr_y 
| eval curr_time=_time 
| head 1

Head 1 gets the latest event only, which now has data for the 2 points the prediction line will pass through. Now I'll do the math

| eval m=(curr_y - prev_y)/(curr_time - prev_time) 
| eval b=(prev_y * curr_time - curr_y * prev_time) / (curr_time - prev_time) 
| eval jf=1 
| fields + m b jf
] 

I now have a single result with three fields only, jf (join field) is just for the join operation.

| eval y= m*starttime + b
| eval _time=starttime
| chart values(y) over _time

Your predicted y value for the future.

Paolo_Prigione
Builder

It felt like secondary school, solving line equations...just funnier. Thanks Lowell, much appreciated!

0 Karma

Lowell
Super Champion

Wow, that's pretty intense. Looks like this may be a good candidate for a macro; I'd hate to have to retype that several times. 😉 Nice work.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...