Splunk Search

project trendlines into future

ddholstadz
Explorer

Is there a way to make trendline project moving averages into the future?

Tags (2)
1 Solution

Paolo_Prigione
Builder

There actually is no easy way, I fear. You'd need to:

  1. compute the trendline equation to do that (y = m * _time + b) see http://www.tutorvista.com/content/math/geometry/straightlines/two-point-form.php
  2. extend the time field into future
  3. compute the new y over time (the easy part...just an eval)

But.... which is the best window to compute your trendline upon? 5, 20, 30, 1000 events? That totally depends on the case...

Ok, let's move on...here's my approach, in bullet point (I'll use _time as x axis, y as y axis):

  1. You need to compute the best trendline you see fit your data and produce a field "y"
  2. To compute the equation of a line you need 2 (x,y) couples, which you can produce by moving the previous event's y and _time values to the current event. I'll use autoregress and name the two points as (curr_time,curr_y) (prev_time,prev_y)
  3. You do the math and compute slope (m) and y-intercept (b) -> here's your equation!
  4. Now, you said you want the future...so you don't have data for it. You'll have to "gentimes", and then put your slope and intercept into each event.
  5. You compute the predicted value of y
  6. You chart y over time

Here's my try.

| gentimes start=01/01/11 end=02/28/11 increment=6h 
| eval jf=1 
| join jf [

Get a time span and prepare to join the m and b values to all the results:

search <you search and computation of y here>
| autoregress y as prev_y  
| autoregress _time as prev_time 
| rename y as curr_y 
| eval curr_time=_time 
| head 1

Head 1 gets the latest event only, which now has data for the 2 points the prediction line will pass through. Now I'll do the math

| eval m=(curr_y - prev_y)/(curr_time - prev_time) 
| eval b=(prev_y * curr_time - curr_y * prev_time) / (curr_time - prev_time) 
| eval jf=1 
| fields + m b jf
] 

I now have a single result with three fields only, jf (join field) is just for the join operation.

| eval y= m*starttime + b
| eval _time=starttime
| chart values(y) over _time

Your predicted y value for the future.

View solution in original post

Paolo_Prigione
Builder

There actually is no easy way, I fear. You'd need to:

  1. compute the trendline equation to do that (y = m * _time + b) see http://www.tutorvista.com/content/math/geometry/straightlines/two-point-form.php
  2. extend the time field into future
  3. compute the new y over time (the easy part...just an eval)

But.... which is the best window to compute your trendline upon? 5, 20, 30, 1000 events? That totally depends on the case...

Ok, let's move on...here's my approach, in bullet point (I'll use _time as x axis, y as y axis):

  1. You need to compute the best trendline you see fit your data and produce a field "y"
  2. To compute the equation of a line you need 2 (x,y) couples, which you can produce by moving the previous event's y and _time values to the current event. I'll use autoregress and name the two points as (curr_time,curr_y) (prev_time,prev_y)
  3. You do the math and compute slope (m) and y-intercept (b) -> here's your equation!
  4. Now, you said you want the future...so you don't have data for it. You'll have to "gentimes", and then put your slope and intercept into each event.
  5. You compute the predicted value of y
  6. You chart y over time

Here's my try.

| gentimes start=01/01/11 end=02/28/11 increment=6h 
| eval jf=1 
| join jf [

Get a time span and prepare to join the m and b values to all the results:

search <you search and computation of y here>
| autoregress y as prev_y  
| autoregress _time as prev_time 
| rename y as curr_y 
| eval curr_time=_time 
| head 1

Head 1 gets the latest event only, which now has data for the 2 points the prediction line will pass through. Now I'll do the math

| eval m=(curr_y - prev_y)/(curr_time - prev_time) 
| eval b=(prev_y * curr_time - curr_y * prev_time) / (curr_time - prev_time) 
| eval jf=1 
| fields + m b jf
] 

I now have a single result with three fields only, jf (join field) is just for the join operation.

| eval y= m*starttime + b
| eval _time=starttime
| chart values(y) over _time

Your predicted y value for the future.

Paolo_Prigione
Builder

It felt like secondary school, solving line equations...just funnier. Thanks Lowell, much appreciated!

0 Karma

Lowell
Super Champion

Wow, that's pretty intense. Looks like this may be a good candidate for a macro; I'd hate to have to retype that several times. 😉 Nice work.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...