Splunk AppDynamics

Retrieving historic metrics and applying complex calculation logic.

Tony_Ursich
New Member

Hello Team,

I presently have a single numeric value that is published to AppD every day, below is a sample of this extract:

  • name=Custom Metrics|GRA|Pipeline View|WinSec|Total Input, value=7366, aggregator=AVERAGE, time-rollup=AVERAGE, cluster-rollup=INDIVIDUAL

Over the course of a month we will have a curve with 30x data points, as depicted by the screenshot below.

Below are the requirements that we are trying to achieve, but I do not know where to start with the design and implementation, and would really appreciate some tips:

----------------------------------------------------------------------

Variance Warning Alert

The warning alert will be based on a comparison of todays metric versus the metric from the previous day or week, based on the below rules

Resource: Daily cadence report located below.

Rules:

Sat total is compared to prev Sat total
Sun total is compared to prev Sun total
Mon total is compared to prev Mon total
Tues total is compared to Mon (y'day) total
Wed total is compared to Tues (y'day) total
Thur total is compared to Wed (y'day) total
Fri total is compared to Thurs (y'day) total

Calculation (Day to day variation):

allowableWarningMarginValue = todays metric * 0.5
allowableCriticalMarginValue = todays metric * 0.8
ingestVariation = Abs(todays total - comparison day total)
If (ingestVariation >= allowableWarningMarginValue and ingestVariation < allowableCriticalMarginValue) then
  set heatmap value = AMBER
  generate a warning message
Else If (ingestVariation >= allowableCriticalMarginValue) then
  set heatmap value = RED
  generate a critical error message
Else
set heatmap value = GREEN
Endif


Example (Using Friday June 23 WinSec pipeline):
Todays Total (Fri June 23) = 7366
Comparison Total (Thurs June 22) = 300
allowableWarningMarginValue = 7366 * 0.5 = 3683
allowableCriticalMarginValue = 7366 * 0.8 = 5892
ingestVariation = abs(7366 - 300) = 7066

if ingestVariation (7066) > allowableCriticalMarginValue (5892)
  set heatmap value = RED
  generate a critical message
else if ingestVariation (7066) > allowableWarningMarginValue (3683)
  set heatmap value = AMBER
  generate a warning message
else
set heatmap value = GREEN
endif


AppD Result:

For a given day I would like to simply output a single heatmap colour. The below screenshot illustrates how this heatmap colour would look over successive days, but each daily heatmap colour is simply overridden each day.

image.png

Thank you in advance for any help you can provide.

Kind regards,

Tony.

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...