Join the Conversation
- Find Answers
- :
- Using Splunk
- :
- Dashboards & Visualizations
- :
- Trigger alert when rolled up dashboard color goes ...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Trigger alert when rolled up dashboard color goes red
Hi all,
(refer to the screenshot attached)
I have a dashboard with 30 or so panels just like the one below. They feature a search as well as a green dot if the numbers in the last hour are within acceptable ranges.
Looking at the dashboard html below, you can see the MainTrafficSearch causes the color of the svg circle on the row to change colors.
It also updates a color named V7Color. This is the main color for the entire dashboard. It represents the worst color of any of the searches in the dashboard.
Now what I want is to configure an alert so that if the color of the V7Color goes to yellow or red, I send out an email. I would like to not have to do a massive search query which is the combination of all 30 of my searches. If possible, I would like to not have to configure 30 separate alerts (because the app is so tightly monitored that many times, when one part of the app goes bad, several do).
I also already have basically the same search in two places, if possible, I'd like to not have to write the search in a third place. (Unless it is possible to write the search one place and reference it as a variable in all the other places)
I understand that what I am hoping to do might not be possible. But if you were in my shoes, what route would you take? Thanks
<search id="MainTrafficSearch">
<query>sourcetype=provider_api_jidp env=$envselection$ index=$envselection$ "AccessTokenInterceptor" | rex field=_raw "ajp-/(?<ipthread>[0-9\.\-:]+)" | rex field=_raw "(GET|POST|PUT|DELETE|OPTIONS) (?<resource>[a-zA-Z\.]+ \(.*\)) " | dedup _time host resource ipthread | stats count | eval status=case(count > 500000, 2, count > 250000, 1, count < 100, 1, count < 10, 2, true(), 0) | table status</query>
<earliest>-60m</earliest>
<latest>$timepicker.latest$</latest>
<progress>
<eval token="MainTrafficColor">case($result.status$=="0","green",$result.status$=="2","red",$result.status$=="1","yellow",true(),"grey")</eval>
<eval token="V7Color">case($result.status$=="2","red",$result.status$=="1" AND $V7Color$ != "red","yellow",true(),$V7Color$)</eval>
</progress>
</search>
…
<row>
<panel>
<title>Traffic</title>
<html>
<div style="width:100%;height:30px;position:absolute">
<svg xmlns="http://www.w3.org/2000/svg" style="margin-left:95%">
<circle cx="13" cy="13" r="10" fill="$MainTrafficColor$"/>
</svg>
</div>
</html>
<single>
<title>Number of Calls</title>
<search>
<query>sourcetype=provider_api_jidp index=$envselection$ env=$envselection$ "AccessTokenInterceptor" | rex field=_raw "ajp-/(?<ipthread>[0-9\.\-:]+)" | rex field=_raw "(GET|POST|PUT|DELETE|OPTIONS) (?<resource>[a-zA-Z\.]+ \(.*\)) " | dedup _time host resource ipthread | stats count</query>
<earliest>$timepicker.earliest$</earliest>
<latest>$timepicker.latest$</latest>
</search>
<option name="colorMode">block</option>
<option name="drilldown">none</option>
<option name="height">50</option>
<option name="rangeColors">["0x53a051","0x0877a6","0xf8be34","0xf1813f","0xdc4e41"]</option>
<option name="refresh.display">progressbar</option>
<option name="unitPosition">before</option>
<option name="useColors">0</option>
<option name="useThousandSeparators">1</option>
</single>
</panel>
</row>
.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...
A Season of Skills: New Splunk Courses to Light Up Your Learning Journey
Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...
