Splunk Search

How to create a single value panel that changes based on weighted values?

Contributor

I want to create a single value panel that starts at 100, and when a specific alert goes off with an assigned weight, that weight is removed from the single value panel. So alert with a weight of 25 fires, the single value panel is now at 75 which is yellow.

Greater than 80 is green
50-80 is yellow
below 50 is red

alt text

I have this search:

index="nitro_prod_summary" earliest=-1h@m latest=@m [| `nitro_prod_cmdb` | search Category="ECOMM CUSTOMER FACING"  OR Category="ECOMM OPERATIONS" Service="*" Application="*" weight="*"| stats count by Application | table Application] | join Application [ | `nitro_prod_cmdb` ] | search Alert_Type="*" Metric_Category="*" | eval FilterKey=Description.ID | dedup FilterKey | table Alert_Type Category Service Application Metric_Category Description Key ID| rename Metric_Category as "Type" Alert_Type as "Alert" count as Count | sort +Alert

Which creates a table like this:
alt text

I want to create this single panel for "webstore" so I want to filter from the search above only Application="WebStore Services" then add the weights from all of the instances and subtract it from 100, if this number is greater than 80 it is green, 50-80 is yellow and below 50 is red. It should also still say webstore and the color should be responding to that number.

I did not put weight into the table but it is being pulled from the alerts just not being placed anywhere.

Any ideas on making this single value panel?
Few extra points for this loaded question!

0 Karma
1 Solution

Legend

See if this gets you going in the right direction

 index="nitro_prod_summary" earliest=-1h@m latest=@m [| `nitro_prod_cmdb` | search Category="ECOMM CUSTOMER FACING"  OR Category="ECOMM OPERATIONS" Service="*" Application="*" weight="*"| stats count by Application | table Application] | join Application [ | `nitro_prod_cmdb` ] | search Alert_Type="*" Metric_Category="*" | eval FilterKey=Description.ID | dedup FilterKey | table Alert_Type Category Service Application Metric_Category Description Key ID weights | rename Metric_Category as "Type" Alert_Type as "Alert" count as Count | sort +Alert where match(Application, "([Ww]eb[Ss]tore)" | stats sum(weights) as weight | eval weight=100-weight | rangemap field=weight low=100-80 elevated=50-79 default=severe

View solution in original post

0 Karma

Legend

See if this gets you going in the right direction

 index="nitro_prod_summary" earliest=-1h@m latest=@m [| `nitro_prod_cmdb` | search Category="ECOMM CUSTOMER FACING"  OR Category="ECOMM OPERATIONS" Service="*" Application="*" weight="*"| stats count by Application | table Application] | join Application [ | `nitro_prod_cmdb` ] | search Alert_Type="*" Metric_Category="*" | eval FilterKey=Description.ID | dedup FilterKey | table Alert_Type Category Service Application Metric_Category Description Key ID weights | rename Metric_Category as "Type" Alert_Type as "Alert" count as Count | sort +Alert where match(Application, "([Ww]eb[Ss]tore)" | stats sum(weights) as weight | eval weight=100-weight | rangemap field=weight low=100-80 elevated=50-79 default=severe

View solution in original post

0 Karma

Contributor

Not quite, nothing is showing up here. I would need this to work if there were no active alerts as well, but right now there are but it isn't producing anything.

0 Karma

Legend

Did you add the weights field to the table?

0 Karma

Contributor

I did but to no avail

0 Karma

Legend

Try the updated query

0 Karma

Contributor

Hmm still not working, here is an example of my alerts hopefully this can help

[nitro_prod_stores__500_alert]
action.email.inline = 1
action.summary_index = 1
action.summary_index._name = nitro_prod_summary
alert.expires = 10s
alert.suppress = 0
alert.track = 1
auto_summarize.dispatch.earliest_time = -1d@h
cron_schedule = 03 * * * *
description = Webstore crosses 500
enableSched = 1
realtime_schedule = 0
search = index=nitro_webstore  "500" NOT("*.500" OR "200") earliest=-80m@m latest=-20m@m | stats count | where count>74 |**eval weight = 50** |eval Metric_Category="Application" | eval Metric="Health" | eval Metric_Type="Status" | eval Application="Webstore Services" | eval Key="Count" | eval Frequency="60 minutes" | eval ID="NA" | eval Description="Webstore crosses 500" | eval Value=coalesce(NA,count) | eval Alert_Type="low" | eval Alert="Yes" | eval Service-Now_Assignment_Group="EC-IScore" | eval Violation="1" | eval _time=now() | table _time Metric_Category Metric Metric_Type Application Key ID Description Frequency Value Alert_Type Alert Service-Now_Assignment_Group Violation Search_name
0 Karma

Legend

What do you get when you keep only the where segement and remove everything else? Can you copy the list of fields you see?

0 Karma

Contributor

"count
112"

0 Karma

Legend

My bad. I meant keep the where and remove everything else from the query I posted. Not the alert query.

0 Karma

Contributor

So I tried that and I got no results but when I removed the weight category I received the original table again as shown in the first post

0 Karma

Legend

Good. So where does the Weight come from? Is it a field in the main search? or one of the sub-searches? That field is needed for the query to execute

0 Karma

Contributor

Sorry for the late reply, I got the search working but now I need it to return green even when the search provides no results because no alerts are firing currently? Any suggestions?

0 Karma

Contributor

Ok so made some edits, sorry for the delay. The search now works but I need it to still display when there are no alerts coming in, any suggestions for that? Right now it will just say "no results"

0 Karma

Contributor

Makes sense, my bad. I will get all of this set up correctly, and get back here.

0 Karma

Contributor

Can I append weight to the main search? It is in the alerts

0 Karma