Splunk ITSI

ITSI tracked alert events does not show the field to be able to use it in email alert

abhi04
Communicator

I have a KPI for Instance status:

 

index=xxxxx  source="yyyyy"
| eval UpStatus=if(Status=="up",1,0)
| stats last(UpStatus) as val by Instance host Status

Now the val is 0 or 1 and Status field is Up or Down 

The split by field is Instance host
and threshold is based on val 


The alert triggers fine but I want to put the field in email alert $result.Status$ Instead of $result.val$
But I dont see the field Status in tracked alerts. How can I make this field Status shows in tracked alerts index or events generated so that I can use it in my email (This is to avoid confusuion, instead of saying 0, 1 it will say up or down)

Labels (1)
Tags (1)
0 Karma

skramp
SplunkTrust
SplunkTrust

first, you should avoid using stats commands while creating KPIs, sometimes it's not possible but in this case you can do all the stats-magic in the KPI calculation. After this, your alarms will be in the index itsi_tracked_alerts. When you have those alerts, they must be put into Episode(s) by a Notable Event Aggregation Policy. With this NEAP you bundle events by a specific criteria. And when you process events you can also trigger something by this NEAP like sending a email. You can use a lookup to figure out, which email should be used ie by hostname or department or whatever.

0 Karma

PrewinThomas
Motivator

@abhi04 

Try to include Status field explicitly,

index=xxxxx source="yyyyy"
| eval UpStatus=if(Status=="up",1,0)
| stats last(UpStatus) as val, latest(Status) as Status by Instance host

Regards,
Prewin
Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!

0 Karma

livehybrid
SplunkTrust
SplunkTrust

Hi @abhi04 

Are you using Aggregation Policies to trigger your alerts, or KPI Alerts?

Im not sure how to achieve this with KPI Alerts, but if you are using aggregation policies then you might be able to add some logic in here (similar to how you would apply a lookup) to do an eval based on the value. 

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

0 Karma

abhi04
Communicator

Hi @livehybrid 

This KPI is used by different service for their Instance down alert as KPI metric and I am using aggregation policies for these services to trigger email alert.

 

I don't see any option to include this field from NEAPs.

Also, 

even changing the query does not help, as its is not creating any field Status in its events which I can refer.


index=xxxxx    source="yyyyyy"
| eval UpStatus=if(Status=="up",1,0)
| stats last(UpStatus) as val last(Status) as Status by Instance host

0 Karma
Get Updates on the Splunk Community!

.conf25 Community Recap

Hello Splunkers, And just like that, .conf25 is in the books! What an incredible few days — full of learning, ...

Splunk App Developers | .conf25 Recap & What’s Next

If you stopped by the Builder Bar at .conf25 this year, thank you! The retro tech beer garden vibes were ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...