Dashboards & Visualizations

Can you create and apply a tag using an Event Action?

chris2416
Explorer

I've setup several alerts to trigger within my environment and then display the most recent alerts as a search results on a "Review" panel within a dashboard. I would like to have the alerts displayed until a user manually goes in and acknowledges the alert at which point they would move to a second "Acknowledged". Ideally I wanted to set this up using an Event Action that applies only to these alerts and that action would update a tag to say that the event was acknowledged which would make it simple to create my Review and Acknowledged dashboards.

I know workflow actions are limited to links and searches but I wanted to know if anyone knew if something like this was possible within the context of Splunk?

1 Solution

chris2416
Explorer

For anyone interested, I was able to create a work around through the use of a look up table. Basically I created a workflow action that saved off the sid and a new 'Status' field with the status Acknowledged. I then limited my search to only those items that were not acknowledged for new events. I'm not sure on the limitations of look up tables in terms of how many events they can hold but this should work for my purposes although I'm running into an issue with trying to execute this workflow action from a dashboard or report. (http://answers.splunk.com/answers/172544/workflow-action-not-working-within-a-dashboard.html)

View solution in original post

mic1024
Path Finder

I wish splunk had 'event ack' feature available 'out of the box'....

I'm currently trying to develop a view which shows critical alerts - some of them are known, and users should be able to move them to another table in the view ' ack'ed events' (adding a comment would be nice as well, but I don't see how that could have been done right now).

@chris2416, would you be able to share more details on your workflow action ?

mic1024
Path Finder

thanks for that.
I see I'm not the only one facing the same issues... 😉

0 Karma

chris2416
Explorer

I do something similar to the following as a search workflow action (after I've created the initial look up table):

index=_audit action=alert_fired sid=$sid$
| fields + sid
| eval alarm_status="acknowledged"
| inputlookup append=true alarm_lookup
| outputlookup alarm_lookup

This appends the current acknowledged alert to the look up table and then I have two separate search queries, one that searches for all action=alert_fired NOT alarm_status="acknowledge" and a second action=alert_fired alarm_status="acknowledged". This gives me my new errors as well as the acknowledged errors.

I'm not a huge fan of using the look up table and would prefer to have someway to tag events after they've been ingested. The other issue I've run into is the ability to bulk acknowledge, if I suddenly receive 100 alarms I have to acknowledge each one individually.

0 Karma

chris2416
Explorer

For anyone interested, I was able to create a work around through the use of a look up table. Basically I created a workflow action that saved off the sid and a new 'Status' field with the status Acknowledged. I then limited my search to only those items that were not acknowledged for new events. I'm not sure on the limitations of look up tables in terms of how many events they can hold but this should work for my purposes although I'm running into an issue with trying to execute this workflow action from a dashboard or report. (http://answers.splunk.com/answers/172544/workflow-action-not-working-within-a-dashboard.html)

chris2416
Explorer

It would also be nice to be able to add a tag or field for the user that acknowledged the alert and when it was acknowledged.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...