Can you create and apply a tag using an Event Action?

[chris2416]

I've setup several alerts to trigger within my environment and then display the most recent alerts as a search results on a "Review" panel within a dashboard. I would like to have the alerts displayed until a user manually goes in and acknowledges the alert at which point they would move to a second "Acknowledged". Ideally I wanted to set this up using an Event Action that applies only to these alerts and that action would update a tag to say that the event was acknowledged which would make it simple to create my Review and Acknowledged dashboards.

I know workflow actions are limited to links and searches but I wanted to know if anyone knew if something like this was possible within the context of Splunk?

[mic1024]

I wish splunk had 'event ack' feature available 'out of the box'....

I'm currently trying to develop a view which shows critical alerts - some of them are known, and users should be able to move them to another table in the view ' ack'ed events' (adding a comment would be nice as well, but I don't see how that could have been done right now).

@chris2416, would you be able to share more details on your workflow action ?

[mic1024]

thanks for that.
I see I'm not the only one facing the same issues... 😉

[chris2416]

I do something similar to the following as a search workflow action (after I've created the initial look up table):

index=_audit action=alert_fired sid=$sid$
| fields + sid
| eval alarm_status="acknowledged"
| inputlookup append=true alarm_lookup
| outputlookup alarm_lookup

This appends the current acknowledged alert to the look up table and then I have two separate search queries, one that searches for all action=alert_fired NOT alarm_status="acknowledge" and a second action=alert_fired alarm_status="acknowledged". This gives me my new errors as well as the acknowledged errors.

I'm not a huge fan of using the look up table and would prefer to have someway to tag events after they've been ingested. The other issue I've run into is the ability to bulk acknowledge, if I suddenly receive 100 alarms I have to acknowledge each one individually.

[chris2416 · Accepted solution]

For anyone interested, I was able to create a work around through the use of a look up table. Basically I created a workflow action that saved off the sid and a new 'Status' field with the status Acknowledged. I then limited my search to only those items that were not acknowledged for new events. I'm not sure on the limitations of look up tables in terms of how many events they can hold but this should work for my purposes although I'm running into an issue with trying to execute this workflow action from a dashboard or report. (http://answers.splunk.com/answers/172544/workflow-action-not-working-within-a-dashboard.html)

[chris2416]

It would also be nice to be able to add a tag or field for the user that acknowledged the alert and when it was acknowledged.