Alerting

Trigger a report based on an event

brettcave
Builder

Is it possible (and how) to trigger a report to be run based on an event? I have a batch processor that logs to splunk. there are 2 types of events - 1 is job metadata, and the other is job run specifics:

JOB METADATA action=start name=MyJob runId=foobar
JOB DETAIL id=1 action=update result=pass
JOB DETAIL id=2 action=update result=fail
JOB DETAIL id=3 action=delete result=pass
JOB DETAIL id=4 action=insert result=fail
JOB METADATA action=end name=MyJob duration=6300 runId=foobar

given these events, I could create a saved search called FailedModifications that gives all the details where result!=pass. But I would only like to run this report for runId=foobar (runId actually uses a date/time stamp), and only run it once the job completes. Something along the lines of using this search: eventtype=AJobAction action=end as a trigger for my "FailedModifications" saved search to run with an extra "runId" parameter. The FailedModifications search is configured as an alert that emails results (this is a requirement of what I'm trying to configure here).

Currently, I'm scheduling the FailedModifications report to run on a cron schedule, with a window matching the schedule intervals, but this is not an ideal configuration. Possible with splunk? if so, how?

0 Karma

yannK
Splunk Employee
Splunk Employee

The main search looks like a transaction starting with action=start and finishing with action=end. I hope that you do not have multiple jobs in parallel, otherwise you need a field to join them, maybe the source...

If you are using a scheduled search, you can have your report calculated every time, but only sent if a condition is met. (presence of action=end and of result=fail)

it can be done by a simple | WHERE action=end AND of result=fail condition at the very end of the search, and an alert based on "if number of results > 0".

0 Karma

brettcave
Builder

cool, thanks yannK. Will give it a try and post back.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...