topic Alerts and Nested Searches in Alerting

Alerts and Nested Searches

phoenixdigital — Wed, 15 Feb 2012 01:53:19 GMT

Hi All,

I have a system which performs some averaging on a 30 minute window of data coming in based on a data feed using this search. (We will call this search A)

sourcetype="reports" REGIONID=NSW1 earliest=-60m latest=+60m | dedup _time,REGIONID  | eval currentPeriodHour = strftime(now(), "%H") | eval currentPeriodMin = strftime(now(), "%M") | eval currentPeriod=floor((currentPeriodHour*60+currentPeriodMin)/30)+1 | eval period=floor((date_hour*60+date_minute-5)/30)+1 | eval inCurrentPeriod = if(currentPeriod == period, "t", "f") | search inCurrentPeriod="t" | stats avg(RRP) as avg, max(_time) as _time by REGIONID,period, currentPeriod, currentPeriodHour, currentPeriodMin  | table period, currentPeriod, currentPeriodHour, currentPeriodMin, REGIONID, avg, _time

Trust me that search works 🙂

Now this is part of an alert with a custom condition of

search REGIONID=NSW1 avg < 50

The problem is that this alert pretty much fires all the time. I would only like it to fire if another search is met based on a completely unrelated set of data. (We will call this search B)

sourcetype=holdingRegisters SPLUNK=StationStatusCoil station=hunter | dedup station | search stationStatus >= 0

So is it possible to even do this? Each search has completely independent data sets so merging them into the one search would be difficult. Not only that it would make Splunk work even harder than it should need to.

Preferably I would like to perform search B and if that has data then perform search A and if that has data fire the alert.

Is what I am wanting to do even possible?

Re: Alerts and Nested Searches

imrago — Wed, 15 Feb 2012 15:16:26 GMT

if search B meets the alert criteria it could run a script which could start the search A

Re: Alerts and Nested Searches

phoenixdigital — Mon, 20 Feb 2012 01:52:06 GMT

Thanks for the response. Unfortunately this wont really work as I want the alert to trigger a script which will create control files for an external application. So both searches need to be performed before the alert fires.

After some research I think I will need to make a custom search command.
http://docs.splunk.com/Documentation/Splunk/latest/developer/searchscripts

The documentation seems a bit loose and I have been unable to get one working yet even with examples. I will report back with a solution if I manage to get it sorted out.

Re: Alerts and Nested Searches

phoenixdigital — Fri, 23 Mar 2012 00:55:26 GMT

Just a heads up that this issue has been resolved with a custom search command.