Generating alert based on condition

[Ashutosh_30]

Hi All ,

i have configured alerts for the search below:

index="ebs_red_0" host="dev-obiee-ux0*" source="/obiee_12c/app/oracle/product/12212/user_projects/domains/bi/nodemanager/nodemanager.log" "waiting for the process to die"

Output :

8/3/21 9:38:11.000 AM

dev-obiee-ux08

The server 'obips2' with process id 12714242 is no longer alive; waiting for the process to die.

obips2

obiee:nodemanager:log

Aug 3, 2021 5:38:11 AM EDT

but sometimes when my server process dies it restarts automatically within a 60 seconds which can be described as :

index="ebs_red_0" host="dev-obiee-ux0*" source="/obiee_12c/app/oracle/product/12212/user_projects/domains/bi/nodemanager/nodemanager.log" "is running now"

Output : 

8/3/21 9:39:27.000 AM

dev-obiee-ux08

The server 'obis2' is running now.

obis2

obiee:nodemanager:log

Aug 3, 2021 5:39:27 AM EDT

So i want to write the search query in a way so that i generate alert only if the server process dies and doesn't come up again within 120 seconds.


the five fields used in the search are :

_time, host ,Message ,OBIEE_Comp, sourcetype ,time

and to generate the alert the OBIEE_Comp needs to be same 

[Ashutosh_30]

This works but the problem is i'm getting all the results in single row.
for this :
index="ebs_red_0" host="dev-obiee-ux0*" source="/obiee_12c/app/oracle/product/12212/user_projects/domains/bi/nodemanager/nodemanager.log" "Server was killed" OR "waiting for the process to die"
| stats earliest(_time) as _time,values(time) as time,values(host) as host, values(Message) as Message, values(OBIEE_Comp) as OBIEE_Comp , values(sourcetype) as sourcetype, range(_time) as time_gap | where time_gap > 120 | table time, host ,Message ,OBIEE_Comp, sourcetype ,time_gap

i want to get every record in a separate row

[venkatasri]

Hi @Ashutosh_30 

Try this and schedule it for every 5 mins,

index="ebs_red_0" host="dev-obiee-ux0*" source="/obiee_12c/app/oracle/product/12212/user_projects/domains/bi/nodemanager/nodemanager.log" "waiting for the process to die" OR "is running now"
| stats earliest(_time) as time, values(Message) as message, values(OBIEE_Comp) as OBIEE_Comp , values(sourcetype) as sourcetype, range(_time) as time_gap
| where time_gap > 120
| table _time, host ,Message ,OBIEE_Comp, sourcetype ,time_gap