trigger an alert if a process is taking more than the expected time

[kranthimutyala]

Hi Splunkers,

I have the below logs and trying to create an alert if a process run is taking more than the expected time.

2021-03-24T14:00:14.8 STATUS=Successful,ACTIVITY AT=2021-03-24T14:00:14,ACTION TYPE=Process started

2021-03-24T14:05:21.54 STATUS=Successful,ACTIVITY AT=2021-03-24T14:05:21,TYPE=Process finished

Im using the below query to track the same but it is triggering an alert even if the process is completed well within the limit

index="abc" TYPE="Process started"
| eval last_seen=_time
| eval mins_since = round((now() - last_seen) / (60))
|table mins_since |search mins_since>10

From the above logs the alert shouldn't trigger since process has finished in 5mins but i'm getting false positives.The alert should trigger only if Process is not finished and mins_since>10.

I tried search TYPE!="Process finished" AND mins_since>10 but still not getting desired results.Please help me in this scenario.Thanks.

[scelikok · Accepted solution]

Hi @kranthimutyala,

Since you are not checking if Process finished or not  result may wrong. Please try this.

index="abc" TYPE="Process started"
| dedup TYPE | head 1
| eval mins_since = round((now() - _time) / 60)
| where TYPE="Process started" AND mins_since>10
| table min_since