topic Re: How do you set up an alert for when an application process is not running or hung? in Alerting

How do you set up an alert for when an application process is not running or hung?

bsaujla131984 — Thu, 23 Aug 2018 23:04:16 GMT

There are a number of application processes in our environment which either go down or stop responding. I am trying to setup an alert in the event if process is down or hung in Unix/Linux.

Can anyone assist with this please?

Re: How do you set up an alert for when an application process is not running or hung?

jkat54 — Fri, 24 Aug 2018 10:25:22 GMT

Do you have a way of knowing if the process is hung?

ie: curl to see if port is open, or run service $name status

Re: How do you set up an alert for when an application process is not running or hung?

inventsekar — Fri, 24 Aug 2018 10:39:13 GMT

from a linux forum... "ps auxww |grep 'process_ID' " you can have current process' status like S (Sleep), R (Running) , Z (zombie) etc..

[root@server]# cat /proc/2324/status
Name: kmpathd/1
State: S (sleeping)
SleepAVG: 98%
Tgid: 2324
Pid: 2324
PPid: 13
TracerPid: 0
Uid: 0 0 0 0
Gid: 0 0 0 0
FDSize: 32

Also, by using top command, you can see if the running process is taking high cpu(probably, hung process)

then you can create an alert accordingly..

Re: How do you set up an alert for when an application process is not running or hung?

mstjohn_splunk — Fri, 24 Aug 2018 22:25:40 GMT

Hi @bsaujla131984 - Did @inventsekar 's answer below help provide a solution to your question? If yes, please click “Accept” below the best answer to resolve this post and upvote anything that was helpful. If no, please leave a comment with more feedback. Thanks.

Re: How do you set up an alert for when an application process is not running or hung?

bsaujla131984 — Fri, 24 Aug 2018 23:59:15 GMT

Actually I want something which can be done entirely through Splunk instead of logging to UNIX/Linux serves..

Re: How do you set up an alert for when an application process is not running or hung?

bsaujla131984 — Sat, 25 Aug 2018 00:00:38 GMT

Hi @mstjohn... I would need more info. The answer is given is mainly related to commands, however I would like to do it through Splunk...

Re: How do you set up an alert for when an application process is not running or hung?

ddrillic — Sat, 25 Aug 2018 00:03:42 GMT

It's interesting in the context of Splunk processes which can be hung, forwarders or indexers...

Re: How do you set up an alert for when an application process is not running or hung?

mstjohn_splunk — Sat, 25 Aug 2018 00:11:35 GMT

Gotcha, thanks bud! Good luck with your issue.

Re: How do you set up an alert for when an application process is not running or hung?

bsaujla131984 — Sat, 25 Aug 2018 00:29:00 GMT

This is not related to Splunk process. It is related to application process in Linux/Unix servers.