Alerting

wanted to use Splunk to get notification/alerts whenever service goes down or hung

sachindarade
New Member

Hi All,

I am new to Splunk. I have few windows services in our environment. Sometime those services get hung or stopped automatically.
I wanted to use Splunk to get notification/alerts whenever service goes down or hung. If somebody can share any steps that would really appreciated.

Thanks in Advance!

0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @sachindarade,
I think that you already have Splunk Universal Forwarder on these servers, otherwise you have to install it on all.

The first step is (if you haven't yet) to download Splunk_TA_Windows ( https://splunkbase.splunk.com/app/742/ ).
then you have to edit inputs.conf file changing in [WinHostMon://Process] room the option disabled = 0
Then you have to deploy this modified TA to all your servers.
In this way you should receive a list of active processes on your servers with the frequency you have in [WinHostMon://Process] room (default 600 seconds).

Now you should create a lookup (called e.g. services.csv) containing all the processes of each host (two columns: host Name) that you want to monitor.

At least you have to run a search like this:

index=windows sourcetype=WinHostMon Type=Process host="BKKLT00067" Name="*"
| eval host=lower(host), Name=lower(name)
| stats count BY host Name
| append [ | inputlookup services.csv | eval host=lower(host), Name=lower(name), count=0 | fields host Name count ]
| stats sum(count) As total BY host Name
| where total=0

In this way you'll have the list of all services for each host never found.

Ciao.
Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @sachindarade,
I think that you already have Splunk Universal Forwarder on these servers, otherwise you have to install it on all.

The first step is (if you haven't yet) to download Splunk_TA_Windows ( https://splunkbase.splunk.com/app/742/ ).
then you have to edit inputs.conf file changing in [WinHostMon://Process] room the option disabled = 0
Then you have to deploy this modified TA to all your servers.
In this way you should receive a list of active processes on your servers with the frequency you have in [WinHostMon://Process] room (default 600 seconds).

Now you should create a lookup (called e.g. services.csv) containing all the processes of each host (two columns: host Name) that you want to monitor.

At least you have to run a search like this:

index=windows sourcetype=WinHostMon Type=Process host="BKKLT00067" Name="*"
| eval host=lower(host), Name=lower(name)
| stats count BY host Name
| append [ | inputlookup services.csv | eval host=lower(host), Name=lower(name), count=0 | fields host Name count ]
| stats sum(count) As total BY host Name
| where total=0

In this way you'll have the list of all services for each host never found.

Ciao.
Giuseppe

View solution in original post

0 Karma

sachindarade
New Member

Hi,

Similarly, i have another requirement.where I have pass log directory to forwarder. Now i want to read the logs and generate the alerts when log file contains "file(s) count is 2" or greater than 1. (condition is : File(s) count is greater than 1)

your help would be really appreciated.

Thanks in Advance.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.