Splunk Search
Highlighted

How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

I have total 12 hosts which are coming through my sourcetype (input) and are below:

UK1 App Server 1
UK1 App Server 2
UK1 Worker Server 1
UK1 Worker Server 2
UK3 App Server 1
UK3 App Server 2
UK3 Worker Server 1
UK3 Worker Server 2
US2 App Server 1
US2 App Server 2
US2 Worker Server 1
US2 Worker Server 2

I have one splunk search below:

sourcetype="*Process Host" | stats count by source, host

host ----------------------------count
UK1 App Server 1---------------13
UK1 App Server 2 ---------------5
UK1 Worker Server 1-----------205
UK1 Worker Server 2-----------27
UK3 Worker Server 1-----------782
UK3 Worker Server 2-----------193
US2 App Server 1----------------1
US2 Worker Server 2------------25

From the search above, I am not getting any record for four hosts which are below:
UK3 App Server 1,
UK3 App Server 2,
US2 App Server 2,
US2 Worker Server 1

If any record is not returned for any host, then one alert should trigger on it that these hosts are not getting updated OR no record found for these hosts.

Any one please tell me how we can create this type of Alert?

Thanks in advance.
Sachin Singh

0 Karma
Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

Explorer

Hi Sachin,
Did you get the solution for this issue? I am also facing similar issue. If you got the solution please share.

Thanks in advance.

Rajnish Kumar

0 Karma
Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

Yes, i have fix it by applying below search query. it is working and giving expected result:

sourcetype="*" | stats max(time) as lasttime by host, sourcetype | eval latencyminutes=((now()-lasttime)/60) | convert ctime(lasttime) as lasttime | fields + host, sourcetype, lasttime, latencyminutes

View solution in original post

Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

if you like the answer then vote for my answer.

0 Karma
Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

Explorer

Hi Sachin,
Thanks for the response. I am really not sure how above search will list server with count as 0.

I am having a below problem to solve. Please let me know if you can help on this.

I have a search query like
index=tpapps host=* sourcetype="Script:WinService" state=STOPPED |stats count by host

This search query gives me number of services stopped on each host. results are something like this

Host StoppedServices
Host1 2
Host2 1

But the problem with search is that it does not return a row if there is no services stopped on a host. I want to list the host even when there is no service stopped on it. It should show 0 services in StoppedServices column. Something like below

Host StoppedServices
Host1 2
Host2 1
Host3 0

Many thanks in advance.

Regards,
Rajnish Kumar

0 Karma
Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

Engager

Hai can any tell me the solution for the above host problem.

I am also facing the same issue

Thanks

0 Karma
Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

Hi Rajneesh,

I was checking the solution for this problem but i din't find so i have change my expectation and use above query to solve my problem but I am still finding solution for my original query and get back to you after that. we both have same situation.

Regards
Sachin

0 Karma
Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

Explorer

Thanks Sachin,
Lets share the solution whoever gets this first.

Regards,
Rajnish

0 Karma
Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

Engager

Anyone has correct query to solve the host problem

0 Karma
Highlighted

Re: How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

Engager

Hai All
Can any tell me the solution for the above host problem.

I am also facing the same issue

Thanks

0 Karma