Splunk Search

How to search and trigger an alert if we are not getting a record for some hosts from a given host list?

sachinsingh2005
Explorer

I have total 12 hosts which are coming through my sourcetype (input) and are below:

UK1 App Server 1
UK1 App Server 2
UK1 Worker Server 1
UK1 Worker Server 2
UK3 App Server 1
UK3 App Server 2
UK3 Worker Server 1
UK3 Worker Server 2
US2 App Server 1
US2 App Server 2
US2 Worker Server 1
US2 Worker Server 2

I have one splunk search below:

sourcetype="*Process Host" | stats count by source, host

host ----------------------------count
UK1 App Server 1---------------13
UK1 App Server 2 ---------------5
UK1 Worker Server 1-----------205
UK1 Worker Server 2-----------27
UK3 Worker Server 1-----------782
UK3 Worker Server 2-----------193
US2 App Server 1----------------1
US2 Worker Server 2------------25

From the search above, I am not getting any record for four hosts which are below:
UK3 App Server 1,
UK3 App Server 2,
US2 App Server 2,
US2 Worker Server 1

If any record is not returned for any host, then one alert should trigger on it that these hosts are not getting updated OR no record found for these hosts.

Any one please tell me how we can create this type of Alert?

Thanks in advance.
Sachin Singh

0 Karma
1 Solution

sachinsingh2005
Explorer

Yes, i have fix it by applying below search query. it is working and giving expected result:

sourcetype="*" | stats max(_time) as last_time by host, sourcetype | eval latency_minutes=((now()-last_time)/60) | convert ctime(last_time) as last_time | fields + host, sourcetype, last_time, latency_minutes

View solution in original post

mariya
Engager

Hai All
Can any tell me the solution for the above host problem.

I am also facing the same issue

Thanks

0 Karma

mariya
Engager

Anyone has correct query to solve the host problem

0 Karma

sachinsingh2005
Explorer

Yes, i have fix it by applying below search query. it is working and giving expected result:

sourcetype="*" | stats max(_time) as last_time by host, sourcetype | eval latency_minutes=((now()-last_time)/60) | convert ctime(last_time) as last_time | fields + host, sourcetype, last_time, latency_minutes

sachinsingh2005
Explorer

Hi Rajneesh,

I was checking the solution for this problem but i din't find so i have change my expectation and use above query to solve my problem but I am still finding solution for my original query and get back to you after that. we both have same situation.

Regards
Sachin

0 Karma

rajnish1202
Explorer

Thanks Sachin,
Lets share the solution whoever gets this first.

Regards,
Rajnish

0 Karma

sachinsingh2005
Explorer

if you like the answer then vote for my answer.

0 Karma

mariya
Engager

Hai can any tell me the solution for the above host problem.

I am also facing the same issue

Thanks

0 Karma

rajnish1202
Explorer

Hi Sachin,
Thanks for the response. I am really not sure how above search will list server with count as 0.

I am having a below problem to solve. Please let me know if you can help on this.

I have a search query like
index=tpapps host=* sourcetype="Script:WinService" state=STOPPED |stats count by host

This search query gives me number of services stopped on each host. results are something like this

Host StoppedServices
Host1 2
Host2 1

But the problem with search is that it does not return a row if there is no services stopped on a host. I want to list the host even when there is no service stopped on it. It should show 0 services in StoppedServices column. Something like below

Host StoppedServices
Host1 2
Host2 1
Host3 0

Many thanks in advance.

Regards,
Rajnish Kumar

0 Karma

rajnish1202
Explorer

Hi Sachin,
Did you get the solution for this issue? I am also facing similar issue. If you got the solution please share.

Thanks in advance.

Rajnish Kumar

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...