Alerting

Get alerted if same error happening on multiple hosts

jrlesch
New Member

I have the same suite of tests running on multiple hosts. I want to set up an alert if a specific test fails on 2 or more hosts.

sourcetype=teamcity LogFailure name="'tests/cluster/pay/*'"

is my search to get all the tests the failed. The "name" is the name of the test.

How do I tell if the test is throwing the same LogFailure on multiple hosts.

Thanks

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Try this

sourcetype=teamcity LogFailure name="'tests/cluster/pay/*'" | stats dc(host) as count | where count>=2 

Set an alert if above search returns any result.

Update

Try this

sourcetype=teamcity LogFailure name="'tests/cluster/pay/*'" | stats dc(host) as count  by name| where count>=2 

This should get distinct count of host for each values of name i.e. test names and will list of tests which have failed in more than 1 hosts. The alert condition should be same.

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Try this

sourcetype=teamcity LogFailure name="'tests/cluster/pay/*'" | stats dc(host) as count | where count>=2 

Set an alert if above search returns any result.

Update

Try this

sourcetype=teamcity LogFailure name="'tests/cluster/pay/*'" | stats dc(host) as count  by name| where count>=2 

This should get distinct count of host for each values of name i.e. test names and will list of tests which have failed in more than 1 hosts. The alert condition should be same.

jrlesch
New Member

Thanks that helped!

0 Karma

jrlesch
New Member

I guess the problem with that is that it is checking for any test under tests/cluster/pay/ failing on more than 2. I was looking to search for specific tests failing on multiple hosts. But there are 50 tests under pay. Does that make sense?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

See if the updated answer, which counts no of hosts for each specific tests, suits your requirement.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Using the Splunk Threat Research Team’s Latest Security Content

REGISTER HERE Tech Talk | Security Edition Did you know the Splunk Threat Research Team regularly releases ...

SplunkTrust | 2024 SplunkTrust Application Period is Open!

It's that time again, folks! That's right, the application/nomination period for the 2024 SplunkTrust is ...