topic Re: Get alerted if same error happening on multiple hosts in Alerting

Get alerted if same error happening on multiple hosts

jrlesch — Tue, 30 Sep 2014 13:51:24 GMT

I have the same suite of tests running on multiple hosts. I want to set up an alert if a specific test fails on 2 or more hosts.

sourcetype=teamcity LogFailure name="'tests/cluster/pay/*'"

is my search to get all the tests the failed. The "name" is the name of the test.

How do I tell if the test is throwing the same LogFailure on multiple hosts.

Thanks

Re: Get alerted if same error happening on multiple hosts

somesoni2 — Tue, 30 Sep 2014 13:57:10 GMT

Try this

sourcetype=teamcity LogFailure name="'tests/cluster/pay/*'" | stats dc(host) as count | where count>=2

Set an alert if above search returns any result.

Update

Try this

sourcetype=teamcity LogFailure name="'tests/cluster/pay/*'" | stats dc(host) as count  by name| where count>=2

This should get distinct count of host for each values of name i.e. test names and will list of tests which have failed in more than 1 hosts. The alert condition should be same.

Re: Get alerted if same error happening on multiple hosts

jrlesch — Tue, 30 Sep 2014 14:02:41 GMT

I guess the problem with that is that it is checking for any test under tests/cluster/pay/ failing on more than 2. I was looking to search for specific tests failing on multiple hosts. But there are 50 tests under pay. Does that make sense?

Re: Get alerted if same error happening on multiple hosts

somesoni2 — Tue, 30 Sep 2014 14:38:26 GMT

See if the updated answer, which counts no of hosts for each specific tests, suits your requirement.

Re: Get alerted if same error happening on multiple hosts

jrlesch — Tue, 30 Sep 2014 14:44:47 GMT

Thanks that helped!