AppD Archive

Node Health does not work after restart of node

CommunityUser
Splunk Employee
Splunk Employee

Hi,

I have a problem with "Node Health - Transaction". The health rule affects a tier (so all node in this tier).

If a node restarts the health rule has no results for this node! (even if I wait for more then one day, still no results for this node).

When I open the health rule and save it (without change anything) the health rule works for all nodes again.

This is a serious problem, because in a cloud environment nodes start,stop or restart all the time. If the health rules does not work for this nodes then they are useless (because the results of the health rules are wrong).

Regards,

Thomas

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

In addition:

If the health rule is on a tier and one node has "no data available", the whole health rule does not work anymore. So if a violation occurs in one node the health rule does not care at all (even if health rule is violate is on "Any Node").

0 Karma

Arun_Dasetty
Super Champion

Hi Thomas,

We do not expect such behavior , Can you help us provide the following details for clarity of the issue:

a) screenshots of health rule config in edit view for all sections 

b) screenshot from metric browser with metric added to graph for which condition added on health rule

c) screenshots from violations section and screenshot depicting the issue in reference to your comments

d) controller ui version and server.log if you are referring to onPremise controller UIi

Regards,

Arun

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi Arun,

Thomas and I are still focusing the problem! We did a popper documentation about the problem but don't want that's gonna be public. Is there a possibility to send it by private message?

Regard
Christian

0 Karma

Arun_Dasetty
Super Champion

Hi Christian,

 

You can send me a private message using this forum using link "send this user a private message" when you are at target user page Or you can send me details requested to email akumar@appdynamics.com that is fine with you, if you are concerned on sharing details here

 

Regards,

Arun

0 Karma

Arun_Dasetty
Super Champion

Hi Christian,

 

Thanks for the logs in email. We do not see any errors related health rule (com.appdynamics.RULES.PROCESSING) evaluation in server log provided, though we see events related errors in logs as listed below:

 

[#|2015-06-12T11:08:14.017+0200|WARNING|glassfish3.1.2|com.appdynamics.EVENTS.PROCESSING|_ThreadID=166;_ThreadName=Thread-5;|cancelOpenIncident() called with no matching AffectedEntityPolicy in PolicyCache for healthRuleId: 273 and affectedEntity: Type:BUSINESS_TRANSACTION, id:1557|#]

 

- Can you provide screenshot from TroubleShoot -> health rule violations with no filters selected in target screen for past 2 days data period selected? and filter by healt rule using search box in UI at right corner, we do see no violations in health rules screen but wonder if that is UI cache update issue of status and hence would like to check in violations screen once.

 

Regards,

Arun

0 Karma

Arun_Dasetty
Super Champion

Hi Christian,

 

We could see health rule violations for rule "health_mbbc_soko" in violations in reference to latest email with logs provided by you. We see this could be UI cache issue in updating violation status in health rule screen as we could see violations fine under Troubleshoot -> health rule violations in controller UI, upgrading contorller will not fix, please check the behavior post re-logn after clearing browser and flash cachec once in health rules screen. Hope that information helps.

 

above observations are in reference to screenshot  "1 Health Rule Violations plus Filter - AppDynamics.png"

 

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...