AppD Archive

Health Rule Custom Metric

CommunityUser
Splunk Employee
Splunk Employee

Hi,

I added a health rule on a custom metric. After that the historical data of this metric was not available anymore.

Is this behaviour normal?

Regards,

Thomas

0 Karma

Arun_Dasetty
Super Champion

Hi Thomas,

We confirm we do not expect such behavior as defining Health rule should not be affecting old data, can you send the screenshots from UI depicting the issue and relevant controller server.log, database.log for clarity on this to assist you further?

Regards,

Arun

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi Arun,

I made some screenshots, but for security reasons I have to obfuscate some parts.

here the metrics are available:

image.jpeg

Now I configure a health rule:

image.jpeg

After that, the metric browser looks like this:

image.jpeg

And after a while, the metric starts "fresh". But all old data seems lost.

image.jpeg

I also added a part of the server.log from the corresponding timeslot.

From my point it looks like only custom metrics are involved.

Regards,

Thomas

0 Karma

Arun_Dasetty
Super Champion

Hi Thomas,

We do not locate any specific errors as we are not aware of the metric id:

[#|2015-04-30T09:22:15.090+0200|SEVERE|glassfish3.1.2|com.singularity.ee.controller.beans.agent.db.MetricMemoryDataStore|_ThreadID=147;_ThreadName=Thread-5;|Metric id 46922 for Type:APPLICATION_COMPONENT, id:217 not found in cache, but it was expected|#]
[#|2015-04-30T09:14:48.754+0200|WARNING|glassfish3.1.2|com.appdynamics.AGENT.METRICS_UPLOAD|_ThreadID=139;_ThreadName=Thread-5;|machine ID: 135 has no nodes associated with it, can not process metrics for it!|#]

Can you provide complete archive of server.log for logs covering period before defining HR and after issue reproduced and also provide the output of below command issued at controller myql prompt:

mysql> select * from metric where name like %Monitor|<shaded metirc name>|availability%" \G;

Regards,

Arun

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Here the result from the sql statement:

mysql> select * from metric where name like '%Monitor|<shaded metirc name>|availability%'\G
*************************** 1. row ***************************
id: 46922
version: 0
name: Custom Metrics|URL Monitor|<shaded metirc name>|Availability
name_sha1: 833f30a384423235263c53479dba2454b102388b
created_on: 1429708479936
application_id: 10
is_virtual: 0
agent_type: MACHINE_AGENT
time_rollup_type: CURRENT
cluster_rollup_type: INDIVIDUAL
hole_handling_type: REGULAR_COUNTER
account_wide_metric_id: NULL

I hope this helps.

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...