URL Monitoring Extension Crashing intermittently

[Avinash_Isukal1]

Hello All,

In Windows Server, the URL Monitoring Extension of v2.2.0 on Machine agent of v21 is crashing intermittently. The extension is failed to report the metrics on to the Controller during the crash time. But the Machine agent is sending all the Infra metrics to the controller. 

I tried with heap increment for xmX & smX values and tried with metric registration limit to maximum level but these options are not resolving the issue.

However, once I restarted the Machine agent service, the URL monitoring extension could start reporting it's metrics. This process is being repeated for 5 to 6 times per day.

Can someone please help me.

Thanks in advance!

Avinash

[iamryan]

Hi @Kamal.Manchanda,

Thank you so much for coming back and sharing the info with the community. 

[Kamal_Manchand1]

I am facing the same error with Log Extension:

Error MonitorExecutorServiceModule-Log Monitor - Queue Capacity reached!! Rejecting runnable tasks.

Error StreamReaderRunnable - Error reading stream

Error: Error response from Controller: Connection back off in effect.

Any advise to find the root cause. Post restart queue data comes until queue fills back again.

Max Heap Memory: 1 GB, Max Metric Limit: 7000. Number of threads: 80

Any Advise to resolve these errors?

[Kamal_Manchand1]

This issue was resolved by increasing the MetaSpace value to 256MB instead of default value 64MB.

[Dietrich_Meier]

Hi

Thats a lot of restarts, there must be something to change that would at the very least reduce that number.

Here are some of our stats in case it can help you tweak things a bit.
I try to make the Xmx value slightly more than the consumption we see.
(Mine is probably higher than yours might be because of the other extensions on this MA.

We monitor around 130 URLs with the extension.

Metrics uploaded by the Extension

Thread setting in extension config file
numberOfThreads: 12

Regards

Dietrich

[Avinash_Isukal1]

Hi Dietrich Meier,

Thanks for posting the response.

We're doing daily restart of Machine agent on the off peak hours to resume the URL monitoring extension metrics. But, It's stopping 4 times per day to report the URL extension metrics.

Thanks,

Avinash

[Dietrich_Meier]

Hi Avinash

We experience a similar issue at times (once or twice a week) with the v2.1 version of the extension.
The v22.1 Machine Agent runs on Windows with 2 other extensions.
The URL monitoring metrics would stop reporting in while the SIM and other extension's metrics continue to report in as normal. MA metrics and memory allocation also updated over time, but it is difficult to tell if that worked as a temporary fix of if a simple MA restart does the trick.

One thing I picked out from the logs the previous time it happened:
03RNB-APPDDBM==> [Agent-Monitor-Scheduler-4] 07 Nov 2022 06:31:04,956 ERROR MonitorExecutorServiceModule-URL Monitor - Queue Capacity reached!! Rejecting runnable tasks..

Screenshot from one of the dashboards used to monitor the MA that hosts extensions.
(Shows when the URL monitoring extension breaks)
URL Metrics stop reporting in

At the moment I do maintenance restarts at least once a week, and not sure about the exact cause.

Regards

Dietrich

[Ranjith_Kumarka]

@Avinash.Isukaleti 

Can you attach here the log file to check the issue .

Thanks

Ranjit kumar

[Claudia_Landiva]

Hello @Avinash.Isukaleti 

Have you been able to find a solution to this issue? As I understand it, since access to your configuration files and debug logs would be needed, opening a Support ticket is recommended. 

In case it's useful, here's a reference: How do I submit a Support ticket, an FAQ 

Claudia Landivar, Community Manager & Editor