Splunk AppDynamics

Java agent issue_ Metric Reporter Queue full Dropping metrics.

Pallavi_R
Engager

Hi ,

I have installed java agent for one of my application. Agent is not reporting, in the logs I found this message

" Metric Reporter Queue full. Dropping metrics.".Kindly assit me on this.

Labels (1)
0 Karma

CommunityUser
Splunk Employee
Splunk Employee
Do you have a solution for this issue?
0 Karma

CommunityUser
Splunk Employee
Splunk Employee
All my applications are having this same issue. Is there a article that explains the issue an the resolution.
0 Karma

Peter_Holditch
Builder

As stated above, this usually indicates there is a network problem of some kind that is preventing the agent from connecting to the controller.

You need to inspect the agent logs to diagnose the underlying issue.

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hello Peter,

I have also facing same issue like "Metric Reporter Queue full. Dropping metrics" in server. I'm using 4.5 java agent. Can you please do needful for us? Can you provide solution for same?

Regards,

Chetan

0 Karma

Peter_Holditch
Builder

Chetan,

As I stated before, this is most often due to some connectivity problem between the agent and the controller.

If you look in your agent logs, you should see additional messages that will help lead you to the root cause (firewall issues, indicated by connection timeouts; incorrect addresses, indicated by connection refused, etc.)

Without being able to see the logs, it's impossible to determine what the problem is.

Warm regards,

Peter

CommunityUser
Splunk Employee
Splunk Employee

Hi,

I am also facing the same issue. I am not able to attach the logs here. Can you please let me know the alternate way through which i can send you the logs. However i have attached snippet of agent logs where i see the error. [REMOVED- to protect privacy and security]

Kindly let me know the issue and steps to resolve the same.

I have agent running in the same machine where i have installed the controller, Kindlt let me know if i can use agent and controller in different machines,

Regards,

Chandu

0 Karma

Peter_Holditch
Builder

Chandu,

I saw a smoking gun in the log you posted, before it was redacted.

It seems you have no available licenses in your controller:

"Agent license request denied.  Agent type: Java; Host: Chandu; Reason: No license available"

Warm regards,

Peter

0 Karma

shandi_pusghiya
New Member

hy all,

i have same issue with the agent,

in my case, my application owner have 1 server with 4 jvm (jvm1, jvm3, jvm5, jvm7)

the problem in jvm1, where the jvm1 not connected with the controller, if any isssue with the network connection, should be all jvm cant connect with the controller.

how about my problem? somebody can help me?

Thanks
Shandi aji P

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Depending on your agent logs it could be a connectivity issue but in my case it was a licensing issue that was keeping my agent from sending metrics to the controller.

Even though my licenses appeared to be good, you have to check your default rule in your license manager to make sure you're fully allocated, which in my case I wasn't.  I had recently added a new license key ahead of scaling out and adding close to 100 new agents so we wouldn't hit our limit.  What I didn't realize was I had to adjust the rule to allocate all the new licenses we just added.  Once that was done, the rest of our new agents were able send metrics to the controller.

Peter_Holditch
Builder

It looks as if the agent cannot contact the controller.

Can you zip up and attach all the files under the logs and conf directories beneath the versioned agent directory within you agent installation (which will have a name of the form ver4.x.y.z, where x y and z are the version numbers)

Then we can try and see what the connectivity issue is.

Warm regards,

Peter

0 Karma

CommunityUser
Splunk Employee
Splunk Employee


Hi All,
I am also facing the same issue while installing my java agent. I am using app dynamics 4.5 version and i have license file configured with 2 apm agents and 2 machine agent. I have already configured .Net agent for one application and while trying to configuring java agent for another application controller is not communicating with agent. I checked both the controller machine and agent installed machine with ping command where both are communicating. I can see in agent log file ' Metric Reporter Queue full Dropping metric' message. Not sure which is blocking the controller to communicate with agent. Default rule i set 2 apm agent and 2 machine agents and application scope i have added both applications and respective server name. Please look into the issue and  do the needful. I have shared log information of agent.

^Edited by @Ryan.Paredez 

Log file removed for security reasons. We do not recommend anyone share log files publicly on the forums. 

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi,

Even I am facing the same issue. Kindly find the logs below:

^Edited by @Ryan.Paredez 

Log file removed for security reasons. We do not recommend anyone share log files publicly on the forums. 

0 Karma

Sankar_Alkondan
Engager

Hi all,

I'm not sure if you got a solution for this issue "Metric Reporter Queue full Dropping metrics". If not, please check the Rules in the licenses tab of the controller console. If it's fully allocated, you may need to add some licenses or temporarily disable the rule. After doing so, kindly check if the application is reporting to the controller. It worked for me today. 

Settings → Licenses → Rules in controller console. Make sure the Account usage is not 100% 

CommunityUser
Splunk Employee
Splunk Employee

Hi Peter,

did you had a chance to look into what may have been causing this issue?

Regards

Erwin

0 Karma

Peter_Holditch
Builder

Erwin,

I don't see your agent logs on the thread.  Without the full logs, it's impossible to diagnose.

Could you post them?

Warm regards,

Peter

0 Karma
Get Updates on the Splunk Community!

New This Month - Splunk Observability updates and improvements for faster ...

What’s New? This month, we’re delivering several enhancements across Splunk Observability Cloud for faster and ...

What's New in Splunk Cloud Platform 9.3.2411?

Hey Splunky People! We are excited to share the latest updates in Splunk Cloud Platform 9.3.2411. This release ...

Buttercup Games: Further Dashboarding Techniques (Part 6)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...