Splunk AppDynamics

Machine Agent Leaking File Descriptors?

Elijah_Aydnwyld
New Member

We've been seeing this behavior since a recent reinstall of 4.5.10 where, after a day or two, the Machine Agent stops reporting metrics even thought availability looks ok.  Investigation found errors in the logs complaining about having too many open files when trying to run extensions/scripts.  Looking at open file descriptors for the process, it appears that it is leaking file handles for its own logs.  I'm curious if anyone else has seen similar behavior recently.

From a test machine: 

root@mm03:/proc/23667/fd# ps -ef | grep appd
root 11569 806 0 22:30 pts/0 00:00:00 grep --color=auto appd
root 23667 1 7 Nov18 ? 03:46:54 /opt/appdynamics/machine-agent//jre/bin/java -Dlog4j.configuration=file:/opt/appdynamics/machine-agent/conf/logging/log4j.xml -jar /opt/appdynamics/machine-agent/machineagent.jar
root@mm03:/proc/23667/fd# ls -l /proc/23667/fd | wc -l
4097
root@mm03:/proc/23667/fd# ls -l /proc/23667/fd | grep "machine-agent.log" | wc -l
3922

Labels (1)
0 Karma

Elijah_Aydnwyld
New Member

It looks like this behavior is related to a newly added Network Monitor extension in our deployment, though I don't have insight yet into why it is happening.

0 Karma
Get Updates on the Splunk Community!

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...