Splunk AppDynamics

Problems starting app server agent on IBM WebSphere Application Server

Hans_Mueller
New Member

I have already successfully appdynamics app server agent on seven application servers of my WebSphere network deployment environment. Unfortunately, the setup on the last server I could not successfully finish my setup. If I set the following parameter on my JVM:

javaagent:/opt/IBM/AppAgents/AppServerAgent/javaagent.jar -Dappdynamics.agent.tierName=<placeholde> -Dappdynamics.agent.nodeName=<placeholder>

I see the following error on the log of the app server agent:

[AD Agent init] 09 Feb 2022 16:11:21,251 INFO ConfigurationChannel - Container id retrieval enabled: true
[AD Agent init] 09 Feb 2022 16:11:21,252 WARN ConfigurationChannel - Unable to use /proc/self/cgroup for unique hostname, could not locate container ID
[AD Agent init] 09 Feb 2022 16:11:21,252 INFO ConfigurationChannel - Agent node meta-info thus far: ProcessID;111140;appdynamics.ip.addresses;21.0.11.44,21.1.11.44;appdynamicsHostName;xxxx
[AD Agent init] 09 Feb 2022 16:11:21,252 INFO ConfigurationChannel - Detected node meta info: [Name:ProcessID, Value:111140, Name:appdynamics.ip.addresses, Value:xxxx,xxxx, Name:appdynamicsHostName, Value:xxxx, Name:supportsDevMode, Value:true]
[AD Agent init] 09 Feb 2022 16:11:21,252 INFO ConfigurationChannel - Sending Registration request with: Application Name [xxxx], Tier Name [xxxx], Node Name [xxxx], Host Name [xxxx] Node Unique Local ID [xxxx], Version [Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
[AD Agent init] 09 Feb 2022 16:11:21,309 WARN AgentErrorProcessor - Agent error occurred, [name,transformId]=[com.singularity.XMLConfigManager - javax.xml.parsers.FactoryConfigurationError,2147483647]
[AD Agent init] 09 Feb 2022 16:11:21,309 WARN AgentErrorProcessor - 4 instance(s) remaining before error log is silenced
[AD Agent init] 09 Feb 2022 16:11:21,309 WARN XMLConfigManager - Error refreshing agent configuration
javax.xml.parsers.FactoryConfigurationError: Provider javax.xml.parsers.DocumentBuilderFactory could not be instantiated: java.util.ServiceConfigurationError: javax.xml.parsers.DocumentBuilderFactory: Provider org.apache.xerces.jaxp.DocumentBuilderFactoryImpl not a subtype
at javax.xml.parsers.DocumentBuilderFactory.newInstance(Unknown Source) ~[?:?]
at java.util.prefs.XmlSupport.loadPrefsDoc(XmlSupport.java:252) ~[?:1.8.0]
at java.util.prefs.XmlSupport.importMap(XmlSupport.java:388) ~[?:1.8.0]
at java.util.prefs.FileSystemPreferences$6.run(FileSystemPreferences.java:598) ~[?:1.8.0]
at java.util.prefs.FileSystemPreferences$6.run(FileSystemPreferences.java:591) ~[?:1.8.0]
at java.security.AccessController.doPrivileged(AccessController.java:734) ~[?:1.8.0]
at java.util.prefs.FileSystemPreferences.loadCache(FileSystemPreferences.java:590) ~[?:1.8.0]
at java.util.prefs.FileSystemPreferences.initCacheIfNecessary(FileSystemPreferences.java:573) ~[?:1.8.0]
at java.util.prefs.FileSystemPreferences.getSpi(FileSystemPreferences.java:550) ~[?:1.8.0]
at java.util.prefs.AbstractPreferences.get(AbstractPreferences.java:298) ~[?:1.8.0]
at com.ibm.crypto.pkcs11impl.provider.IBMPKCS11Impl.<init>(IBMPKCS11Impl.java:464) ~[ibmpkcs11impl.jar:8.0 build_8.0-20200224-2]
at java.lang.J9VMInternals.newInstanceImpl(Native Method) ~[?:2.9 (06-01-2020)]
at java.lang.Class.newInstance(Class.java:1852) ~[?:2.9 (06-01-2020)]
at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:233) ~[?:1.8.0]
at sun.security.jca.ProviderConfig$2.run(ProviderConfig.java:218) ~[?:1.8.0]
at java.security.AccessController.doPrivileged(AccessController.java:678) ~[?:1.8.0]
at sun.security.jca.ProviderConfig.doLoadProvider(ProviderConfig.java:218) ~[?:1.8.0]
at sun.security.jca.ProviderConfig.getProvider(ProviderConfig.java:199) ~[?:1.8.0]
at sun.security.jca.ProviderList.getProvider(ProviderList.java:245) ~[?:1.8.0]
at sun.security.jca.ProviderList.getIndex(ProviderList.java:275) ~[?:1.8.0]
at sun.security.jca.ProviderList.getProviderConfig(ProviderList.java:259) ~[?:1.8.0]
at sun.security.jca.ProviderList.getProvider(ProviderList.java:265) ~[?:1.8.0]
at java.security.Security.getProvider(Security.java:479) ~[?:1.8.0]
at com.ibm.jsse2.ac.<clinit>(ac.java:159) ~[?:8.0 build_20200327--103]
at com.ibm.jsse2.ag.<init>(ag.java:18) ~[?:8.0 build_20200327--103]
at com.ibm.jsse2.ag.<init>(ag.java:111) ~[?:8.0 build_20200327--103]
at com.ibm.jsse2.av.a(av.java:87) ~[?:8.0 build_20200327--103]
at com.ibm.jsse2.av.<init>(av.java:636) ~[?:8.0 build_20200327--103]
at com.ibm.jsse2.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:2) ~[?:8.0 build_20200327--103]
at com.singularity.ee.util.httpclient.EasySSLProtocolSocketFactory.createLayeredSocket(EasySSLProtocolSocketFactory.java:135) ~[appagent.jar:?]
at com.singularity.ee.util.httpclient.EasySSLProtocolSocketFactory.connectSocket(EasySSLProtocolSocketFactory.java:193) ~[appagent.jar:?]
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.13.jar:4.5.13]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72) ~[httpclient-4.5.13.jar:4.5.13]
at com.singularity.ee.util.httpclient.SimpleHttpClientWrapper.executeHttpOperation(SimpleHttpClientWrapper.java:302) ~[appagent.jar:?]
at com.singularity.ee.util.httpclient.SimpleHttpClientWrapper.executeHttpOperation(SimpleHttpClientWrapper.java:217) ~[appagent.jar:?]
at com.singularity.ee.rest.RESTRequest.sendRequestTracked(RESTRequest.java:384) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.rest.RESTRequest.sendRequest(RESTRequest.java:337) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.rest.controller.request.AControllerRequest.sendRequest(AControllerRequest.java:130) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.rest.controller.request.ABinaryControllerRequest.sendRequest(ABinaryControllerRequest.java:36) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.config.xml.ConfigurationChannel.registerApplicationServer(ConfigurationChannel.java:1436) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.config.xml.ConfigurationChannel.access$100(ConfigurationChannel.java:121) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.config.xml.ConfigurationChannel$UnregisteredConfigurationState.nextTransition(ConfigurationChannel.java:784) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.config.xml.ConfigurationChannel.refreshConfiguration(ConfigurationChannel.java:554) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.config.xml.XMLConfigManager$AgentConfigurationRefreshTask.run(XMLConfigManager.java:656) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.config.xml.XMLConfigManager.initialize(XMLConfigManager.java:332) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.AgentKernel.start(AgentKernel.java:166) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.JavaAgent.initialize(JavaAgent.java:451) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at com.singularity.ee.agent.appagent.kernel.JavaAgent.initialize(JavaAgent.java:346) ~[appagent.jar:Server Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90) ~[?:1.8.0]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) ~[?:1.8.0]
at java.lang.reflect.Method.invoke(Method.java:508) ~[?:1.8.0]
at com.singularity.ee.agent.appagent.AgentEntryPoint$1.run(AgentEntryPoint.java:655) ~[?:Server IBM Agent #21.11.3.33314 v21.11.3 GA compatible with 4.4.1.0 r1f94344f9fd88fc14fe39a33494b03e4bb555a6d release/21.11.0]

Due to this error the agent could not register sucessfully at the controller. Does anyone have an idea how to solve this problem. Is this problem maybe related to the security provider com.ibm.crypto.pkcs11impl.provider.IBMPKCS11Impl which is listed in the file java.security of the jdk on this server? On the other servers which I could sucessfully register at the controller this security provider is not listed in the java.security file. But maybe the cause for the problem is not related to this security provider. 

Labels (1)
0 Karma

iamryan
Community Manager
Community Manager

Hi @Hans.Mueller,

I found this existing AppD Docs page. Please take a look and let me know if it helps out at all.

https://docs.appdynamics.com/22.2/en/application-monitoring/install-app-server-agents/java-agent/ins...

0 Karma

arredondolopezh
New Member

I am getting same error and this is happening when migrating onprem to SaaS. please provide next steps

0 Karma

iamryan
Community Manager
Community Manager
0 Karma
Get Updates on the Splunk Community!

Developer Spotlight with Brett Adams

In our third Spotlight feature, we're excited to shine a light on Brett—a Splunk consultant, innovative ...

Index This | What can you do to make 55,555 equal 500?

April 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Say goodbye to manually analyzing phishing and malware threats with Splunk Attack ...

In today’s evolving threat landscape, we understand you’re constantly bombarded with phishing and malware ...