Splunk AppDynamics

error in java agent unable to identify issue

CommunityUser
Splunk Employee
Splunk Employee

[AD Thread Pool-Global17] 16 Apr 2018 06:01:35,753 INFO BackendResolutionDelegate - Backend resolution strategy changed from: [ LegacyBackendResolver ] to [ SmallProfileBackendResolver ] [AD Thread Pool-Global20] 16 Apr 2018 06:36:43,001 INFO ErrorProcessor - Sending ADDs to register [ApplicationDiagnosticData{key='java.net.SocketTimeoutException:617419158', name=java.net.SocketTimeoutException:, diagnosticType=STACK_TRACE, configEntities=[Type:ERROR, id:4341660], summary='java.net.SocketTimeoutException:'}, ApplicationDiagnosticData{key='java.lang.NullPointerException:-2072495027', name=java.lang.NullPointerException:, diagnosticType=STACK_TRACE, configEntities=[Type:ERROR, id:4341661], summary='java.lang.NullPointerException:'}, ApplicationDiagnosticData{key='com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:-1115698017', name=com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:, diagnosticType=STACK_TRACE, configEntities=[Type:ERROR, id:4341663], summary='com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:'}] [AD Thread Pool-Global20] 16 Apr 2018 06:36:43,145 INFO ErrorProcessor - Stack Trace Objects registered with controller :{java.net.SocketTimeoutException:617419158=4367675, java.lang.NullPointerException:-2072495027=4367676, com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:-1115698017=4367677} [AD Thread Pool-Global20] 16 Apr 2018 06:36:43,145 INFO ErrorProcessor - Adding entry to errorKeyToUniqueKeyMap [4367675], StackTraceErrorKey{hashCode=617419158}, java.net.SocketTimeoutException:617419158 [AD Thread Pool-Global20] 16 Apr 2018 06:36:43,145 INFO ErrorProcessor - Adding entry to errorKeyToUniqueKeyMap [4367676], StackTraceErrorKey{hashCode=-2072495027}, java.lang.NullPointerException:-2072495027 [AD Thread Pool-Global20] 16 Apr 2018 06:36:43,145 INFO ErrorProcessor - Adding entry to errorKeyToUniqueKeyMap [4367677], StackTraceErrorKey{hashCode=-1115698017}, com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:-1115698017 [AD Thread-Metric Reporter1] 16 Apr 2018 06:47:42,638 INFO XMLConfigManager - Full certificate chain validation performed using default certificate file [AD Thread Pool-Global21] 16 Apr 2018 06:58:03,001 INFO ErrorProcessor - Sending ADDs to register [ApplicationDiagnosticData{key='java.net.SocketTimeoutException:2026412840', name=java.net.SocketTimeoutException:, diagnosticType=STACK_TRACE, configEntities=[Type:ERROR, id:4341660], summary='java.net.SocketTimeoutException:'}, ApplicationDiagnosticData{key='com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:574025845', name=com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:, diagnosticType=STACK_TRACE, configEntities=[Type:ERROR, id:4341663], summary='com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:'}] [AD Thread Pool-Global21] 16 Apr 2018 06:58:03,185 INFO ErrorProcessor - Stack Trace Objects registered with controller :{java.net.SocketTimeoutException:2026412840=4368570, com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:574025845=4368571} [AD Thread Pool-Global21] 16 Apr 2018 06:58:03,185 INFO ErrorProcessor - Adding entry to errorKeyToUniqueKeyMap [4368570], StackTraceErrorKey{hashCode=2026412840}, java.net.SocketTimeoutException:2026412840 [AD Thread Pool-Global21] 16 Apr 2018 06:58:03,185 INFO ErrorProcessor - Adding entry to errorKeyToUniqueKeyMap [4368571], StackTraceErrorKey{hashCode=574025845}, com.mycompany.lodging.apiharness.exceptions.HttpClientException:java.net.SocketTimeoutException:574025845 [AD Thread Pool-Global23] 16 Apr 2018 07:30:53,000 INFO ErrorProcessor - Sending ADDs to register [ApplicationDiagnosticData{key='com.mycompany.database.KeyNotFoundException:-2141366734', name=com.mycompany.database.KeyNotFoundException:, diagnosticType=STACK_TRACE, configEntities=[Type:ERROR, id:4341645], summary='com.mycompany.database.KeyNotFoundException:'}, ApplicationDiagnosticData{key='com.mycompany.database.KeyNotFoundException:157832225', name=com.mycompany.database.KeyNotFoundException:, diagnosticType=STACK_TRACE, configEntities=[Type:ERROR, id:4341645], summary='com.mycompany.database.KeyNotFoundException:'}] [AD Thread Pool-Global23] 16 Apr 2018 07:30:53,143 INFO ErrorProcessor - Stack Trace Objects registered with controller :{com.mycompany.database.KeyNotFoundException:-2141366734=4367528, com.mycompany.database.KeyNotFoundException:157832225=4367529} [AD Thread Pool-Global23] 16 Apr 2018 07:30:53,143 INFO ErrorProcessor - Adding entry to errorKeyToUniqueKeyMap [4367528], StackTraceErrorKey{hashCode=-2141366734}, com.mycompany.database.KeyNotFoundException:-2141366734 [AD Thread Pool-Global23] 16 Apr 2018 07:30:53,143 INFO ErrorProcessor - Adding entry to errorKeyToUniqueKeyMap [4367529], StackTraceErrorKey{hashCode=157832225}, com.mycompany.database.KeyNotFoundException:157832225 [AD Thread Pool-Global27] 16 Apr 2018 08:27:43,000 INFO ErrorProcessor - Sending ADDs to register [ApplicationDiagnosticData{key='java.util.ConcurrentModificationException:53587210', name=java.util.ConcurrentModificationException:, diagnosticType=STACK_TRACE, configEntities=[Type:ERROR, id:4341654], summary='java.util.ConcurrentModificationException:'}] [AD Thread Pool-Global27] 16 Apr 2018 08:27:43,130 INFO ErrorProcessor - Stack Trace Objects registered with controller :{java.util.ConcurrentModificationException:53587210=4367684} [AD Thread Pool-Global27] 16 Apr 2018 08:27:43,130 INFO ErrorProcessor - Adding entry to errorKeyToUniqueKeyMap [4367684], StackTraceErrorKey{hashCode=53587210}, java.util.ConcurrentModificationException:53587210

Labels (1)
Tags (1)
0 Karma

Atyuha_Pal
Contributor

Hi,

Could you please let us know the issue you are facing currently and also please share the zip file of the complete log to debug it further.

Thanks,

Atyuha

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

did some get chance to look logs and sugggest

0 Karma

Peter_Holditch
Builder

The agent is working fine (although I do note it is a v4.2 agent (which is now end of life) reporting to a 4.4.1 controller, so you should plan an upgrade)

The problem looks to be that you have business transaction overflow on your controller

[AD Thread Pool-Global0] 16 Apr 2018 01:13:23,417  INFO BusinessTransactionRegistry - Registered Transactions list in BT registration response is empty, controller may have reached maximum Business Transaction Limit. You may exclude Business Transactions in BT Dashboard to allow new BTs to register

The documentation talks through how you should manage Business Transactions.  

Warm regards,
Peter

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

did you get chance to look at my reply ?

i can not limit transaction as those are not being registered in controller so how i can exclude those without knowing what all are there (choose which to select which not). ?

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hi Sir,

Could you share if it is the only error  how you identify and causing issue ? Second how i can enable all business transaction(any limit i can increase or what ?) to register instead of excluding as at the moment i do not know which all is important and what to exclude it is after controller register all i can filter and exclude ? because i checked in documentation its written way to exclude is after you get transaction register in controller. so stuck how to exclude . 

as per doc - You can choose to exclude or delete a business transaction from the right click menu for the transaction on the business transaction list. 

0 Karma

CommunityUser
Splunk Employee
Splunk Employee
waiting on this ?
0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Attached log file . There are errors in processor i can see and others.

My issue is tiers are reporting to controller but no data is coming . Its just empty tier names with all nodes and 100% agent reported to controller but empty data and flow maps.

0 Karma
Get Updates on the Splunk Community!

New This Month - Splunk Observability updates and improvements for faster ...

What’s New? This month, we’re delivering several enhancements across Splunk Observability Cloud for faster and ...

What's New in Splunk Cloud Platform 9.3.2411?

Hey Splunky People! We are excited to share the latest updates in Splunk Cloud Platform 9.3.2411. This release ...

Buttercup Games: Further Dashboarding Techniques (Part 6)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...