All TKB Articles in Learn Splunk

Learn Splunk

All TKB Articles in Learn Splunk

To collect a thread dump for a Controller App Server process, use the Glassfish "asadmin" tool. In the computer terminal, use the command generate-jvm-report to generate a threadump.txt file. ... See more...
To collect a thread dump for a Controller App Server process, use the Glassfish "asadmin" tool. In the computer terminal, use the command generate-jvm-report to generate a threadump.txt file. <controller_home>/appserver/glassfish/bin/asadmin -u admin --passwordfile=<controller_home>/.passwordfile  generate-jvm-report --type=thread > threaddump.txt Create a series of thread dumps at regular intervals. The following example would create 20 thread dumps in 60-second intervals: mkdir /tmp/threaddumps cd /tmp/threaddumps for i in {1..20}; do /opt/AppDynamics/Controller/appserver/glassfish/bin/asadmin -u admin --passwordfile=/opt/AppDynamics/Controller/.passwordfile generate-jvm-report --type=thread > threaddump$i.txt; sleep 60; done Additionally, the generate-jvm-report command can generate other types of reports including memory or summary. For more information, view Glassfish documentation.
Symptoms Server.log contains the following error message, while EUM metrics continue to be collected successfully: com.singularity.ee.controller.beans.eumcloud.MetricsCountLimit|_ThreadID=2909... See more...
Symptoms Server.log contains the following error message, while EUM metrics continue to be collected successfully: com.singularity.ee.controller.beans.eumcloud.MetricsCountLimit|_ThreadID=2909754;_ThreadName=Thread-5;|Metric name "EUM|Browser|Chrome|Chrome 56|First Byte Time 80th percentile (ms)" matches but the limit of 100000 OTHER_WEB metrics has been reached for app 57 [count=118121]|#] Diagnosis This issue occurs when the property  appdynamics.controller.eum.otherWeb.metricsCount.limit  maximum has been reached. Note: The com.singularity.ee.controller.beans.eumcloud.MetricsCountLimit is a shared limit for the total of distinct EUM metric names. Solution Change the "metric count limit" property to a higher value. 1. Login to the Controller:  http://<controllerurl>/controller/admin.jsp; 2. Navigate to Controller Flags Servlet: http ://<controllerurl>/controller/private/ControllerFlagsServlet 3. Edit the property  appdynamics.controller.eum.otherWeb.metricsCount.limit value and click submit to save.  The Controller will pick up the change. Note: Before increasing the metrics count limit, try to manage all pages, AJAX, iFrames, and mobile request names using Custom Naming/Exclude rules. Be aware that deleting any pages, mobile requests, etc, will also delete the associated metrics (and metric names). Additionally, increasing the metric count limit will increase disk utilization, which could lead to performance issues. The user assumes all risks.
Symptoms Users who are configuring a snapshot repository to take backups of the Events Service cluster, as outlined in the documentation (Back Up Events Service Data), within a Windows environmen... See more...
Symptoms Users who are configuring a snapshot repository to take backups of the Events Service cluster, as outlined in the documentation (Back Up Events Service Data), within a Windows environment may encounter the following error message: Command to configure snapshot repository: D:\events-service>bin\events-service.exe snapshot-configure-fs -p conf\events-service-api-store.properties Output of the above command: [2017-05-10T16:36:39,389+04:00] Error occurred [Error executing request to event service. Status code: [500]. Exception message: ["Unknown server error: [[\\localhost\\appdynamics-events-service-backup] location [\\\\localhost\\appdynamics-events-service-backup\\appdynamics-events-service-cluster] doesn't match any of the locations specified by path.repo]"]. For more information, view the event service logs.] Examples for property ad.es.backupmanager.path.repo in the events-service-api-store.properties file that all yield the same error: \\localhost\appdynamics-events-service-backup \\\\localhost\\appdynamics-events-service-backup {BACKUP_HOME} as an environment variable Diagnosis The following prerequisites need to be completed before configuring an Events Service snapshot repository within a Windows environment: Map the directory as a writable network drive. Create a network share with the user running the snapshot commands and give the administrator read/write access. Verify that the network share is accessible from ALL the nodes of the Events Service cluster.  The value of the  ad.es.backupmanager.path.repo property must be the same between all nodes. Solution Update the ad.es.backupmanager.path.repo=\\\\localhost\\appdynamics-events-Service-backup  on all nodes and restart them all before attempting the snapshot command. Example:
Question Why do I get a "401 unauthorized" error when running Platform Admin Application commands in version 4.3? Example: bin/platform-admin.sh show-platforms Command failed due to an error... See more...
Question Why do I get a "401 unauthorized" error when running Platform Admin Application commands in version 4.3? Example: bin/platform-admin.sh show-platforms Command failed due to an error: Unauthorized API code 401 Session expired. Please login and run the command again.   Answer From version 4.3, users will need to first login to the Platform Admin Application before issuing the desired commands. The default username is admin and the password is the one give to you at the time of installation. Example of setting the Platform Admin database password:   How to login: bin/platform-admin.sh login --user-name=admin   Forgot admin password? How to reset: bin/platform-admin.sh reset-password --help usage: platform-admin.sh reset-password [-h] [-u USER-NAME] optional arguments: -h, --help show this help message and exit -u USER-NAME, --user-name USER-NAME The username whose password to reset. If not supplied, 'admin' is assumed. And this will reset the password for the user to the default which is admin​. Note: After resetting the password, the default username and password will be admin/admin. Immediately change the password to something more secure. How to change password: bin/platform-admin.sh change-password --help usage: platform-admin.sh change-password [-h] --user-name USER-NAME [--password PASSWORD] [--new-password NEW-PASSWORD] Once the user is successfully logged in, they can enter the desired command.
Question How can a user capture mobile crash stack traces automatically using an API?  Users may be gathering crash data from the mobile_crash_reports table using ADQL, but there is no stack trace ... See more...
Question How can a user capture mobile crash stack traces automatically using an API?  Users may be gathering crash data from the mobile_crash_reports table using ADQL, but there is no stack trace information in this table. Stack trace data is stored within the AppDynamics system, and displayed in a web browser. Example:  Answer As of AppDynamics version 20.x, there is no API available to gather crash data from the mobile_crash_reports analytics schema. As a workaround, set the following variables and use curl to request the full crash report. Note: The URLs may change in different controller versions because these are not public APIs. The following example is for AppDynamics Controllers version >=20.x. The sample cURL command below retrieves a single stack trace for the specified crash ID. Customers are advised to develop their own automation solutions tailored to their platform to handle bulk crash report processing. To obtain the stack trace, follow these steps: 1. Generate an Access Token for the API call to your controller. Refer to the following documentation for guidance: https://help.splunk.com/en/appdynamics-saas/extend-splunk-appdynamics/25.7.0/extend-splunk-appdynamics/splunk-appdynamics-apis/api-clients#generate-the-token-through-the-administration-ui-0. 2. After generating the Access Token, execute the command below to retrieve a crash stack trace. # Set these variables: APPLICATION= # You can find this in the url when viewing an app. It will say "application=X" where X is an integer CRASH_ID= # This is the crashid field from mobile_crash_reports analytics schema (not groupid) # Get a stacktrace for a particular crashid from the controller: curl --location --request GET \ "http://${CONTROLLER_URL}/controller/restui/crashDetails/download/${APPLICATION}/${CRASH_ID}" Sample request: curl --location --request GET 'https:/eumanalytics.saas.appdynamics.com/controller/restui/crashDetails/download/6789/64D17A66-6A84-49E0-B8DE-0731E42244CF' --header 'Authorization: Bearer eyJraWQiOiJhOWUyMTM2NC1lYTc3LTQ3OWUtOTdmZi1lYmJkMjc2NjNmYmUiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJBcHBEeW5hbWljcyIsImF1ZCI6IkFwcERfQVBJcyIsImp0aSI6IjlybDVyU0tWZDZwN3hKM3A5NWRfemciLCJzdWIiOiJwb3N0bWFuIiwiaWRUeXBlIjoiQVBJX0NMSUVOVCIsImlkIjoiNjgzZmZmODMtNTY5YS00OGRkLWE4MjMtYWE0ODA4NGYzN2UxIiwiYWNjdElkIjoiYTllMjEzNjQtZWE3Ny00NzllLTk3ZmYtZWJiZDI3NjYzZmJlIiwidG50SWQiOiJhOWUyMTM2NC1lYTc3LTQ3OWUtOTdmZi1lYmJkMjc2NjNmYmUiLCJhY2N0TmFtZSI6ImFwcGR5bmFtaWNzIiwidGVuYW50TmFtZSI6ImFwcGR5bmFtaWNzIiwiZm1tVG50SWQiOm51bGwsImFjY3RQZXJtIjpbXSwicm9sZUlkcyI6W10sImlhdCI6MTY1OTQ4Njk1NiwibmJmIjoxNjU5NDg2ODM2LCJleHAiOjE2NTk1NzMzNTYsInRva2VuVHlwZSI6IkFDQ0VTUyJ9.sKG2hkFofWK9avqr7RlzVEM_APiCUiXRXmA7qm12Wr4' Sample Output  
Disable HTTP listener Users who do not wish to delete http-listener-1 because they may want to use it later can disable the HTTP listener. Note: Starting Controller v4.4+, the changes need to b... See more...
Disable HTTP listener Users who do not wish to delete http-listener-1 because they may want to use it later can disable the HTTP listener. Note: Starting Controller v4.4+, the changes need to be done using the Enterprise Console. Editing Controller Port Configurations Option 1: Shut down the Controller App Server using the following command:  <controller_home>/bin/controller.sh stop-appserver Take a backup of the <controller_home>/appserver/glassfish/domains/domain1/config/domain.xml file. Edit the domain.xml file and add address="127.0.0.1" so that the access to non-secure port is restricted to the controller server only. Change from: <network-listener port="8090" protocol="http-listener-1" transport="tcp" name="http-listener-1" thread-pool="http-thread-pool"></network-listener> To: <network-listener port="8090" protocol="http-listener-1" transport="tcp" name="http-listener-1" thread-pool="http-thread-pool"></network-listener> Option 2:  Another option is to disable the listener by setting enabled="false". Change from: <network-listener port="8090" protocol="http-listener-1" transport="tcp" name="http-listener-1" thread-pool="http-thread-pool"></network-listener> To: <network-listener port="8090" enabled="false" protocol="http-listener-1" transport="tcp" name="http-listener-1" thread-pool="http-thread-pool"></network-listener>\ Start the App Server: <controller_home>/bin/controller.sh start-appserver Note: The port number 8090 is used for representation purposes. Actual port for http-listener-1 might differ if a different port is selected during installation. Relevant Links: Oracle network-listener
Question Can I query a range of IP addresses when using ADQL to find browser records or session data? Answer It is not possible to query a range of IP addresses; however, users can add wildcar... See more...
Question Can I query a range of IP addresses when using ADQL to find browser records or session data? Answer It is not possible to query a range of IP addresses; however, users can add wildcard search to their query to achieve similar results. Example: Using the '|' (pipe) operator will not work: 'x.x.x.x'|'x.x.x.y'|'x.x.x.z' Instead, use the wildcard search: 'x.x.x.*' The above wildcard search will return all IP addresses beginning with x.x.x.  
What are the steps for disabling TLS versions 1.0 and 1.1 on the on-premises Controller? In this article... Disable TLS 1.0 and 1.1  |  Test the Controller after disabling TLS 1.0 and 1.1 ... See more...
What are the steps for disabling TLS versions 1.0 and 1.1 on the on-premises Controller? In this article... Disable TLS 1.0 and 1.1  |  Test the Controller after disabling TLS 1.0 and 1.1 Disable TLS 1.0 and 1.1  Stop appserver <controller_home>/bin/controller.sh stop-appserver Take a backup of <controller_home>/appserver/glassfish/domains/domain1/config/domain.xml Locate the "ssl" element in domain.xml <ssl ssl3-tls ciphers="...." ssl3-enabled="false" classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl" cert-nickname="s1as" tls-enabled="false"></ssl> To disable TLS 1.0 Set the attribute tls-enabled="false" NOTE | This attribute may already exist and be disabled by default (set to false). If set to "true" change it to "false" to disable TLS 1.0) To disable TLS 1.1 Add an attribute tls11-enabled="false" after the tls-enabled="false" attribute, with the two attributes separated by a space To enable TLS 1.2 explicitly TLS v1.2 is enabled by default after setting the two properties above. If you want to still indicate it in domain.xml , add the attribute tls12-enabled="true" separated from the previous attribute by a space After making the above changes, the SSL element looks like: <ssl ssl3-tls ciphers="...." ssl3-enabled="false" classname="com.sun.enterprise.security.ssl.GlassfishSSLImpl" cert-nickname="s1as" tls-enabled="false" tls11-enabled="false" tls12-enabled="true"></ssl> ​ STEP 4 NOTES  Do not change any other attributes other than tls-enabled, tls11-enabled and tls12-enabled The ciphers="...." attribute is indicated with ellipses(...) because there are many ciphers and we didn't include them above since they were not to be changed. The actual domain.xml contains the list of ciphers and they are not to be changed. Start the appserver <controller_home>/bin/controller.sh start-appserver STEP 5 NOTE  | Allow a few minutes for the appserver to start. Test the Controller after disabling TLS 1.0 and 1.1 To test whether the Controller now accepts TLS 1.0, TLS 1.1 and TLS 1.2 protocol you could use OpenSSL to test the connectivity and for TLS 1 (which is TLS 1.0) and TLS 1_1 (which is TLS 1.1) the connectivity fails with "handshake failure" openssl s_client -connect <controller_host>:<controller_ssl_port> -tls1 openssl s_client -connect <controller_host>:<controller_ssl_port> -tls1_1 openssl s_client -connect <controller_host>:<controller_ssl_port> -tls1_2​ STEP 6 NOTES  The the <controller_ssl_port> is the port tied to "http-listener-2" in <controller_home>/appserver/glassfish/domains/domain1/config/domain.xml <network-listener port="<PORT>" protocol="http-listener-2" transport="tcp" name="http-listener-2" thread-pool="http-thread-pool"></network-listener> <controller_host> is the hostname of the server on which the controller is installed
Symptoms The following error message can be found in the SSL-enabled debug logs for the EUM processor, while the processor is running and listening on port 7002:   javax.net.ssl.SSLHandshakeE... See more...
Symptoms The following error message can be found in the SSL-enabled debug logs for the EUM processor, while the processor is running and listening on port 7002:   javax.net.ssl.SSLHandshakeException: no cipher suites in common   The following error message appears on the  <eum-server-host>:<eum-server-https-port>/eumaggregator/get-version  page.   Diagnosis This root cause of this issue is a wrong entry type field value in the custom keystore for the EUM processor alias. Example: Alias name: 'eum-processor' Creation date: 14 Mar, 2017 Entry type: trustedCertEntry Solution Regenerate the keystore and copy into  <eum installation directory>\ eum-processor\bin .   Make sure that the entry type value is not trustedCertEntry for the alias 'eum-processor' and it should be as "Entry type: PrivateKeyEntry". Restart the EUM Server. In your computer terminal, from the EUM processor directory use the following commands: bin/eum.sh stop bin/eum.sh start Verify the new SSL server cert works by opening the following page in a browser: https://<hostname>:7002/eumcollector/get-version Confirm a successful response. Related Links: Secure the EUM Server
Symptoms EUM Server fails to start due to SSL Handshake error when the EUM account synchronizes to the analytics account. Example: 20 May 2017 16:24:50.174 +1000 main AD.AL... See more...
Symptoms EUM Server fails to start due to SSL Handshake error when the EUM account synchronizes to the analytics account. Example: 20 May 2017 16:24:50.174 +1000 main AD.ALL INFO ------start synchronize eum acct to analytics acct!--------- 20 May 2017 16:24:50.177 +1000 main AD.AnalyticsAccountManage INFO start updating MobileSessionRecord 20 May 2017 16:24:50.388 +1000 main AD.AnalyticsAccountManage ERROR | failed bulk updating MobileSessionRecord | javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target | at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) | at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949) | at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302) | at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) | at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509) | at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216) | at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) | at sun.security.ssl.Handshaker.process_record(Handshaker.java:914) | at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062) | at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) | at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) | at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) | at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394) | at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353) | at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134) | at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) | at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380) | at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) | at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184) | at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88) | at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) | at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184) | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) | at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) | at com.appdynamics.eumcloud.analytics.AnalyticsAccountManager.bulkUpdateAnEventType(AnalyticsAccountManager.java:266) | at com.appdynamics.eumcloud.analytics.AnalyticsAccountManager.bulkUpdateEventTypes(AnalyticsAccountManager.java:229) | at com.appdynamics.eum.processor.EUMProcessorServerApplication.syncEumAccountsToAnalyticsAccounts(EUMProcessorServerApplication.java:233) | at com.appdynamics.eum.processor.EUMProcessorServerApplication.run(EUMProcessorServerApplication.java:173) | at com.appdynamics.eum.processor.EUMProcessorServerApplication.run(EUMProcessorServerApplication.java:114) | at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:42) | at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:76) | at io.dropwizard.cli.Cli.run(Cli.java:70) | at io.dropwizard.Application.run(Application.java:72) | at com.appdynamics.eumcloud.EUMProcessorServer.main(EUMProcessorServer.java:40) | Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target | at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387) | at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292) | at sun.security.validator.Validator.validate(Validator.java:260) | at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324) | at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229) | at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124) | at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491) | ... 29 more | Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target | at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) | at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) | at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) | at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382) | ... 35 more | +--------------------------------------------------------------------------------------- Diagnosis When the EUM Server tries to connect to the Analytics or Events Service, either directly or via a proxy server over SSL, the EUM Server does not trust the incoming certificate. This is due to a missing intermediate or root certificate in the trust keystore of the EUM Server.  Solution Import the intermediate/root certificates into the trust keystore of the EUM Server.  Example: <EUM_HOME>/jre/lib/security/cacerts   How to import the root & intermediate certificate to the "cacerts" trust keystore ../jre/bin/keytool -import -trustcacerts -alias myorg-rootca -keystore cacerts -file /path/to/CA-cert.txt -storepass changeit ../jre/bin/keytool -import -trustcacerts -alias myorg-interca -keystore cacerts -file /path/to/CA-inter.txt -storepass changeit Relevant Links: Secure the EUM Server
Problem You want to capture a heap dump of a Java process on Windows. You launch a command prompt with admin privileges and use "jmap," which is a command-line tool that prints memory-related stat... See more...
Problem You want to capture a heap dump of a Java process on Windows. You launch a command prompt with admin privileges and use "jmap," which is a command-line tool that prints memory-related statistics, but the command returns either an "Access Denied" error message or an "Insufficient Memory" or "Insufficient Privilege to attach" error message. Examples:   Solution If Java processes are running in the background, Windows does not allow jmap or jconsole tools to see JVM started as a Windows service. Instead of using jmap to capture a heap dump for a Java process on Windows, use the Microsoft tool PsExec to generate a heap dump. Example: 
When I can’t sign in to the AppDynamics Support portal, what are the steps to resetting my password? If you're having trouble signing in to AppDynamics Support portal and need to reset your pass... See more...
When I can’t sign in to the AppDynamics Support portal, what are the steps to resetting my password? If you're having trouble signing in to AppDynamics Support portal and need to reset your password: Navigate to the Sign-in page at https://login.appdynamics.com/sso/authenticate Click the Reset Password link below the Next button.    AppDynamics sign in page, with Reset Password link   You should receive an email with a link allowing you to reset your password. When you reset your password, it  should be at least 8 characters long, including a lowercase letter, an uppercase letter, and a symbol. AppDynamics password reset dialog If you don’t receive this email, check your email client's spam folder.  
When migrating an EUM Server from Windows to Linux, a user might incorrectly assume that the Controller is also migrated, which takes care of the MySQL backup. The Controller database and the EUM P... See more...
When migrating an EUM Server from Windows to Linux, a user might incorrectly assume that the Controller is also migrated, which takes care of the MySQL backup. The Controller database and the EUM Processor store must be migrated separately.   Users should always do a fresh install of the EUM server on the new machine, and then check the host dependent settings.   Example: In the eum.properties file, modify the following Controller host IP/name properties:   "onprem.dbHost" : This is the controller-db host detail, port 3388 remains unchanged. "analytics.serverHost"* Change this property if you are using the local Events Service that comes embedded inside the Controller, not a standalone installation of the Events Service.   Lastly, users should copy the data from the  ../store directory from the old machine to the new one. Dependencies on the Events Service and Controller will remain intact.   Step-by-step instructions:   Backup the EUM  ..\store  directory on your Windows machine. Backup the Controller MySQL database on Windows, and restore the db backup on the Linux machine. Install EUM Server on the Linux machine.  Configure all the components to the Linux machine after installation. Change the Event Service and EUM hostnames in the Controller config. Change the IP address in the Controller config.  Restore the directory ..\store backup on the Linux machine.    Related Links: Users on 4.3.x (or earlier version) migrate EUM database on Controller host to 4.4.x EUM host Migrate the Controller EUM Server installation Published 6/23/2017
Users can get a report of license usage by applications for each Controller. On-premises instructions SaaS instructions    Note: Disabled agents will not be returned by the query but ... See more...
Users can get a report of license usage by applications for each Controller. On-premises instructions SaaS instructions    Note: Disabled agents will not be returned by the query but will still consume licenses. However, this query will be in the ballpark of actual consumption. If you want to track historical usage by application, you need to restrict one application to one license rule under Controller Settings (gear icon ) > License > Rules -> Application Scope. On-premises instructions On-premises customers can extract application usage from their database by using the following steps: Login to the controller MySQL database located in <controller_home>/bin/controller.sh login-db . Execute the following query to get a list of all types of agents that have been reporting for the last five minutes. SELECT a.name AccountName, agj.type AgentType,agj.agent_version AgentVersion, agj.app_and_tier ApplicationAndTier,agj.node_name NodeName FROM (SELECT DISTINCT(node_id) AS node_id FROM metricdata_min WHERE ts_min > ((SELECT max(ts_min) FROM metricdata_min) - 5) ) mdm, (SELECT ag.type type, ag.agent_version agent_version, acn.name node_name,acn.last_known_tier_app_config AS app_and_tier, acnam.application_component_node_id node_id, ag.account_id account_id FROM agent ag, application_component_node_agent_mapping acnam,application_component_node acn WHERE acnam.agent_id = ag.id AND acn.id=acnam.application_component_node_id UNION DISTINCT SELECT ag.type type, ag.agent_version agent_version,acn.name node_name,acn.last_known_tier_app_config AS app_and_tier, acn.id, ag.account_id FROM agent ag, application_component_node acn, machine_instance_agent_mapping miam, machine_instance mi WHERE miam.machine_instance_id = mi.id AND miam.agent_id = ag.id AND acn.machine_instance_id = mi.id) agj, account a WHERE mdm.node_id = agj.node_id AND a.id = agj.account_id ORDER BY AccountName,AgentType,ApplicationAndTier,NodeName Change the  (SELECT max(ts_min) FROM metricdata_min) - 5) clause to determine the time range. SaaS instructions  For SaaS customers, application usage is not available currently. Account-level usage information can be found by clicking on the "gear" menu in the top right side of the controller UI, then clicking on License in the drop-down menu. Below is the account usage. REST API for both SAAS and On-Premises customers: Create your own scripts using agent availability metric data API for last 5 min at node level.  - 1 = license consumed - 0 = no license consumed Reference the API document Note: It is not currently possible to check the historical license consumption. Related Links: License Management
Symptoms EUM Server will fail to start if the sync between EUM and Events Service or Analytics Server fails during the startup process. Users should check the EUM Server and Events Service log... See more...
Symptoms EUM Server will fail to start if the sync between EUM and Events Service or Analytics Server fails during the startup process. Users should check the EUM Server and Events Service logs for the following error messages: EUM Server log snippet ------start synchronize eum acct to analytics acct!--------- 24 Mar 2017 11:25:50.112 +0100 main AD.AnalyticsAccountManage INFO ************** Starting Registration of Analytics Account for EUM account="xyz" ************** 24 Mar 2017 11:25:50.476 +0100 main AD.AnalyticsAccountManage ERROR | CreateAccount : account="xyz", Error Code [500], Exception [RestException(statusCode=500, code=Unexpected.Exception, HttpResponseProxy{HTTP/1.1 500 Internal Server Error [Date: Fri, 24 Mar 2017 10:25:50 GMT, X-Content-Security-Policy: default-src 'self', X-XSS-Protection: 0, X-Content-Type-Options: nosniff, Frame-Options: DENY, X-Frame-Options: DENY, Content-Type: application/json, Transfer-Encoding: chunked] ResponseEntityProxy{[Content-Type: application/json,Chunked: true]}} 24 Mar 2017 11:25:50.551 +0100 main AD.AnalyticsAccountManage ERROR | failed bulk updating BrowserRecord | java.lang.RuntimeException: bulk update event type=BrowserRecord failed!!! url=http:/xyz:9080/v2/events/BrowserRecord/_bulk, key=... | {"statusCode":500,"code":"Unexpected.Exception", Events Service logs: [2017-03-24T11:12:59,937+01:00] [INFO ] [dw-83 - POST /v1/account] [c.a.a.p.a.resource.AccountResource] Account Service Request: [POST v1/account] - request [[{"accountName":"customer1""accessKey":"REDACTED","expirationDate":"2017-05-20T23:59Z","eumAccountName":"XYZ","licensingConfigurations":[]}]] [2017-03-24T11:12:59,941+01:00] [ERROR] [dw-83 - POST /v1/account] [c.a.a.p.a.ElasticsearchAccountManager] Error occurred while upserting account configuration for accounts [customer1] com.appdynamics.analytics.processor.exception.UnavailableException: Failed to find accounts [customer1] Diagnosis Review the output of the following health check commands to find if the shards are in an unassigned state. curl http://<events_service_machine>:9081/healthcheck?pretty=true curl http://<events_service_machine>:9200/_cat/shards?v curl http://<events_service_machine>:9200/_cat/indices?v curl -XGET localhost:9200/_cat/shards?h=index,shard,prirep,state,unassigned.reason| grep UNASSIGNED Output from the above health check commands: "events-service-api-store / elasticsearch-singlenode-module" : { "healthy" : false, "message" : "Current [appdynamics-events-service-cluster] cluster state: [RED], data nodes: [1], nodes: [1], active shards: [3], relocating shards: [0], initializing shards: [0], unassigned shards: [12], timed out: [false]" } } curl -XGET localhost:9200/_cat/shards?h=index,shard,prirep,state,unassigned.reason| grep UNASSIGNED appdynamics_meters_v2 0 p UNASSIGNED CLUSTER_RECOVERED appdynamics_meters_v2 1 p UNASSIGNED CLUSTER_RECOVERED appdynamics_api_keys_v1 0 p UNASSIGNED CLUSTER_RECOVERED appdynamics_api_keys_v1 1 p UNASSIGNED CLUSTER_RECOVERED appdynamics_accounts 0 p UNASSIGNED CLUSTER_RECOVERED appdynamics_accounts 1 p UNASSIGNED CLUSTER_RECOVERED event_type_metadata 0 p UNASSIGNED CLUSTER_RECOVERED event_type_metadata 1 p UNASSIGNED CLUSTER_RECOVERED event_type_extracted_fields 0 p UNASSIGNED CLUSTER_RECOVERED event_type_extracted_fields 1 p UNASSIGNED CLUSTER_RECOVERED event_type_metadata_hidden_fields_v1 0 p UNASSIGNED CLUSTER_RECOVERED event_type_metadata_hidden_fields_v1 1 p UNASSIGNED CLUSTER_RECOVERED    Check the events-service-api-store.log file (found in <Events Service home directory>/logs) for an error message that includes:  Failed to execute phase [query], all shards failed.   [2017-03-24T10:33:00,445+01:00] [ERROR] [dw-83 - POST /v1/account] [c.a.a.p.a.ElasticsearchAccountManager] Error occurred while upserting account configuration for accounts [] com.appdynamics.analytics.processor.exception.UnavailableException: Failed to find accounts ] at com.appdynamics.analytics.processor.elasticsearch.exception.ElasticSearchExceptionUtils.propagateAsEventException(ElasticSearchExceptionUtils.java:147) ~[analytics-processor.jar:na] at com.appdynamics.analytics.processor.elasticsearch.exception.ElasticSearchExceptionUtils.propagateAsEventException(ElasticSearchExceptionUtils.java:117) ~[analytics-processor.jar:na] at com.appdynamics.analytics.processor.account.ElasticsearchAccountManager.propagateAsException(ElasticsearchAccountManager.java:324) ~[analytics-processor.jar:na] at Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to execute phase [query], all shards failed at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:237) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.start(TransportSearchTypeAction.java:160) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:62) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction.doExecute(TransportSearchQueryThenFetchAction.java:52) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:100) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:43) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:75) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.client.node.NodeClient.execute(NodeClient.java:98) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:338) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.search.SearchRequestBuilder.doExecute(SearchRequestBuilder.java:1112) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:91) ~[elasticsearch-1.7.3.jar:na] at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:65) ~[elasticsearch-1.7.3.jar:na] at com.appdynamics.analytics.processor.account.ElasticsearchAccountManager.getAllQueryMatches(ElasticsearchAccountManager.java:252) ~[analytics-processor.jar:na] at com.appdynamics.analytics.processor.account.ElasticsearchAccountManager.findAccountConfigurationsInElasticsearch(ElasticsearchAccountManager.java:222) ~[analytics-processor.jar:na] ... 54 common frames omitted Solution 1. Manually allocate the unassigned shards using the reroute API. In the following example, there are 12 unassigned shards so each shard must be separately allocated. curl -XPOST 'localhost:9200/_cluster/reroute' -d '{ "commands" : [ { "allocate" : { "index" : "appdynamics_meters_v2", "shard" : 1, "node" : "node-12345798-7867-4ea1-b546-557a6bc546afg", "allow_primary" : true } } ]}' 2. Verify if the cluster health state is now a green state using the following curl commands.  curl http://<events_service_machine>:9081/healthcheck?pretty=true curl http://<events_service_machine>:9200/_cat/shards?v curl http://<events_service_machine>:9200/_cat/indices?v  3. Start the EUM Server to sync with the Events Service or Analytics Server.
Symptoms After updating a license from Trial to Pro and configuring the EUM Server to work with the Events Service in gathering data analytics, some on-premise users may see an error on the Web A... See more...
Symptoms After updating a license from Trial to Pro and configuring the EUM Server to work with the Events Service in gathering data analytics, some on-premise users may see an error on the Web Analyze page that says "The supplied auth information is for an account that expired on [{YYYY-MM-DDTHH:MM:SS.sssT}]."  Diagnosis The controller server.log file will show a stack trace similar to the following: [#|2017-05-05T19:06:19.518+0530|SEVERE|glassfish3.1.2|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=664176;_ThreadName=Thread-5;|RestException(statusCode=401, code=Auth.Expired, message=The supplied auth information is for an account that expired on [{2017-03-10T08:00:00.000Z}], developerMessage=) at com.appdynamics.analytics.shared.rest.exceptions.RestExceptionFactory.makeException(RestExceptionFactory.java:41) at com.appdynamics.analytics.shared.rest.client.utils.RestClientUtils.validateResponse(RestClientUtils.java:110) at com.appdynamics.analytics.shared.rest.client.utils.RestClientUtils.resolve(RestClientUtils.java:68) at com.appdynamics.analytics.shared.rest.client.utils.GenericHttpRequestBuilder.executeAndReturnRawResponseString(GenericHttpRequestBuilder.java:245) at com.appdynamics.analytics.shared.rest.client.eventservice.DefaultEventServiceClient.getEventType(DefaultEventServiceClient.java:162) at sun.reflect.GeneratedMethodAccessor5132.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.singularity.ee.controller.beans.analytics.client.AccountCreatingAnalyticsClient$ProxyingEventServiceClient.invoke(AccountCreatingAnalyticsClient.java:98) at com.sun.proxy.$Proxy702.getEventType(Unknown Source) at com.appdynamics.analytics.shared.rest.client.DefaultAnalyticsClient.getEventType(DefaultAnalyticsClient.java:37) at com.singularity.ee.controller.beans.analytics.client.DefaultAnalyticsDataTransferClient.getEventTypeAsJsonNode(DefaultAnalyticsDataTransferClient.java:873) at com.singularity.ee.controller.beans.analytics.client.DefaultAnalyticsDataTransferClient.getBrowserRecordFieldDefinitions(DefaultAnalyticsDataTransferClient.java:638) at com.singularity.ee.controller.ui.services.analytics.AnalyticsMetaDataUiService.getBrowserFieldDefinitions(AnalyticsMetaDataUiService.java:291) at sun.reflect.GeneratedMethodAccessor5130.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:708) at javax.servlet.http.HttpServlet.service(HttpServlet.java:770) at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1550) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:343) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:217) at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:89) at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119) at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133) at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130) at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203) at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:217) at com.singularity.ee.controller.ui.filter.RestSessionFilter.doFilter(RestSessionFilter.java:107) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:217) at com.singularity.ee.controller.servlet.CsrfFilter.doFilter(CsrfFilter.java:84) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:217) at com.singularity.ee.controller.servlet.RequestOriginMarkingFilter.doFilter(RequestOriginMarkingFilter.java:26) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:217) at com.singularity.ee.controller.servlet.CacheControlFilter.doFilter(CacheControlFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:217) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:279) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175) at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:161) at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:331) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231) at com.sun.enterprise.v3.services.impl.ContainerMapper$AdapterCallable.call(ContainerMapper.java:317) at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195) at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:860) at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:757) at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1056) at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:229) at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90) at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79) at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54) at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59) at com.sun.grizzly.ContextTask.run(ContextTask.java:121) at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:554) at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:533) at java.lang.Thread.run(Unknown Source) |#] Solution The events-service-api-store.properties file under events-service/conf directory should have the following properties: in 4.3.x ad.accountmanager.key.controller = <the value that you see admin.jsp > controller setting for the 'appdyamics.on.premise.event.service.key' property> ad.accountmanager.key.eum= <the value that you see admin.jsp > controller setting for the 'appdynamics.es.eum.key' property> in 4.2.x ad.accountmanager.key.controller = <the value that you see admin.jsp > controller setting for the  'appdynamics.analytics.server.store.controller.key' property> ad.accountmanager.key.eum= <the value that you see admin.jsp > controller setting for the 'appdynamics.es.eum.key' property> Note: Restart the events service after making the above changes. The controller does not require a restart. Additionally, double check that the  admin.jsp > controller setting properties  has the following property:  eum.es.host = <the Events Service endpoint URL>  
Question When restarting the Events Service or Analytic Agent, do I need to delete leftover process ID (PID) from the previous session, such as events-service-api-store.pid or analytics-agent.id, m... See more...
Question When restarting the Events Service or Analytic Agent, do I need to delete leftover process ID (PID) from the previous session, such as events-service-api-store.pid or analytics-agent.id, manually by the user? Answer In previous versions, users who needed to restart their Events Service or Analytic Agent were required to manually delete the ID for any stalled or leftover processes in case of abrupt/forced shutdown of the process.   Starting in version 4.3, when a PID does not get deleted due to a forced shutdown or restart, users will no longer have to manually delete it.   AppDynamics version 4.3 adds a -f flag to the start command which removes any leftover PID due to an abrupt or forced shutdown.   Linux example:   To start events-service using -f flag, use the below command bin/events-service.sh start -f -p conf/events-service-api-store.properties To start analytic-agent using -f flag, use the below command ./analytics-agent.sh start -f For more commands, use the help information found in  ./events-service.sh --help and  ./analytics-agent.sh --help.
Users who are unable to start version 4.3 of the Events Service should check the logs for the following error: Incompatible Java version detected while running the program. A newer version of Java... See more...
Users who are unable to start version 4.3 of the Events Service should check the logs for the following error: Incompatible Java version detected while running the program. A newer version of Java may be required java.lang.UnsupportedClassVersionError: com/appdynamics/analytics/processor/executor/AnalyticsServiceExecutor : Unsupported major.minor version 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.security.SecureClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.access$100(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Unknown Source) at com.appdynamics.common.executor.standalone.ProxyMain.callActualMain(ProxyMain.java:158) at com.appdynamics.common.executor.standalone.ProxyMain.main(ProxyMain.java:106) The Events Service for version 4.3 requires Java 1.8. This is outlined in our documentation under the Events Service "General Requirements" section.  To set up 1.8 JRE manually using the environment variables in your computer terminal:  export JAVA_HOME=<path to 1.8 JRE> export PATH=$JAVA_HOME/bin:$PATH Verify that the above command worked:  java -version Start the Events Service:  nohup bin/events-service.sh start -p conf/events-service-api-store.properties
Symptoms If you are unable to start the Events Service when using AppDynamics version 4.3, first review the nohup.out log file for the following error message:   [main] INFO c.a.a.processor.An... See more...
Symptoms If you are unable to start the Events Service when using AppDynamics version 4.3, first review the nohup.out log file for the following error message:   [main] INFO c.a.a.processor.AnalyticsService - Starting analytics processor with arguments [-p, /export/appdynamics/appdynamics/events-service/conf/events-service-api-store.properties, -y, /export/appdynamics/appdynamics/events-service/bin/../conf/events-service-api-store.yml] /tmp/tmp-7078993570062239070-events-service-api-store.yml has an error: * Incorrect type of value at: logging.appenders.[1].throttleEnabled; is of type: String, expected: boolean Diagnosis If you manually upgraded the Events Service from version 4.2 to version 4.3 using the steps outlined in the documentation, note that step 5 of the upgrade instructions says to copy the configuration settings from the  events-service-api -store.properties file, not the file itself. Step 5 of the Manual Upgrade process: Migrate configuration changes from the properties files in the backup Events Service directory to the conf\events-store-api-store.properties file in the new Events Service directory. Depending on which type of deployment you are using, this involves inspecting and migrating settings from: events-service-all.properties, or events-service-api-store.properties Reference: https://docs.appdynamics.com/display/PRO43/Upgrade+the+Events+Service#UpgradetheEventsService-UpgradetheEventsServiceManually Solution To resolve this issue, do not copy the 4.2  events-service-api-store.properties  file when upgrading to version 4.3. Instead, copy only the required configuration property values from the 4.2 file, such as  ad.es.path.home , ad.dw.log.path , and ad.es.backupmanager.path.repo, and update these values in the 4.3 property file located in <Event_service_home_directory>/conf. Make sure to save your changes and restart. 
As of version 4.2.x, the EUM Server is not available for High Availability out of the box. High Availability Controller deployment is available to help minimize disruptions caused by server or n... See more...
As of version 4.2.x, the EUM Server is not available for High Availability out of the box. High Availability Controller deployment is available to help minimize disruptions caused by server or network failure, administrative downtime, and other interruptions. Those interested can view this overview of High Availability. If you are interested in having your EUM Server run in High Availability, please contact your account representative and our professional services engineers to discuss the available options.