About Abdulrahman_Ka1

Abdulrahman_Ka1 · ‎06-27-2021

Hello @Ryan.Paredez I made a couple of configuration and at the end each time I do the installation the ports in each machine changes to 0.0.0.0 For now, the issue was in DNS resolve as I had to make it manual in each machine at /etc/hosts file to give an example eventsservice-01 10.0.30.xx eventsservice-02 10.0.30.xx eventsservice-03 10.0.30.xx the installation done successfully after this configuration and yet still checking the health as it's not good. *another update * after the last configuration, the events service were not able to keep running so, I changed every hostname to IP in the conf/events-service-api-store.properties in each machine. I stopped/disabled the firewall and iptables, and now it's up and running in green health since two hours! kindly I would like to thank you for your support. Thanks Abdulrahman

Abdulrahman_Ka1 · ‎06-08-2021

hi @Gaurav.Soni I'm trying to install the events service from the console but i get Spoiler (Highlight to read) [root@console platform-admin]# bin/platform-admin.sh install-events-service --profile prod --hosts eventsservice eventsservice-02 eventsservice-03 --data-dir /opt/appdynamics/eventsservice --platform-name AppDPlatform Installing Events Service on new nodes. ( 1/ 26) Clean up orphaned Events Service: SUCCESS ( 2/ 26) Register Events Service Lifecycle Listener before installing: SUCCESS ( 3/ 26) Validate and set parameters while installing ES: SUCCESS ( 4/ 26) Setup cluster configuration: SUCCESS ( 5/ 26) Initialize Events service cluster: SUCCESS ( 14/ 26) Set JRE Versions: SUCCESS ( 15/ 26) Check Events Service hostnames: SUCCESS ( 16/ 26) Verify hosts: Check total free memory ( 16/ 26) Verify hosts: Check total number of processors ( 16/ 26) Verify hosts: Verify if the required directories exists ( 16/ 26) Verify hosts: Get permissions of given directories ( 16/ 26) Verify hosts: Check if the given directories are writable ( 16/ 26) Verify hosts: Get stats of install/data directories ( 16/ 26) Verify hosts: Check if the destination installation directory has sufficient space ( 16/ 26) Verify hosts: Check if the destination data directory has sufficient space ( 16/ 26) Verify hosts: Stop the Events Service api store node ( 16/ 26) Verify hosts: Check if ports are available ( 16/ 26) Verify hosts: Fail if ports unavailable ( 16/ 26) Verify hosts: SUCCESS ( 17/ 26) Save Events Service Cluster Configuration: SUCCESS ( 18/ 26) Add new nodes to Events Service cluster: SUCCESS ( 19/ 26) Install Events Service: Getting status of Events Service artifact ( 19/ 26) Install Events Service: Verifying the presence of Events Service artifacts ( 19/ 26) Install Events Service: Copying Events Service binaries to target host ( 19/ 26) Install Events Service: Extracting Events Service binaries on remote host ( 19/ 26) Install Events Service: Moving extracted folder into Events Service processor ( 19/ 26) Install Events Service: Adding executable permissions to the Events Service startup script for NIX system ( 19/ 26) Install Events Service: Adding executable permissions to the Events Service module scripts for NIX system ( 19/ 26) Install Events Service: Removing temporary directories ( 19/ 26) Install Events Service: Creating Events Service unarchived template directory ( 19/ 26) Install Events Service: Unarchive Events Service properties template ( 19/ 26) Install Events Service: Updating Events Service configuration ( 19/ 26) Install Events Service: Removing Events Service unarchived template directory ( 19/ 26) Install Events Service: Removing Events Service template in nodes ( 19/ 26) Install Events Service: Create JRE 8 link on windows ( 19/ 26) Install Events Service: Install service if on windows ( 19/ 26) Install Events Service: Starting the Events Service api store node ( 19/ 26) Install Events Service: SUCCESS ( 20/ 26) Load Events Service cluster configuration: SUCCESS ( 21/ 26) Load all ES cluster hosts into execution context.: SUCCESS ( 22/ 26) es_cluster_health_stage: FAILED Events Service installation failed. Failure occurred: es_cluster_health_stage Error message: Unable to check health of Events Service hosts [eventsservice-03, eventsservice-02, eventsservice] through port 9080. [root@console platform-admin]# bin/platform-admin.sh install-events-service --profile prod --hosts eventsservice eventsservice-02 eventsservice-03 --data-dir /opt/appdynamics/eventsservice --platform-name AppDPlatformInstalling Events Service on new nodes.( 1/ 26) Clean up orphaned Events Service: SUCCESS( 2/ 26) Register Events Service Lifecycle Listener before installing: SUCCESS( 3/ 26) Validate and set parameters while installing ES: SUCCESS( 4/ 26) Setup cluster configuration: SUCCESS( 5/ 26) Initialize Events service cluster: SUCCESS( 14/ 26) Set JRE Versions: SUCCESS( 15/ 26) Check Events Service hostnames: SUCCESS( 16/ 26) Verify hosts: Check total free memory( 16/ 26) Verify hosts: Check total number of processors( 16/ 26) Verify hosts: Verify if the required directories exists( 16/ 26) Verify hosts: Get permissions of given directories( 16/ 26) Verify hosts: Check if the given directories are writable( 16/ 26) Verify hosts: Get stats of install/data directories( 16/ 26) Verify hosts: Check if the destination installation directory has sufficient space( 16/ 26) Verify hosts: Check if the destination data directory has sufficient space( 16/ 26) Verify hosts: Stop the Events Service api store node( 16/ 26) Verify hosts: Check if ports are available( 16/ 26) Verify hosts: Fail if ports unavailable( 16/ 26) Verify hosts: SUCCESS( 17/ 26) Save Events Service Cluster Configuration: SUCCESS( 18/ 26) Add new nodes to Events Service cluster: SUCCESS( 19/ 26) Install Events Service: Getting status of Events Service artifact( 19/ 26) Install Events Service: Verifying the presence of Events Service artifacts( 19/ 26) Install Events Service: Copying Events Service binaries to target host( 19/ 26) Install Events Service: Extracting Events Service binaries on remote host( 19/ 26) Install Events Service: Moving extracted folder into Events Service processor( 19/ 26) Install Events Service: Adding executable permissions to the Events Service startup script for NIX system( 19/ 26) Install Events Service: Adding executable permissions to the Events Service module scripts for NIX system( 19/ 26) Install Events Service: Removing temporary directories( 19/ 26) Install Events Service: Creating Events Service unarchived template directory( 19/ 26) Install Events Service: Unarchive Events Service properties template( 19/ 26) Install Events Service: Updating Events Service configuration( 19/ 26) Install Events Service: Removing Events Service unarchived template directory( 19/ 26) Install Events Service: Removing Events Service template in nodes( 19/ 26) Install Events Service: Create JRE 8 link on windows( 19/ 26) Install Events Service: Install service if on windows( 19/ 26) Install Events Service: Starting the Events Service api store node( 19/ 26) Install Events Service: SUCCESS( 20/ 26) Load Events Service cluster configuration: SUCCESS( 21/ 26) Load all ES cluster hosts into execution context.: SUCCESS( 22/ 26) es_cluster_health_stage: FAILEDEvents Service installation failed.Failure occurred: es_cluster_health_stageError message:Unable to check health of Events Service hosts [eventsservice-03, eventsservice-02, eventsservice] through port 9080.

Abdulrahman_Ka1 · ‎06-03-2021

Hi @Anonymous I did the above and still having the same issue in the installation, the error is port 9080 is not pingable even I tried different ways to open make it listening but still the same issue, * I tried with opened and stopped firewall here is my thread https://community.appdynamics.com/t5/Business-iQ-Analytics/Starting-Events-Service-cluster/m-p/44162/emcs_t/S2h8ZW1haWx8bWVudGlvbl9zdWJzY3JpcHRpb258S1BINkUwQjdMOFQwVTJ8NDQxNjJ8QVRfTUVOVElPTlN8aEs#M358

Abdulrahman_Ka1 · ‎06-02-2021

Hi @Ryan.Paredez I made some edit like ssh by hostname, not only with the IP address maximize each node to 25 G ram reinstalled the events service cluster by bin/platform-admin.sh install-events-service --profile prod --hosts 10.0.30.45 10.0.30.46 10.0.30.47 --data-dir /opt/appdynamics/eventsservice --platform-name AppDPlatform also in each machine iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 9080 -j ACCEPT iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 9081 -j ACCEPT iptables -A INPUT -m state --state NEW -m multiport -p tcp --dports 9300:9400 -j ACCEPT however, the firewall was off and tried again to restarted the cluster by Spoiler (Highlight to read) [root@console platform-admin]# bin/platform-admin.sh submit-job --platform-name AppDPlatform --service events-service --job restart-cluster ( 1/ 6) Load Events Service cluster configuration: SUCCESS ( 2/ 6) Load all ES cluster hosts into execution context.: SUCCESS ( 3/ 6) Disable transient allocations on masters: Disable transient allocation in the api store node ( 3/ 6) Disable transient allocations on masters: SUCCESS ( 4/ 6) Restart all nodes: Stop the Events Service api store node ( 4/ 6) Restart all nodes: Starting the Events Service api store node ( 4/ 6) Restart all nodes: Stop the Events Service api store node ( 4/ 6) Restart all nodes: Starting the Events Service api store node ( 4/ 6) Restart all nodes: Stop the Events Service api store node ( 4/ 6) Restart all nodes: Starting the Events Service api store node ( 4/ 6) Restart all nodes: SUCCESS ( 5/ 6) Enable transient allocations on masters: Enable transient allocation in the api store node ( 5/ 6) Enable transient allocations on masters: FAILED Job failed to execute. Failure occurred: Enable transient allocations on masters Error message: Task failed: Enable transient allocation in the api store node on host: 10.0.30.45 as user: root with message: Connection to [http://10.0.30.45:9080/v1/elasticsearch/_cluster/settings] failed due to [Failed to connect to /10.0.30.45:9080]. [root@console platform-admin]# bin/platform-admin.sh submit-job --platform-name AppDPlatform --service events-service --job restart-cluster( 1/ 6) Load Events Service cluster configuration: SUCCESS( 2/ 6) Load all ES cluster hosts into execution context.: SUCCESS( 3/ 6) Disable transient allocations on masters: Disable transient allocation in the api store node( 3/ 6) Disable transient allocations on masters: SUCCESS( 4/ 6) Restart all nodes: Stop the Events Service api store node( 4/ 6) Restart all nodes: Starting the Events Service api store node( 4/ 6) Restart all nodes: Stop the Events Service api store node( 4/ 6) Restart all nodes: Starting the Events Service api store node( 4/ 6) Restart all nodes: Stop the Events Service api store node( 4/ 6) Restart all nodes: Starting the Events Service api store node( 4/ 6) Restart all nodes: SUCCESS( 5/ 6) Enable transient allocations on masters: Enable transient allocation in the api store node( 5/ 6) Enable transient allocations on masters: FAILEDJob failed to execute.Failure occurred: Enable transient allocations on mastersError message:Task failed: Enable transient allocation in the api store nodeon host: 10.0.30.45as user: rootwith message: Connection to [http://10.0.30.45:9080/v1/elasticsearch/_cluster/settings] failed due to [Failed to connect to /10.0.30.45:9080].

Abdulrahman_Ka1 · ‎06-01-2021

I'm trying to start the Events service nodes but it doesn't work Spoiler (Highlight to read) [root@console platform-admin]# bin/platform-admin.sh submit-job --platform-name AppDPlatform --service events-service -- job start ( 1/ 7) Load Events Service cluster configuration: SUCCESS ( 2/ 7) Load all ES cluster hosts into execution context.: SUCCESS ( 3/ 7) Start Events Service Cluster: Starting the Events Service api store node ( 3/ 7) Start Events Service Cluster: FAILED Job failed to execute. Failure occurred: Start Events Service Cluster Error message: Task failed: Starting the Events Service api store node on host: 10.0.30.45 as user: root with message: Connection to [http://10.0.30.45:9080/_ping] failed due to [Failed to connect to /10.0.30.45:9080]. [root@console platform-admin]# bin/platform-admin.sh submit-job --platform-name AppDPlatform --service events-service --job start( 1/ 7) Load Events Service cluster configuration: SUCCESS( 2/ 7) Load all ES cluster hosts into execution context.: SUCCESS ( 3/ 7) Start Events Service Cluster: Starting the Events Service api store node( 3/ 7) Start Events Service Cluster: FAILEDJob failed to execute.Failure occurred: Start Events Service ClusterError message:Task failed: Starting the Events Service api store nodeon host: 10.0.30.45as user: rootwith message: Connection to [http://10.0.30.45:9080/_ping] failed due to [Failed to connect to /10.0.30.45:9080]. I also tried to start each node alone I get this error Spoiler (Highlight to read) pwd /opt/appdynamics/platform/product/events-service/processor [root@events-service-03 processor]# echo $JAVA_HOME /usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/ [root@events-service-03 processor]# ./bin/events-service.sh start -p ./conf/events-service-api-store.properties & [1] 43742 [root@events-service-03 processor]# ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging. [2021-06-01T10:52:52,627+02:00] [APPLICATION_HOME] has been overridden to [/opt/appdynamics/platform/product/events-service/processor/bin/..] [2021-06-01T10:52:52,631+02:00] [APPLICATION_HOME] has been overridden to [/opt/appdynamics/platform/product/events-service/processor/bin/..] [2021-06-01T10:52:52,636+02:00] Program arguments: |-Xmx2990m|-Xms2990m|-Djava.net.preferIPv4Stack=true|-Dfile.encoding=UTF-8|-Djute.maxbuffer=30000000|-Dzookeeper.snapshot.trust.empty=true|-XX:+UseParNewGC|-XX:+UseConcMarkSweepGC|-XX:CMSInitiatingOccupancyFraction=75|-XX:+UseCMSInitiatingOccupancyOnly|-XX:+DisableExplicitGC|-XX:+PrintGCDetails|-XX:+PrintGCDateStamps|-XX:+PrintGCTimeStamps|-XX:+PrintClassHistogram|-XX:+PrintTenuringDistribution|-XX:+PrintGCApplicationStoppedTime|-XX:+PrintPromotionFailure|-verbose:gc|-XX:GCLogFileSize=64m|-XX:NumberOfGCLogFiles=4|-XX:+UseGCLogFileRotation|-XX:CompileCommand=exclude,org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addSortedNumericField|-XX:CompileCommand=exclude,org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addBinaryField|-XX:CompileCommand=exclude,org/elasticsearch/search/aggregations/metrics/percentiles/tdigest/AbstractTDigestPercentilesAggregator$1.collect|-XX:CompileCommand=exclude,org/apache/lucene/index/SortedNumericDocValuesWriter.flush|-XX:CompileCommand=exclude,org/apache/lucene/codecs/PushPostingsWriterBase.writeTerm|-Xloggc:"/opt/appdynamics/platform/product/events-service/processor/bin/../logs/%p-gc.log"|-DAPPLICATION_HOME="/opt/appdynamics/platform/product/events-service/processor/bin/.."|-classpath|"/opt/appdynamics/platform/product/events-service/processor/bin/../lib/*"|com.appdynamics.analytics.processor.AnalyticsService|-p|"/opt/appdynamics/platform/product/events-service/processor/./conf/events-service-api-store.properties"|-y|"/opt/appdynamics/platform/product/events-service/processor/bin/../conf/events-service-api-store.yml" CompilerOracle: exclude org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addSortedNumericField CompilerOracle: exclude org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addBinaryField CompilerOracle: exclude org/elasticsearch/search/aggregations/metrics/percentiles/tdigest/AbstractTDigestPercentilesAggregatorstart.collect CompilerOracle: exclude org/apache/lucene/index/SortedNumericDocValuesWriter.flush CompilerOracle: exclude org/apache/lucene/codecs/PushPostingsWriterBase.writeTerm 10:52:52.934 [main] INFO com.appdynamics.analytics.processor.AnalyticsService - Starting analytics processor with arguments [-p, /opt/appdynamics/platform/product/events-service/processor/./conf/events-service-api-store.properties, -y, /opt/appdynamics/platform/product/events-service/processor/bin/../conf/events-service-api-store.yml] ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging. pwd/opt/appdynamics/platform/product/events-service/processor[root@events-service-03 processor]# echo $JAVA_HOME/usr/lib/jvm/jre-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/[root@events-service-03 processor]# ./bin/events-service.sh start -p ./conf/events-service-api-store.properties &[1] 43742[root@events-service-03 processor]# ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.[2021-06-01T10:52:52,627+02:00] [APPLICATION_HOME] has been overridden to [/opt/appdynamics/platform/product/events-service/processor/bin/..][2021-06-01T10:52:52,631+02:00] [APPLICATION_HOME] has been overridden to [/opt/appdynamics/platform/product/events-service/processor/bin/..][2021-06-01T10:52:52,636+02:00] Program arguments: |-Xmx2990m|-Xms2990m|-Djava.net.preferIPv4Stack=true|-Dfile.encoding=UTF-8|-Djute.maxbuffer=30000000|-Dzookeeper.snapshot.trust.empty=true|-XX:+UseParNewGC|-XX:+UseConcMarkSweepGC|-XX:CMSInitiatingOccupancyFraction=75|-XX:+UseCMSInitiatingOccupancyOnly|-XX:+DisableExplicitGC|-XX:+PrintGCDetails|-XX:+PrintGCDateStamps|-XX:+PrintGCTimeStamps|-XX:+PrintClassHistogram|-XX:+PrintTenuringDistribution|-XX:+PrintGCApplicationStoppedTime|-XX:+PrintPromotionFailure|-verbose:gc|-XX:GCLogFileSize=64m|-XX:NumberOfGCLogFiles=4|-XX:+UseGCLogFileRotation|-XX:CompileCommand=exclude,org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addSortedNumericField|-XX:CompileCommand=exclude,org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addBinaryField|-XX:CompileCommand=exclude,org/elasticsearch/search/aggregations/metrics/percentiles/tdigest/AbstractTDigestPercentilesAggregator$1.collect|-XX:CompileCommand=exclude,org/apache/lucene/index/SortedNumericDocValuesWriter.flush|-XX:CompileCommand=exclude,org/apache/lucene/codecs/PushPostingsWriterBase.writeTerm|-Xloggc:"/opt/appdynamics/platform/product/events-service/processor/bin/../logs/%p-gc.log"|-DAPPLICATION_HOME="/opt/appdynamics/platform/product/events-service/processor/bin/.."|-classpath|"/opt/appdynamics/platform/product/events-service/processor/bin/../lib/*"|com.appdynamics.analytics.processor.AnalyticsService|-p|"/opt/appdynamics/platform/product/events-service/processor/./conf/events-service-api-store.properties"|-y|"/opt/appdynamics/platform/product/events-service/processor/bin/../conf/events-service-api-store.yml" CompilerOracle: exclude org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addSortedNumericFieldCompilerOracle: exclude org/apache/lucene/lucene54/Lucene54DocValuesConsumer.addBinaryFieldCompilerOracle: exclude org/elasticsearch/search/aggregations/metrics/percentiles/tdigest/AbstractTDigestPercentilesAggregatorstart.collectCompilerOracle: exclude org/apache/lucene/index/SortedNumericDocValuesWriter.flushCompilerOracle: exclude org/apache/lucene/codecs/PushPostingsWriterBase.writeTerm10:52:52.934 [main] INFO com.appdynamics.analytics.processor.AnalyticsService - Starting analytics processor with arguments [-p, /opt/appdynamics/platform/product/events-service/processor/./conf/events-service-api-store.properties, -y, /opt/appdynamics/platform/product/events-service/processor/bin/../conf/events-service-api-store.yml]ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. Set system property 'log4j2.debug' to show Log4j2 internal initialization logging.

Abdulrahman_Ka1 · ‎04-08-2021

Hello Alan, I got the same issue and after checking the documentation we should scale up events service as long as we run it as a single node especially in the dev & test environment however it's recommended to use a cluster of multi-nodes in a production environment https://docs.appdynamics.com/display/PRO21/Events+Service+Deployment

Abdulrahman_Ka1 · ‎04-07-2021

I'm trying to install the Events service from the Console host: my configuration 1- host has been added successfully and ssh 10.0.30.43 "passwordless" which is the other host 2- Java_Home variable is defined in both hosts 3- vim /etc/security/limits.d/appdynamics.conf 4- checked the error directory path and found "/opt/appdynamics/platform/product/orcha/21.2.0.363/orcha-modules/bin/orcha-modules" 5- running both commands from the enterprise console host and both give me the same error bin/platform-admin.sh install-events-service --profile prod --hosts 10.0.30.43 --data-dir /opt/appdynamics/eventsservice --platform-name AppDPlatform ./bin/platform-admin.sh submit-job --service events-service --job install --args serviceActionHost=10.0.30.43 profile=Prod [root@console platform-admin]# bin/platform-admin.sh install-events-service --profile prod --hosts 10.0.30.43 --data-dir /opt/appdynamics/eventsservice --platform-name AppDPlatform Installing Events Service on new nodes. ( 1/ 26) Clean up orphaned Events Service: SUCCESS ( 2/ 26) Register Events Service Lifecycle Listener before installing: SUCCESS ( 3/ 26) Validate and set parameters while installing ES: SUCCESS ( 4/ 26) Setup cluster configuration: SUCCESS ( 5/ 26) Initialize Events service cluster: SUCCESS ( 14/ 26) Set JRE Versions: SUCCESS ( 15/ 26) Check Events Service hostnames: SUCCESS ( 16/ 26) Verify hosts: FAILED Events Service installation failed. Failure occurred: Verify hosts Error message: Task failed: Facts collection on host: 10.0.30.43 as user: root with message: Error occurred while executing the task [facts]. env: /opt/appdynamics/platform/product/orcha/21.2.0.363/orcha-modules/bin/orcha-modules: No such file or directory

Abdulrahman_Ka1 · ‎04-05-2021

I'm trying to add Host through on-prem Linux enterprise console Server with below, the machine I want to add is windows  in the GUI I added Credential Name which I created through Credential Manager  and added the private key for the host I want to add, when I try to add the host through the GUI or the command line it tells me as screenshots below  Enterprise Console host expansion failed: A failure occurred: Validate SFTP configuration Error message: Task failed: Verify connection on host: controller as user: Abdulrahman.kazamel with a message: Failed to connect to the remote host. Please verify that the hostname and credentials you provided are correct.  

Posts	8
Solutions	1
Karma Given	0
Karma Received	5
Member Since	‎01-25-2021

Online Status	Offline
Date Last Visited	2 weeks ago

Starting Events Service cluster

installing Events service in different host

Setting up Events service host on windows

Re: Starting Events Service cluster

Re: How do I install Events Service via Enterprise...

Re: Events Service restart failed. Disable Transie...

Re: Starting Events Service cluster

Starting Events Service cluster

Re: Event Service health keeps turning critical

installing Events service in different host

Setting up Events service host on windows