AppD Archive

eum-sh script fails

CommunityUser
Splunk Employee
Splunk Employee

hello, 
i have installed my controller, events-service,platform admin, services (ALL ON SITE) and they're running.

i have also installed eum and enabled in the Controller UI but when i try to run the eum.sh (script) to start the eum-processor it fails with the below message....
what could be the issue here please?

Using EUM_HOME: /data/app/AppDynamics/EUM/eum-processor
Using EUM_PID: pid.txt
EUM Processor started (PID=20636).
[root@AppDynamic eum-processor]# java.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException: Connect to AppDynamic:9080 [AppDynamic/172.16.2.10] failed: Connection refused
at com.appdynamics.eumcloud.analytics.AnalyticsAccountManager.bulkUpdateAnEventType(AnalyticsAccountManager.java:353)
at com.appdynamics.eumcloud.analytics.AnalyticsAccountManager.bulkUpdateEventTypes(AnalyticsAccountManager.java:275)
at com.appdynamics.eumcloud.EUMProcessorServer.main(EUMProcessorServer.java:228)
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to AppDynamic:9080 [AppDynamic/172.16.2.10] failed: Connection refused
at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:140)
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:318)
at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at com.appdynamics.eumcloud.analytics.AnalyticsAccountManager.bulkUpdateAnEventType(AnalyticsAccountManager.java:324)
... 2 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:72)
at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:123)
... 12 more
Error Starting EUM Processor Serverjava.lang.RuntimeException: org.apache.http.conn.HttpHostConnectException: Connect to AppDynamic:9080 [AppDynamic/172.16.2.10] failed: Connection refused

0 Karma

Arun_Dasetty
Super Champion

Hi,

- Can you please issue below command and see how it goes with eum scripts again , if not started events service jvm:

shell> cd /data/app/AppDynamics/Controller/bin (change to controller bin directory if this is not the same path)

shell>./controller.sh stop-events-service

shell>./controller.sh start-events-service

- Can you provide below command outout if issue persists:

shell> ps -ef | grep "java"

shell> netstat -a

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

[root@appdynamics-evnt-service bin]# ps -ef | grep "java"
root 21860 20241 0 10:05 pts/1 00:00:00 grep java
root 26767 1 0 Sep21 pts/0 00:05:40 /usr/lib/jvm/jre-1.7.0-openjdk.x86_64/bin/java -Xmx8G -Xms8G -Xss256k -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF-8 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintClassHistogram -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionFailure -verbose:gc -XX:GCLogFileSize=256m -XX:NumberOfGCLogFiles=4 -XX:+UseGCLogFileRotation -Xloggc:/data/apps/events-service/logs/events-service-all-gc.log -classpath /data/apps/events-service/lib/* com.appdynamics.analytics.processor.AnalyticsService -y ./conf/events-service-all.yml -p ./conf/events-service-all.properties

================================================================================================

[root@appdynamics-evnt-service bin]# netstat -a
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:zabbix-agent *:* LISTEN
tcp 0 0 *:sunrpc *:* LISTEN
tcp 0 0 *:wap-wsp *:* LISTEN
tcp 0 0 *:vrace *:* LISTEN
tcp 0 0 *:ssh *:* LISTEN
tcp 0 0 localhost.localdomain:ipp *:* LISTEN
tcp 0 0 *:glrpc *:* LISTEN
tcp 0 0 *:9081 *:* LISTEN
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51319 ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51320 ESTABLISHED
tcp 0 0 localhost.localdomain:51313 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51316 ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51314 ESTABLISHED
tcp 0 0 localhost.localdomain:51321 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 localhost.localdomain:51317 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51309 ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51313 ESTABLISHED
tcp 0 0 appdynamics-evnt-serv:glrpc AppDynamic:fairview ESTABLISHED
tcp 0 0 localhost.localdomain:51319 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 appdynamics-evnt-serv:glrpc AppDynamic:38199 ESTABLISHED
tcp 0 0 localhost.localdomain:51314 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51311 ESTABLISHED
tcp 0 0 localhost.localdomain:51315 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51310 ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51321 ESTABLISHED
tcp 0 0 localhost.localdomain:51320 localhost.localdomain:vrace ESTABLISHED
tcp 0 1200 appdynamics-evnt-servic:ssh ip-172-16-1-5.ec2.int:60876 ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51312 ESTABLISHED
tcp 0 0 localhost.localdomain:51316 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51317 ESTABLISHED
tcp 0 0 localhost.localdomain:51310 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 localhost.localdomain:51309 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 appdynamics-evnt-servic:ssh ip-172-16-1-5.ec2.int:55008 ESTABLISHED
tcp 0 0 localhost.localdomain:51318 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51318 ESTABLISHED
tcp 0 0 localhost.localdomain:vrace localhost.localdomain:51315 ESTABLISHED
tcp 0 0 localhost.localdomain:51312 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 appdynamics-evnt-serv:glrpc AppDynamic:38184 TIME_WAIT
tcp 0 0 localhost.localdomain:51311 localhost.localdomain:vrace ESTABLISHED
tcp 0 0 *:zabbix-agent *:* LISTEN
tcp 0 0 *:sunrpc *:* LISTEN
tcp 0 0 *:http *:* LISTEN
tcp 0 0 *:ssh *:* LISTEN
tcp 0 0 localhost:ipp *:* LISTEN
udp 0 0 *:realm-rusd *:*
udp 0 0 *:bootpc *:*
udp 0 0 *:sunrpc *:*
udp 0 0 *:ipp *:*
udp 0 0 appdynamics-evnt-service:ntp *:*
udp 0 0 localhost.localdomain:ntp *:*
udp 0 0 *:ntp *:*
udp 0 0 *:realm-rusd *:*
udp 0 0 *:sunrpc *:*
udp 0 0 fe80::895:2fff:fe49:642b:ntp *:*
udp 0 0 localhost:ntp *:*
udp 0 0 *:ntp *:*
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ACC ] STREAM LISTENING 9114 /var/run/dbus/system_bus_socket
unix 2 [ ACC ] STREAM LISTENING 6630 @/com/ubuntu/upstart
unix 2 [ ACC ] STREAM LISTENING 9228 @/var/run/hald/dbus-n7SpJHGvun
unix 2 [ ACC ] STREAM LISTENING 9001 /var/run/rpcbind.sock
unix 2 [ ACC ] STREAM LISTENING 9233 @/var/run/hald/dbus-aGuvZMYaDX
unix 11 [ ] DGRAM 8843 /dev/log
unix 2 [ ] DGRAM 9258 @/org/freedesktop/hal/udev_event
unix 2 [ ] DGRAM 7055 @/org/kernel/udev/udevd
unix 2 [ ACC ] STREAM LISTENING 9176 /var/run/cups/cups.sock
unix 2 [ ] DGRAM 659302
unix 3 [ ] STREAM CONNECTED 656927
unix 3 [ ] STREAM CONNECTED 656926
unix 2 [ ] DGRAM 656923
unix 2 [ ] STREAM CONNECTED 601623
unix 2 [ ] STREAM CONNECTED 601311
unix 2 [ ] DGRAM 574761
unix 3 [ ] STREAM CONNECTED 574714
unix 3 [ ] STREAM CONNECTED 574713
unix 2 [ ] DGRAM 574710
unix 2 [ ] DGRAM 336044
unix 2 [ ] DGRAM 131901
unix 2 [ ] DGRAM 11964
unix 3 [ ] STREAM CONNECTED 10580 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 10579
unix 2 [ ] DGRAM 10313
unix 2 [ ] DGRAM 10096
unix 2 [ ] DGRAM 9531
unix 3 [ ] STREAM CONNECTED 9253 @/var/run/hald/dbus-aGuvZMYaDX
unix 3 [ ] STREAM CONNECTED 9252
unix 3 [ ] STREAM CONNECTED 9230 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 9229
unix 2 [ ] DGRAM 9181
unix 3 [ ] STREAM CONNECTED 9126 /var/run/dbus/system_bus_socket
unix 3 [ ] STREAM CONNECTED 9125
unix 3 [ ] STREAM CONNECTED 9119
unix 3 [ ] STREAM CONNECTED 9118
unix 3 [ ] STREAM CONNECTED 8642
unix 3 [ ] STREAM CONNECTED 8641
unix 3 [ ] DGRAM 7070
unix 3 [ ] DGRAM 7069

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

as it stands now the eum.sh script is running but is failing to push any data to the Controller UI
thanks

0 Karma

Arun_Dasetty
Super Champion

Hi,

Can you send the events and eum processor logs and controller server.log under /data/app/AppDynamics/Controller//logs path?

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Hey Akumar,

here are the logs from the controller and eum-processor............

at this moment the eum.sh script has run and i can confirm connection between the controller and eum using http://172.16.2.xxx:7001/eumaggregator/ping

but no data is available on the controller UI

0 Karma
Get Updates on the Splunk Community!

See just what you’ve been missing | Observability tracks at Splunk University

Looking to sharpen your observability skills so you can better understand how to collect and analyze data from ...

Weezer at .conf25? Say it ain’t so!

Hello Splunkers, The countdown to .conf25 is on-and we've just turned up the volume! We're thrilled to ...

How SC4S Makes Suricata Logs Ingestion Simple

Network security monitoring has become increasingly critical for organizations of all sizes. Splunk has ...