Splunk AppDynamics

I am using jsvc in unix enviroment to launch apache_tomcat, no matter where i put the -javaagent:agn

CommunityUser
Splunk Employee
Splunk Employee

NO matter where do i put the javaagent, no logs show up. Here is the command using ps -elf |grep javaagent

jsvc.exec -javaagent:/home/nohe/agent/javaagent.jar -java-home /usr/java -user nobody -pidfile /opt/apache-tomcat_1/logs/catalina-daemon.pid -wait 10 -errfile &1 -classpath /opt/apache-tomcat_1/bin/bootstrap.jar:/opt/apache-tomcat_1/bin/commons-daemon.jar:/opt/apache-tomcat_1/bin/tomcat-juli.jar -Djava.util.logging.config.file=/opt/apache-tomcat_1/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dorg.apache.catalina.security.SecurityListener.UMASK=0007 -Dorg.apache.catalina.STRICT_SERVLET_COMPLIANCE=true -Dorg.apache.catalina.connector.RECYCLE_FACADES=false -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=false -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=false -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=false -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=true -Dsun.net.inetaddr.ttl=30 -Dcom.sun.management.config.file=/opt/apache-tomcat_1/jmxconf/jvmmgmt/management.properties -Xms6144m -Xmx6144m -XX:MaxPermSize=1024m -Djava.net.preferIPv4Stack=true -Dorg.apache.tomcat.util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE=true -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -XX:+UseConcMarkSweepGC -Xmn2048M -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -Dlog4j.configurationFile=/opt/webex/conf/webapps/log4j2-ingrian.xml -Dorg.apache.tomcat.util.http.ServerCookie.FWD_SLASH_IS_SEPARATOR=false -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=false -Dsun.net.maxDatagramSockets=100 -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -javaagent:/home/nohe/agent/javaagent.jar -verbose:gc -Xloggc:/opt/apache-tomcat_1/logs/gc_2017-02-24_0135.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M -Djava.endorsed.dirs=/opt/apache-tomcat_1/endorsed -Dcatalina.base=/opt/apache-tomcat_1 -Dcatalina.home=/opt/apache-tomcat_1 -Djava.io.tmpdir=/opt/apache-tomcat_1/temp -Djava.security.manager -Djava.security.policy==/opt/apache-tomcat_1/conf/catalina.policy org.apache.catalina.startup.Bootstrap

 

 

5 S nobody   11312 11311 24  80   0 - 2363148 hrtime 01:35 ?      00:03:04 jsvc.exec -javaagent:/home/nohe/agent/javaagent.jar -java-home /usr/java -user nobody -pidfile /opt/apache-tomcat_1/logs/catalina-daemon.pid -wait 10 -errfile &1 -classpath /opt/apache-tomcat_1/bin/bootstrap.jar:/opt/apache-tomcat_1/bin/commons-daemon.jar:/opt/apache-tomcat_1/bin/tomcat-juli.jar -Djava.util.logging.config.file=/opt/apache-tomcat_1/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dorg.apache.catalina.security.SecurityListener.UMASK=0007 -Dorg.apache.catalina.STRICT_SERVLET_COMPLIANCE=true -Dorg.apache.catalina.connector.RECYCLE_FACADES=false -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=false -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=false -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=false -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=true -Dsun.net.inetaddr.ttl=30 -Dcom.sun.management.config.file=/opt/apache-tomcat_1/jmxconf/jvmmgmt/management.properties -Xms6144m -Xmx6144m -XX:MaxPermSize=1024m -Djava.net.preferIPv4Stack=true -Dorg.apache.tomcat.util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE=true -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -XX:+UseConcMarkSweepGC -Xmn2048M -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -Dlog4j.configurationFile=/opt/webex/conf/webapps/log4j2-ingrian.xml -Dorg.apache.tomcat.util.http.ServerCookie.FWD_SLASH_IS_SEPARATOR=false -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=false -Dsun.net.maxDatagramSockets=100 -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -javaagent:/home/nohe/agent/javaagent.jar -verbose:gc -Xloggc:/opt/apache-tomcat_1/logs/gc_2017-02-24_0135.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M -Djava.endorsed.dirs=/opt/apache-tomcat_1/endorsed -Dcatalina.base=/opt/apache-tomcat_1 -Dcatalina.home=/opt/apache-tomcat_1 -Djava.io.tmpdir=/opt/apache-tomcat_1/temp -Djava.security.manager -Djava.security.policy==/opt/apache-tomcat_1/conf/catalina.policy org.apache.catalina.startup.Bootstrap

 

Where to put the javaagent line in the jsvc launch scripts?

Labels (1)
Tags (2)
0 Karma

CommunityUser
Splunk Employee
Splunk Employee

there is no java process showing when using jsvc.exe

Here is the all process contain in java:

jsvc.exec -java-home /usr/java -user nobody -pidfile /opt/apache-tomcat_1/logs/catalina-daemon.pid -wait 10 -errfile &1 -classpath /opt/apache-tomcat_1/bin/bootstrap.jar:/opt/apache-tomcat_1/bin/commons-daemon.jar:/opt/apache-tomcat_1/bin/tomcat-juli.jar -Djava.util.logging.config.file=/opt/apache-tomcat_1/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dorg.apache.catalina.security.SecurityListener.UMASK=0007 -Dorg.apache.catalina.STRICT_SERVLET_COMPLIANCE=true -Dorg.apache.catalina.connector.RECYCLE_FACADES=false -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=false -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=false -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=false -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=true -Dsun.net.inetaddr.ttl=30 -Dcom.sun.management.config.file=/opt/apache-tomcat_1/jmxconf/jvmmgmt/management.properties -Xms6144m -Xmx6144m -XX:MaxPermSize=1024m -Djava.net.preferIPv4Stack=true -Dorg.apache.tomcat.util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE=true -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -XX:+UseConcMarkSweepGC -Xmn2048M -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -Dlog4j.configurationFile=/opt/webex/conf/webapps/log4j2-ingrian.xml -Dorg.apache.tomcat.util.http.ServerCookie.FWD_SLASH_IS_SEPARATOR=false -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=false -Dsun.net.maxDatagramSockets=100 -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -verbose:gc -Xloggc:/opt/apache-tomcat_1/logs/gc_2017-04-25_1844.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M -Djava.endorsed.dirs=/opt/apache-tomcat_1/endorsed -Dcatalina.base=/opt/apache-tomcat_1 -Dcatalina.home=/opt/apache-tomcat_1 -Djava.io.tmpdir=/opt/apache-tomcat_1/temp -Djava.security.manager -Djava.security.policy==/opt/apache-tomcat_1/conf/catalina.policy org.apache.catalina.startup.Bootstrap

5 S nobody   31605 31604 23  80   0 - 2340337 hrtime 18:44 ?      00:03:07 jsvc.exec -java-home /usr/java -user nobody -pidfile /opt/apache-tomcat_1/logs/catalina-daemon.pid -wait 10 -errfile &1 -classpath /opt/apache-tomcat_1/bin/bootstrap.jar:/opt/apache-tomcat_1/bin/commons-daemon.jar:/opt/apache-tomcat_1/bin/tomcat-juli.jar -Djava.util.logging.config.file=/opt/apache-tomcat_1/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dorg.apache.catalina.security.SecurityListener.UMASK=0007 -Dorg.apache.catalina.STRICT_SERVLET_COMPLIANCE=true -Dorg.apache.catalina.connector.RECYCLE_FACADES=false -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=false -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=false -Dorg.apache.coyote.USE_CUSTOM_STATUS_MSG_IN_HEADER=false -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=true -Dsun.net.inetaddr.ttl=30 -Dcom.sun.management.config.file=/opt/apache-tomcat_1/jmxconf/jvmmgmt/management.properties -Xms6144m -Xmx6144m -XX:MaxPermSize=1024m -Djava.net.preferIPv4Stack=true -Dorg.apache.tomcat.util.http.ServerCookie.ALLOW_EQUALS_IN_VALUE=true -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -XX:+UseConcMarkSweepGC -Xmn2048M -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -Dlog4j.configurationFile=/opt/webex/conf/webapps/log4j2-ingrian.xml -Dorg.apache.tomcat.util.http.ServerCookie.FWD_SLASH_IS_SEPARATOR=false -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=false -Dsun.net.maxDatagramSockets=100 -Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2 -verbose:gc -Xloggc:/opt/apache-tomcat_1/logs/gc_2017-04-25_1844.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M -Djava.endorsed.dirs=/opt/apache-tomcat_1/endorsed -Dcatalina.base=/opt/apache-tomcat_1 -Dcatalina.home=/opt/apache-tomcat_1 -Djava.io.tmpdir=/opt/apache-tomcat_1/temp -Djava.security.manager -Djava.security.policy==/opt/apache-tomcat_1/conf/catalina.policy org.apache.catalina.startup.Bootstrap

0 Karma

Rajesh_Putta
Communicator

Hi,

Thanks for the update. We would like to do a GoTo Meeting(GTM) an (online meeting session) to debug this issue further. If you are ready to do a GTM, please let us know atleast 2-3  time slots along with time zone that works for you. Based on time slots that you provide, we will send out a GTM invite

Thanks

Rajesh

0 Karma

Rajesh_Putta
Communicator

Hi,

In addition to earlier update

It appears that tomcat 6 has a JSVC in it that has a bug not allowing the full command line args including the -javaagent arg.  

Could you please provide us what version you are using for JSVC.

Please run following command

./jsvc -version

For more information about the bug

Apache bug tracking the issue

https://issues.apache.org/jira/browse/DAEMON-84

There is a fix in the Apache Commons source repository. For more information:



may also be able to use a binary build of jsvc .

Thanks

Rajesh

0 Karma

CommunityUser
Splunk Employee
Splunk Employee
Java version is : 1.8.0_112-b15,
Jsvc version is: 1.0.15-dev


0 Karma

Rajesh_Putta
Communicator

Hi,

Thanks for the update. I have asked my colleague Shreyas, who works in PST time zone to schedule a GTM and send you the invitation as per your preferred slots. Please do watch out for GTM invite from him.

Thanks

Rajesh

0 Karma

Shreyas_Joshi
New Member

Hi,

Could you please provide us your availability for early next week. We are loacted in PST.

Thanks,

Shreyas Joshi.

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

I have the same issue.

i couldn't affect 2 different javaagent parametres because they are using jsvc.

Did you resolved this case ?

CommunityUser
Splunk Employee
Splunk Employee
Thursday 4:30PM- 5:30PM or Friday 10:00- 11:00 am or Firday 2:30PM- 3:30PM. All in Pacific Time Zone.
0 Karma

Rajesh_Putta
Communicator

Hi,

Could you please try the option listed here

A) Please remove the java agent configurations where ever you have configured and follow the steps listed here

  • The user under which the JVM runs must have write privileges to the conf and logs directories in the Java Agent home. To achieve this, you can install the agent as the same user that owns the JVM or as an administrator on the host machine. The other contents of the agent directory can be restricted to read only access. 

To attach the agent to the JVM, follow these steps: 

  1. Determine the PID of the JVM to which you want to attach. For example, on Linux, use:

    ps -A | grep java

    On Windows, use: 

    jps -l
  2. Run the following command, replacing the placeholders for the path to the tools.jar file in your JDK, path to the AppDynamics Java Agent home directory, and the JVM process ID with values appropriate for your environment:

    java -Xbootclasspath/a:<path_to_jdk>/lib/tools.jar -jar /<agent_home>/javaagent.jar <jvm_process_id>

    Use the equivalent paths for Windows, including drive letter. The following shows an example with system output included:

    [appduser@my_centos6 ~]$ ps -A | grep java
     6780 pts/1    00:00:04 java
    [appduser@my_centos6 ~]$ java -Xbootclasspath/a:/usr/java/jdk1.7.0_79/lib/tools.jar -jar /home/appduser/appagent/javaagent.jar 6780
    Attaching to VM [6780]
    agent path >>>/home/appduser/appagent/javaagent.jar

    Thanks
    Rajesh
0 Karma
Get Updates on the Splunk Community!

New This Month - Splunk Observability updates and improvements for faster ...

What’s New? This month, we’re delivering several enhancements across Splunk Observability Cloud for faster and ...

What's New in Splunk Cloud Platform 9.3.2411?

Hey Splunky People! We are excited to share the latest updates in Splunk Cloud Platform 9.3.2411. This release ...

Buttercup Games: Further Dashboarding Techniques (Part 6)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...