Splunk AppDynamics

AppDynamics and ServiceNow integration for incidents

CommunityUser
Splunk Employee
Splunk Employee

Hello Experts,

We are testing the AppDynamics (we have on premises controller) integration with ServiceNow for creating incidents on health rule violation events. For that we referenced the following link https://community.appdynamics.com/t5/Knowledge-Base/How-to-use-AppDynamics-with-ServiceNow/ta-p/3009....

We have defined the action and policy for a health rule violation.

But, when the violation event is occuring, it is not generating any incidents in SNOW and summary states that "Action servicenow-api-alert failed.".

While checking in the server.log we observed the following error :-

[#|2018-04-13T10:15:07.677+0530|WARNING|glassfish 4.1|com.singularity.ee.controller.beans.action.CustomActionManagerBean.MARKED|_ThreadID=93;_ThreadName=p: thread-pool-1; w: 1;_TimeMillis=1523594707677;_LevelValue=900;|Action servicenow-api-alert failed. {ACTION=[2]}
com.singularity.ee.controller.api.exceptions.IllegalOperationException: com.singularity.ee.controller.api.exceptions.IllegalObjectStateException: Process [servicenow-api-alert] Exited with error code 1
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeCustomActionCommandList(CustomActionManagerBean.java:831)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.execute(CustomActionManagerBean.java:1015)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.execute(CustomActionManagerBean.java:137)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.appdynamics.platform.MethodExecutionTimeReporterEJBInterceptor.intercept(MethodExecutionTimeReporterEJBInterceptor.java:33)
    at sun.reflect.GeneratedMethodAccessor487.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.appdynamics.platform.KRolesInterceptor.methodInterceptor(KRolesInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor316.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy465.execute(Unknown Source)
    at com.singularity.ee.controller.beans.action.ActionExecutorBean.handleAction(ActionExecutorBean.java:352)
    at com.singularity.ee.controller.beans.action.ActionExecutorBean.executeEventNotificationActions(ActionExecutorBean.java:229)
    at sun.reflect.GeneratedMethodAccessor693.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy376.executeEventNotificationActions(Unknown Source)
    at com.singularity.ee.controller.beans.action.EventActionMessage.doUnitOfWork(EventActionMessage.java:134)
    at com.singularity.ee.controller.beans.shared.NoTxnUnitOfWorkProcessorBean.onMessage(NoTxnUnitOfWorkProcessorBean.java:86)
    at sun.reflect.GeneratedMethodAccessor681.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at org.glassfish.ejb.mdb.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1219)
    at org.glassfish.ejb.mdb.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:81)
    at com.sun.enterprise.connectors.inbound.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:171)
    at com.sun.proxy.$Proxy902.onMessage(Unknown Source)
    at com.sun.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:283)
    at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:107)
    at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
    at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)
Caused by: com.singularity.ee.controller.api.exceptions.IllegalObjectStateException: Process [servicenow-api-alert] Exited with error code 1
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeProgram(CustomActionManagerBean.java:755)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeCustomActionCommandList(CustomActionManagerBean.java:829)
    ... 126 more
|#]

So, we are sort of stuck here. Please let us know if we have missed any configuration. Also don't know if the attached extention in the above link is supported to the AppDynamics version - 4.3.0.4.

Thanks,

Varun

Labels (3)
Tags (1)
0 Karma
1 Solution

CommunityUser
Splunk Employee
Splunk Employee

Issue was caused because of java version mismatch, post updated the java version resolved the issue.

Thanks,

Varun

View solution in original post

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Issue was caused because of java version mismatch, post updated the java version resolved the issue.

Thanks,

Varun

0 Karma
Get Updates on the Splunk Community!

What's New in Splunk Cloud Platform 9.3.2411?

Hey Splunky People! We are excited to share the latest updates in Splunk Cloud Platform 9.3.2411. This release ...

Buttercup Games: Further Dashboarding Techniques (Part 6)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...