Splunk AppDynamics

AppDynamics and ServiceNow integration for incidents

CommunityUser
Splunk Employee
Splunk Employee

Hello Experts,

We are testing the AppDynamics (we have on premises controller) integration with ServiceNow for creating incidents on health rule violation events. For that we referenced the following link https://community.appdynamics.com/t5/Knowledge-Base/How-to-use-AppDynamics-with-ServiceNow/ta-p/3009....

We have defined the action and policy for a health rule violation.

But, when the violation event is occuring, it is not generating any incidents in SNOW and summary states that "Action servicenow-api-alert failed.".

While checking in the server.log we observed the following error :-

[#|2018-04-13T10:15:07.677+0530|WARNING|glassfish 4.1|com.singularity.ee.controller.beans.action.CustomActionManagerBean.MARKED|_ThreadID=93;_ThreadName=p: thread-pool-1; w: 1;_TimeMillis=1523594707677;_LevelValue=900;|Action servicenow-api-alert failed. {ACTION=[2]}
com.singularity.ee.controller.api.exceptions.IllegalOperationException: com.singularity.ee.controller.api.exceptions.IllegalObjectStateException: Process [servicenow-api-alert] Exited with error code 1
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeCustomActionCommandList(CustomActionManagerBean.java:831)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.execute(CustomActionManagerBean.java:1015)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.execute(CustomActionManagerBean.java:137)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.appdynamics.platform.MethodExecutionTimeReporterEJBInterceptor.intercept(MethodExecutionTimeReporterEJBInterceptor.java:33)
    at sun.reflect.GeneratedMethodAccessor487.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.appdynamics.platform.KRolesInterceptor.methodInterceptor(KRolesInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor316.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy465.execute(Unknown Source)
    at com.singularity.ee.controller.beans.action.ActionExecutorBean.handleAction(ActionExecutorBean.java:352)
    at com.singularity.ee.controller.beans.action.ActionExecutorBean.executeEventNotificationActions(ActionExecutorBean.java:229)
    at sun.reflect.GeneratedMethodAccessor693.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy376.executeEventNotificationActions(Unknown Source)
    at com.singularity.ee.controller.beans.action.EventActionMessage.doUnitOfWork(EventActionMessage.java:134)
    at com.singularity.ee.controller.beans.shared.NoTxnUnitOfWorkProcessorBean.onMessage(NoTxnUnitOfWorkProcessorBean.java:86)
    at sun.reflect.GeneratedMethodAccessor681.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at org.glassfish.ejb.mdb.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1219)
    at org.glassfish.ejb.mdb.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:81)
    at com.sun.enterprise.connectors.inbound.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:171)
    at com.sun.proxy.$Proxy902.onMessage(Unknown Source)
    at com.sun.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:283)
    at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:107)
    at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
    at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)
Caused by: com.singularity.ee.controller.api.exceptions.IllegalObjectStateException: Process [servicenow-api-alert] Exited with error code 1
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeProgram(CustomActionManagerBean.java:755)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeCustomActionCommandList(CustomActionManagerBean.java:829)
    ... 126 more
|#]

So, we are sort of stuck here. Please let us know if we have missed any configuration. Also don't know if the attached extention in the above link is supported to the AppDynamics version - 4.3.0.4.

Thanks,

Varun

Labels (3)
Tags (1)
0 Karma
1 Solution

CommunityUser
Splunk Employee
Splunk Employee

Issue was caused because of java version mismatch, post updated the java version resolved the issue.

Thanks,

Varun

View solution in original post

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Issue was caused because of java version mismatch, post updated the java version resolved the issue.

Thanks,

Varun

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...