Splunk AppDynamics

AppDynamics and ServiceNow integration for incidents

CommunityUser
Splunk Employee
Splunk Employee

Hello Experts,

We are testing the AppDynamics (we have on premises controller) integration with ServiceNow for creating incidents on health rule violation events. For that we referenced the following link https://community.appdynamics.com/t5/Knowledge-Base/How-to-use-AppDynamics-with-ServiceNow/ta-p/3009....

We have defined the action and policy for a health rule violation.

But, when the violation event is occuring, it is not generating any incidents in SNOW and summary states that "Action servicenow-api-alert failed.".

While checking in the server.log we observed the following error :-

[#|2018-04-13T10:15:07.677+0530|WARNING|glassfish 4.1|com.singularity.ee.controller.beans.action.CustomActionManagerBean.MARKED|_ThreadID=93;_ThreadName=p: thread-pool-1; w: 1;_TimeMillis=1523594707677;_LevelValue=900;|Action servicenow-api-alert failed. {ACTION=[2]}
com.singularity.ee.controller.api.exceptions.IllegalOperationException: com.singularity.ee.controller.api.exceptions.IllegalObjectStateException: Process [servicenow-api-alert] Exited with error code 1
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeCustomActionCommandList(CustomActionManagerBean.java:831)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.execute(CustomActionManagerBean.java:1015)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.execute(CustomActionManagerBean.java:137)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.appdynamics.platform.MethodExecutionTimeReporterEJBInterceptor.intercept(MethodExecutionTimeReporterEJBInterceptor.java:33)
    at sun.reflect.GeneratedMethodAccessor487.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.appdynamics.platform.KRolesInterceptor.methodInterceptor(KRolesInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor316.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy465.execute(Unknown Source)
    at com.singularity.ee.controller.beans.action.ActionExecutorBean.handleAction(ActionExecutorBean.java:352)
    at com.singularity.ee.controller.beans.action.ActionExecutorBean.executeEventNotificationActions(ActionExecutorBean.java:229)
    at sun.reflect.GeneratedMethodAccessor693.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy376.executeEventNotificationActions(Unknown Source)
    at com.singularity.ee.controller.beans.action.EventActionMessage.doUnitOfWork(EventActionMessage.java:134)
    at com.singularity.ee.controller.beans.shared.NoTxnUnitOfWorkProcessorBean.onMessage(NoTxnUnitOfWorkProcessorBean.java:86)
    at sun.reflect.GeneratedMethodAccessor681.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at org.glassfish.ejb.mdb.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1219)
    at org.glassfish.ejb.mdb.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:81)
    at com.sun.enterprise.connectors.inbound.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:171)
    at com.sun.proxy.$Proxy902.onMessage(Unknown Source)
    at com.sun.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:283)
    at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:107)
    at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
    at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)
Caused by: com.singularity.ee.controller.api.exceptions.IllegalObjectStateException: Process [servicenow-api-alert] Exited with error code 1
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeProgram(CustomActionManagerBean.java:755)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeCustomActionCommandList(CustomActionManagerBean.java:829)
    ... 126 more
|#]

So, we are sort of stuck here. Please let us know if we have missed any configuration. Also don't know if the attached extention in the above link is supported to the AppDynamics version - 4.3.0.4.

Thanks,

Varun

Labels (3)
Tags (1)
0 Karma
1 Solution

CommunityUser
Splunk Employee
Splunk Employee

Issue was caused because of java version mismatch, post updated the java version resolved the issue.

Thanks,

Varun

View solution in original post

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Issue was caused because of java version mismatch, post updated the java version resolved the issue.

Thanks,

Varun

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...