Splunk AppDynamics

AppDynamics and ServiceNow integration for incidents

CommunityUser
Splunk Employee
Splunk Employee

Hello Experts,

We are testing the AppDynamics (we have on premises controller) integration with ServiceNow for creating incidents on health rule violation events. For that we referenced the following link https://community.appdynamics.com/t5/Knowledge-Base/How-to-use-AppDynamics-with-ServiceNow/ta-p/3009....

We have defined the action and policy for a health rule violation.

But, when the violation event is occuring, it is not generating any incidents in SNOW and summary states that "Action servicenow-api-alert failed.".

While checking in the server.log we observed the following error :-

[#|2018-04-13T10:15:07.677+0530|WARNING|glassfish 4.1|com.singularity.ee.controller.beans.action.CustomActionManagerBean.MARKED|_ThreadID=93;_ThreadName=p: thread-pool-1; w: 1;_TimeMillis=1523594707677;_LevelValue=900;|Action servicenow-api-alert failed. {ACTION=[2]}
com.singularity.ee.controller.api.exceptions.IllegalOperationException: com.singularity.ee.controller.api.exceptions.IllegalObjectStateException: Process [servicenow-api-alert] Exited with error code 1
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeCustomActionCommandList(CustomActionManagerBean.java:831)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.execute(CustomActionManagerBean.java:1015)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.execute(CustomActionManagerBean.java:137)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.appdynamics.platform.MethodExecutionTimeReporterEJBInterceptor.intercept(MethodExecutionTimeReporterEJBInterceptor.java:33)
    at sun.reflect.GeneratedMethodAccessor487.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.appdynamics.platform.KRolesInterceptor.methodInterceptor(KRolesInterceptor.java:52)
    at sun.reflect.GeneratedMethodAccessor316.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy465.execute(Unknown Source)
    at com.singularity.ee.controller.beans.action.ActionExecutorBean.handleAction(ActionExecutorBean.java:352)
    at com.singularity.ee.controller.beans.action.ActionExecutorBean.executeEventNotificationActions(ActionExecutorBean.java:229)
    at sun.reflect.GeneratedMethodAccessor693.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
    at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
    at sun.reflect.GeneratedMethodAccessor75.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
    at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
    at com.sun.proxy.$Proxy376.executeEventNotificationActions(Unknown Source)
    at com.singularity.ee.controller.beans.action.EventActionMessage.doUnitOfWork(EventActionMessage.java:134)
    at com.singularity.ee.controller.beans.shared.NoTxnUnitOfWorkProcessorBean.onMessage(NoTxnUnitOfWorkProcessorBean.java:86)
    at sun.reflect.GeneratedMethodAccessor681.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
    at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
    at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
    at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.EntityManagerInterceptor.manageEntityManagerLifecycle(EntityManagerInterceptor.java:57)
    at sun.reflect.GeneratedMethodAccessor294.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
    at com.singularity.ee.controller.beans.ExceptionHandlingInterceptor.handleExceptions(ExceptionHandlingInterceptor.java:55)
    at sun.reflect.GeneratedMethodAccessor293.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
    at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
    at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
    at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
    at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
    at org.glassfish.ejb.mdb.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1219)
    at org.glassfish.ejb.mdb.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:81)
    at com.sun.enterprise.connectors.inbound.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:171)
    at com.sun.proxy.$Proxy902.onMessage(Unknown Source)
    at com.sun.messaging.jms.ra.OnMessageRunner.run(OnMessageRunner.java:283)
    at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:107)
    at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.performWork(ThreadPoolImpl.java:497)
    at com.sun.corba.ee.impl.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:540)
Caused by: com.singularity.ee.controller.api.exceptions.IllegalObjectStateException: Process [servicenow-api-alert] Exited with error code 1
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeProgram(CustomActionManagerBean.java:755)
    at com.singularity.ee.controller.beans.action.CustomActionManagerBean.executeCustomActionCommandList(CustomActionManagerBean.java:829)
    ... 126 more
|#]

So, we are sort of stuck here. Please let us know if we have missed any configuration. Also don't know if the attached extention in the above link is supported to the AppDynamics version - 4.3.0.4.

Thanks,

Varun

Labels (3)
Tags (1)
0 Karma
1 Solution

CommunityUser
Splunk Employee
Splunk Employee

Issue was caused because of java version mismatch, post updated the java version resolved the issue.

Thanks,

Varun

View solution in original post

0 Karma

CommunityUser
Splunk Employee
Splunk Employee

Issue was caused because of java version mismatch, post updated the java version resolved the issue.

Thanks,

Varun

0 Karma
Get Updates on the Splunk Community!

What the End of Support for Splunk Add-on Builder Means for You

Hello Splunk Community! We want to share an important update regarding the future of the Splunk Add-on Builder ...

Solve, Learn, Repeat: New Puzzle Channel Now Live

Welcome to the Splunk Puzzle PlaygroundIf you are anything like me, you love to solve problems, and what ...

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...