Splunk AppDynamics

How do I prevent SocketTimeoutException exceptions from triggering Actions?

CommunityUser
Splunk Employee
Splunk Employee

I'm trying to report errors and exceptions that happen in a test environment into a Slack channel.  Everytime new code is deployed, every instance triggers an error like the following:

Count	Event Type
1	APPLICATION_ERROR
ERROR	Application Error
Thu Aug 31 14:27:51 PDT 2017 | app17 | app17_app17-test-app-1.corp.company.net

SocketTimeoutException: connect timed out

I have tried everything I can think of to prevent these exceptions from triggering Actions.

image.pngimage.pngimage.png

Does this configuration not apply to the Alert & Respond policies?  My controller version is 4.2.15.3, build 41.

Labels (3)
0 Karma

CommunityUser
Splunk Employee
Splunk Employee

I have had some success with integrating with slack. I did this using a slack webhook which was nothing more than just an app I created and enabled the webhook for. You mainly need this to provide the URL with a token to be used by AppDynamics for posting the events. In AppDynamics just create an html request template with the following settings:

Request URL method: post

Raw URL: (insert URL for the webhook for your slack app here)

Payload mime type: application/json

Here is the json I use, it creates a message with attachments to organize the content, colorizes based on event level (critical, warn, info), and normalizes the html tags in the messages to markdown which slack can display for rich outout.

#set( $em = ${latestEvent.eventMessage} )
#set( $em = $em.replace("<b>", "*") )
#set( $em = $em.replace("</b>", "*") )
#set( $em = $em.replace("<br>", "\n") )

#if ( ${latestEvent.tier.name} )#set ( $tier-name = ${latestEvent.tier.name} )#else#set ( $tier-name = "n/a" )#end
#if ( ${latestEvent.node.name} )#set ( $node-name = ${latestEvent.node.name} )#else#set ( $node-name = "n/a" )#end

#if ( ${topSeverity} == "ERROR" )#set ( $severity = "danger" )#end
#if ( ${topSeverity} == "WARN" )#set ( $severity = "warning" )#end
#if ( ${topSeverity} == "INFO" )#set ( $severity = "good" )#end

{
    "attachments": [
        {
            "fallback": "AppDynamics Alert",
            "color": "$severity",
            "pretext": "AppDynamics ${topSeverity} Alert",
            "title": "${latestEvent.deepLink}",
            "title_link": "${latestEvent.deepLink}",
            "text":"${latestEvent.healthRule.name}",
            "mrkdwn_in": ["fields"],
            "fields": [
                {
                    "title": "Severity",
                    "value": "${topSeverity}",
                    "short": true
                },
                {
                    "title": "Application",
                    "value": "${latestEvent.application.name}",
                    "short": true
                },
                {
                    "title": "Tier",
                    "value": "$tier-name",
                    "short": true
                },
                {
                    "title": "Node",
                    "value": "$node-name",
                    "short": true
                },
                {
                    "title": "Message",
                    "value": "$em",
                    "short": false
                }
            ],
            "image_url": "latestEvent.severityImage.deepLink"
        }
    ]
}

I hope this helps you.

Get Updates on the Splunk Community!

Finding Based Detections General Availability

Overview  We’ve come a long way, folks, but here in Enterprise Security 8.4 I’m happy to announce Finding ...

Get Your Hands Dirty (and Your Shoes Comfy): The Splunk Experience

Hands-On Learning and Technical Seminars  Sometimes, you just need to see the code. For those looking for a ...

What’s New in Splunk Observability Cloud: January Feature Highlights & Deep Dives

Splunk Observability Cloud continues to evolve, empowering engineering and operations teams with advanced ...