All Apps and Add-ons

Bug in Splunk App for AWS: User unable to configure a CloudTrail input

connerswann
New Member

Hi Splunk Answers, this isn't a question per se, however, I think it will help people who might find this via Google.

I found a bug in the Splunk App for AWS that results in a user being unable to configure a CloudTrail input to ingest data from AWS CloudTrail. Even if a user has successfully set up their chain of AWS services (CloudTrail > S3 > SNS > SQS), the config form does not auto-populate the names of the existing Simple Queues that fit the app's criteria. I found this to be due to a bug in the code governing which queues populate this drop-down.

The file in question is $AWS_APP_ROOT/bin/aws/aws_utils.py in the function get_sqs_of_topics() on line 128.

The relevant excerpt of the get_sqs_topics function from the app's original code is as follows:

    # match queues
    sqs = set()
    for subscription in subscriptions:
        if subscription['Protocol'] == 'sqs':
            topic_name = extract_name_from_arn(subscription['TopicArn'])
            if topic_name in topics:
                sqs.add((extract_name_from_arn(subscription['Endpoint']), subscription['Endpoint']))

    return sqs

The fix is below:

 # match queues
    sqs = set()
    for subscription in subscriptions:
        if subscription['Protocol'] == 'sqs':
            #FIX IS BELOW
            topic_name = subscription['TopicArn']
            # FIX IS ABOVE
            if topic_name in topics:
                sqs.add((extract_name_from_arn(subscription['Endpoint']), subscription['Endpoint']))

    return sqs

In a nutshell, the original code is extracting a portion of an ARN and then attempting to use the excerpt to do an exact string match to a full-ARN which doesn't work. To fix this, I modified it to simply not extract the "name" from the ARN and to simply use the full ARN to perform the string match.

Hope this helps someone, and I hope that the Splunk Product guys see this and implement a fix!

Best,
Conner Swann

0 Karma
1 Solution

pchen_splunk
Splunk Employee
Splunk Employee

Thanks for sharing this.

There is a bug fired from customer side (AWSAPP-1014). AWS app development team has resolved it and put in into v4.2.1 release.

The fix Conner provided works if the trail is configured recently, but not works for old trails.

If you want to apply the fix before v4.2.1 release, you can try fix one line code as below:

File: $AWS_APP_ROOT/bin/aws/aws_utils.py
Function: get_cloudtrail_sqs

Before fix: topic_names = set([x['SnsTopicName'] for x in trails])
After fix: topic_names = set([extract_name_from_arn(x['SnsTopicARN']) for x in trails if 'SnsTopicARN' in x])

View solution in original post

pchen_splunk
Splunk Employee
Splunk Employee

Thanks for sharing this.

There is a bug fired from customer side (AWSAPP-1014). AWS app development team has resolved it and put in into v4.2.1 release.

The fix Conner provided works if the trail is configured recently, but not works for old trails.

If you want to apply the fix before v4.2.1 release, you can try fix one line code as below:

File: $AWS_APP_ROOT/bin/aws/aws_utils.py
Function: get_cloudtrail_sqs

Before fix: topic_names = set([x['SnsTopicName'] for x in trails])
After fix: topic_names = set([extract_name_from_arn(x['SnsTopicARN']) for x in trails if 'SnsTopicARN' in x])

View solution in original post

dmckean
Engager

When will version 4.2.1 be released to SplunkCloud to fix this issue? It's a major blocker for adopting Splunk in my organization.

SplunkCloud is still using version 4.0.0, by the way.

0 Karma

pchen_splunk
Splunk Employee
Splunk Employee

AWS App v4.2.1 is release in July 2016. Please ask support to upgrade your app. Or, you can do upgrade by yourself in the app management page.

0 Karma

connerswann
New Member

That's a better code snippet! Hope nobody else ran into this, took me forever to debug!

0 Karma

pchen_splunk
Splunk Employee
Splunk Employee

Please submit a support ticket to Splunk when you meet any issue in the future. Or, you can ask questions in this place, we will try our best to answer in the first time. Thanks for understanding.

0 Karma

ppablo
Community Manager
Community Manager

Hi @connerswann

Thanks for sharing this with the Splunk community. Did you actually file a ticket for this bug? If you have a support entitlement, you can submit your findings here:
http://www.splunk.com/r/bugs

0 Karma

connerswann
New Member

I haven't filed a ticket as I'm on a Developer License and do not have a support account. I figured this was the best place to post it given the circumstances.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.