topic Re: Bug in Splunk App for AWS: User unable to configure a CloudTrail input in All Apps and Add-ons

Bug in Splunk App for AWS: User unable to configure a CloudTrail input

connerswann — Tue, 28 Jun 2016 00:22:27 GMT

Hi Splunk Answers, this isn't a question per se, however, I think it will help people who might find this via Google.

I found a bug in the Splunk App for AWS that results in a user being unable to configure a CloudTrail input to ingest data from AWS CloudTrail. Even if a user has successfully set up their chain of AWS services (CloudTrail > S3 > SNS > SQS), the config form does not auto-populate the names of the existing Simple Queues that fit the app's criteria. I found this to be due to a bug in the code governing which queues populate this drop-down.

The file in question is $AWS_APP_ROOT/bin/aws/aws_utils.py in the function get_sqs_of_topics() on line 128.

The relevant excerpt of the get_sqs_topics function from the app's original code is as follows:

    # match queues
    sqs = set()
    for subscription in subscriptions:
        if subscription['Protocol'] == 'sqs':
            topic_name = extract_name_from_arn(subscription['TopicArn'])
            if topic_name in topics:
                sqs.add((extract_name_from_arn(subscription['Endpoint']), subscription['Endpoint']))

    return sqs

The fix is below:

 # match queues
    sqs = set()
    for subscription in subscriptions:
        if subscription['Protocol'] == 'sqs':
            #FIX IS BELOW
            topic_name = subscription['TopicArn']
            # FIX IS ABOVE
            if topic_name in topics:
                sqs.add((extract_name_from_arn(subscription['Endpoint']), subscription['Endpoint']))

    return sqs

In a nutshell, the original code is extracting a portion of an ARN and then attempting to use the excerpt to do an exact string match to a full-ARN which doesn't work. To fix this, I modified it to simply not extract the "name" from the ARN and to simply use the full ARN to perform the string match.

Hope this helps someone, and I hope that the Splunk Product guys see this and implement a fix!

Best,
Conner Swann

Re: Bug in Splunk App for AWS: User unable to configure a CloudTrail input

ppablo — Tue, 28 Jun 2016 00:41:02 GMT

Hi @connerswann

Thanks for sharing this with the Splunk community. Did you actually file a ticket for this bug? If you have a support entitlement, you can submit your findings here:
http://www.splunk.com/r/bugs

Re: Bug in Splunk App for AWS: User unable to configure a CloudTrail input

connerswann — Tue, 28 Jun 2016 00:48:37 GMT

I haven't filed a ticket as I'm on a Developer License and do not have a support account. I figured this was the best place to post it given the circumstances.

Re: Bug in Splunk App for AWS: User unable to configure a CloudTrail input

pchen_splunk — Tue, 29 Sep 2020 10:03:28 GMT

Thanks for sharing this.

There is a bug fired from customer side (AWSAPP-1014). AWS app development team has resolved it and put in into v4.2.1 release.

The fix Conner provided works if the trail is configured recently, but not works for old trails.

If you want to apply the fix before v4.2.1 release, you can try fix one line code as below:

File: $AWS_APP_ROOT/bin/aws/aws_utils.py
Function: get_cloudtrail_sqs

Before fix: topic_names = set([x['SnsTopicName'] for x in trails])
After fix: topic_names = set([extract_name_from_arn(x['SnsTopicARN']) for x in trails if 'SnsTopicARN' in x])

Re: Bug in Splunk App for AWS: User unable to configure a CloudTrail input

connerswann — Tue, 28 Jun 2016 03:03:07 GMT

That's a better code snippet! Hope nobody else ran into this, took me forever to debug!

Re: Bug in Splunk App for AWS: User unable to configure a CloudTrail input

pchen_splunk — Tue, 28 Jun 2016 03:07:15 GMT

Please submit a support ticket to Splunk when you meet any issue in the future. Or, you can ask questions in this place, we will try our best to answer in the first time. Thanks for understanding.

Re: Bug in Splunk App for AWS: User unable to configure a CloudTrail input

dmckean — Tue, 30 Aug 2016 00:37:04 GMT

When will version 4.2.1 be released to SplunkCloud to fix this issue? It's a major blocker for adopting Splunk in my organization.

SplunkCloud is still using version 4.0.0, by the way.

Re: Bug in Splunk App for AWS: User unable to configure a CloudTrail input

pchen_splunk — Tue, 30 Aug 2016 01:41:03 GMT

AWS App v4.2.1 is release in July 2016. Please ask support to upgrade your app. Or, you can do upgrade by yourself in the app management page.