All Apps and Add-ons

How to configure the Splunk App for AWS and Splunk Add-on for Amazon Web Services to ingest AWS Config logs?

rarneson
Explorer

Hi,

I'm running the latest Splunk App for AWS (3.0) and Splunk Add-on for Amazon Webservices (1.01).

I'm trying to set it up to ingest my AWS Config logs. There doesn't seem to be a specific "AWS Config" data input, so I'm using the CloudTrail input and choosing my AWS Config SQS Queue. I created an aws-config index and am tagging the events with aws:config.

Here's the errors I'm seeing in /opt/splunk/var/log/splunk/aws_cloudtrail.log

2015-02-11 16:48:50,566 ERROR pid=6635 tid=MainThread file=aws_cloudtrail.py:process_notifications:351 | problems unzipping from s3:config-splunk//: IOError Not a gzipped file
2015-02-11 16:48:50,566 CRITICAL pid=6635 tid=MainThread file=aws_cloudtrail.py:stream_events:286 | Outer catchall: KeyError: 's3Bucket'

I'm a bit stuck as to what the problem is. I've tried two different AWS accounts, one using the root keys. I see my Config > SNS > SQS > S3 data looking correct in AWS. I can even download some of the files from the S3 bucket onto the Splunk server and run file and/or gunzip on it, and everything checks out fine for being a valid gzip with normal json data.

Anyone have any ideas?

Forgot to mention, the ultimate error is nothing is being ingested via the app into my index/populating the dashboards/etc...

0 Karma
1 Solution

_d_
Splunk Employee
Splunk Employee

Couple of things:
- You're using the wrong Add-on version. You'll need 1.1.0 instead of 1.0.1. https://apps.splunk.com/app/1274/#/documentation
- You will need to use the AWS Config input instead of the AWS CloudTrail input to collect Config data.

Update the Add-on to 1.1.0 and you should be fine.

View solution in original post

_d_
Splunk Employee
Splunk Employee

Couple of things:
- You're using the wrong Add-on version. You'll need 1.1.0 instead of 1.0.1. https://apps.splunk.com/app/1274/#/documentation
- You will need to use the AWS Config input instead of the AWS CloudTrail input to collect Config data.

Update the Add-on to 1.1.0 and you should be fine.

rarneson
Explorer

Thank you! I have no idea how I missed that, even checking for it and thinking I was on the latest version, but going back to Manage Apps I saw an upgrade option there.

That was 100% the problem.

Much appreciated, all is working

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

Don't feel bad, you were looking while we were updating 🙂

0 Karma
Get Updates on the Splunk Community!

Splunk AI Assistant for SPL | Key Use Cases to Unlock the Power of SPL

Splunk AI Assistant for SPL | Key Use Cases to Unlock the Power of SPL  The Splunk AI Assistant for SPL ...

Buttercup Games: Further Dashboarding Techniques (Part 5)

This series of blogs assumes you have already completed the Splunk Enterprise Search Tutorial as it uses the ...

Customers Increasingly Choose Splunk for Observability

For the second year in a row, Splunk was recognized as a Leader in the 2024 Gartner® Magic Quadrant™ for ...