All Apps and Add-ons

Can't get files from S3 to SNS to SQS using Splunk app for AWS

avery2007
Explorer

I am using the Splunk app for AWS and utilizing the Custom Data Type Input, SQS Based S3 to pull in reports in xml format that are being dropped off in an S3 bucket. I setup a notification for the S3 to send a message to SNS which goes to SQS. Simple text files seem to work fine when PUTing them into the S3 bucket, however the reports I'm receiving are anywhere from 5 - 25MBs big so it goes over the SQS message size limit and I am guessing this is why I cannot receive these reports. I cannot find anywhere in the documentation that goes over what to do in this type of situation. There seems to be no errors in Cloudtrail or the splunkd.log either so I have no way to trace where it is failing.

Any ideas would be of great help.

0 Karma
Get Updates on the Splunk Community!

Unlock New Opportunities with Splunk Education: Explore Our Latest Courses!

At Splunk Education, we’re dedicated to providing top-tier learning experiences that cater to every skill ...

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...