Splunk App for AWS: S3 Event Notification options

[aknsun]

Hi,
Just wondering if using the 2nd method has any drawbacks:

CloudTrail -> S3 -> SQS -> AWS Add-On

CloudTrail -> S3 -> SNS -> SQS -> AWS Add-On

Regards,

AKN

[mccartneyc]

Would like to know this as well. Documentation seems a bit spotty as to why we should use Cloudtrail -> S3 -> SNS -> SQS -> Addon

I'm currently having S3 send event notifications to SQS which in turn goes to the Addon and is working, but I'm trying to figure out why I should use SNS, is there a benefit or downside, or even a reason to use it with cloudtrail.

[splunk_zen]

It's not true the documentation doesn't explain the advantages.
"Each incremental S3 input is a single point of failure.

(...)

"Any SQS message not successfully processed in time by the SQS-based S3 input will reappear in the queue and will be retrieved and processed again.
In addition, data collection can be horizontally scaled out so that if one SQS-based S3 input fails, other inputs can still continue to pick up messages from the SQS queue and ingest corresponding data from the S3 bucket.
"
https://docs.splunk.com/Documentation/AddOns/released/AWS/ConfigureInputs