All Apps and Add-ons

Splunk App for AWS: S3 Event Notification options

Path Finder

Hi,
Just wondering if using the 2nd method has any drawbacks:

CloudTrail -> S3 -> SQS -> AWS Add-On

CloudTrail -> S3 -> SNS -> SQS -> AWS Add-On

Regards,

AKN

Path Finder

Would like to know this as well. Documentation seems a bit spotty as to why we should use Cloudtrail -> S3 -> SNS -> SQS -> Addon

I'm currently having S3 send event notifications to SQS which in turn goes to the Addon and is working, but I'm trying to figure out why I should use SNS, is there a benefit or downside, or even a reason to use it with cloudtrail.

0 Karma

Builder

It's not true the documentation doesn't explain the advantages.
"Each incremental S3 input is a single point of failure.

(...)

"Any SQS message not successfully processed in time by the SQS-based S3 input will reappear in the queue and will be retrieved and processed again.
In addition, data collection can be horizontally scaled out so that if one SQS-based S3 input fails, other inputs can still continue to pick up messages from the SQS queue and ingest corresponding data from the S3 bucket.
"
https://docs.splunk.com/Documentation/AddOns/released/AWS/ConfigureInputs

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!