All Apps and Add-ons

Splunk App for AWS: S3 Event Notification options

aknsun
Path Finder

Hi,
Just wondering if using the 2nd method has any drawbacks:

CloudTrail -> S3 -> SQS -> AWS Add-On

CloudTrail -> S3 -> SNS -> SQS -> AWS Add-On

Regards,

AKN

mccartneyc
Path Finder

Would like to know this as well. Documentation seems a bit spotty as to why we should use Cloudtrail -> S3 -> SNS -> SQS -> Addon

I'm currently having S3 send event notifications to SQS which in turn goes to the Addon and is working, but I'm trying to figure out why I should use SNS, is there a benefit or downside, or even a reason to use it with cloudtrail.

0 Karma

splunk_zen
Builder

It's not true the documentation doesn't explain the advantages.
"Each incremental S3 input is a single point of failure.

(...)

"Any SQS message not successfully processed in time by the SQS-based S3 input will reappear in the queue and will be retrieved and processed again.
In addition, data collection can be horizontally scaled out so that if one SQS-based S3 input fails, other inputs can still continue to pick up messages from the SQS queue and ingest corresponding data from the S3 bucket.
"
https://docs.splunk.com/Documentation/AddOns/released/AWS/ConfigureInputs

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...