Getting Data In

Best AWS ingestion approach



I am trying to settle on a new AWS event collection strategy.  We are currently collecting using the older pull (SQS/SNS) method, and would like to move to a more modern and flexible way of doing it.  Would like to collect AWS Config, CloudTrail, VPC Flow logs, CloudWatch, GuardDuty events from 100+ accounts into Splunk.  Also would like a filtering capability at the source where logs can be discarded based on some criteria (account or arn) and not sent to Splunk to be filtered.   

Seems like Splunk has changed their recommendations in the last few years (lambda push, firehouse, etc)_, and I am not certain what is the recommended approach now to do this now with as little complexity as possible.  Project Trumpet seems like a good option, but  I am not seeing Splunk steer people to that.   Also unclear what caveats are with each of these approaches.  If you go the Firehose route, how do you discard unwanted events?  Also there are cost considerations and it's unclear which approach is more cost effective.

Wondering what people have settled on in similar circumstances and why.  Thanks!

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...