Hello,
I am trying to settle on a new AWS event collection strategy. We are currently collecting using the older pull (SQS/SNS) method, and would like to move to a more modern and flexible way of doing it. Would like to collect AWS Config, CloudTrail, VPC Flow logs, CloudWatch, GuardDuty events from 100+ accounts into Splunk. Also would like a filtering capability at the source where logs can be discarded based on some criteria (account or arn) and not sent to Splunk to be filtered.
Seems like Splunk has changed their recommendations in the last few years (lambda push, firehouse, etc)_, and I am not certain what is the recommended approach now to do this now with as little complexity as possible. Project Trumpet seems like a good option, but I am not seeing Splunk steer people to that. Also unclear what caveats are with each of these approaches. If you go the Firehose route, how do you discard unwanted events? Also there are cost considerations and it's unclear which approach is more cost effective.
Wondering what people have settled on in similar circumstances and why. Thanks!
