We currently are using SQS-Based S3 data input to get our cloudtrail data in splunk enterprise. It works fine. The only issue is that it cannot, like the cloudtrail data input, filter out by either selecting to drop read-only events or use blacklists. I have tried switch to using the cloudtrail data input but when I do there seems to be an issue getting data from the S3 bucket that I can't identify. The SQS queue is getting cleared out but I am getting no data from the S3 bucket. This is running with no filtering turned on. I have unchecked the default option to filter read-only events.
I see log entries like below in the data input log but no data makes it to splunk. If I switch back to the SQS-Based S3 data input it works fine
2019-03-27 14:07:12,502 level=INFO pid=2539 tid=Thread-4 logger=splunk_ta_aws.modinputs.cloudtrail.aws_cloudtrail_data_loader pos=aws_cloudtrail_sqs_collector.py:_collect:132 | | message="Pulled 1195 messages from SQS"
Not sure where to look to trace the issue
