Getting Data In

SQS-Based-S3 Input for Cloudtrail black list events

pstephens93
Explorer

I'm currently using SQS Based s3 input for cloudtrail and I'm trying to drop nosey events such as GET and LIST.
The documentation says the standard input supports exclude_describe_events and blacklist to filter out unwanted events which I set but after further looking into it it seems that a props/transform is needed. 

 

I have configured the following in props and transforms

props.conf 
[aws:cloudtrail]
TRANSFORMS-filter = eventsDrop

transforms.conf
#Filters out events that are not needed

[eventsDrop]
REGEX = "^Describe|Get|List\p{Lu}|LookupEvents"
DEST_KEY = queue
FORMAT = nullQueue                  

 

I tested the regex and it matches events but the events are not being dropped as expected. This is on a HF as that collects the logs before going into indexers 

Labels (3)
0 Karma

dwhitehead
New Member

I tried the same thing and found the same issue. I think the blacklist config is only compatible with the cloudtrail input NOT the sqs_based_s3 input. Really unfortunate as I wanted to switch to role based cloudtrail logging rather than aws account. Please put this on your bug litst Splunk.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Thanks for the Memories: .conf26 Took Learning to New Heights

Thank you, Splunk Community, for making .conf26 in Denver one for the books. From packed Splunk University ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...