AWS Addon for Splunk doesn't recognize config messages

[jarrell]

I'm not getting anything in my config input, even though I know events are logging. I started searching _internal and found that if I watch splunk_ta_aws.modinputs.config I can see the sqs processing. WHenever a messages pops up in the queue, splunk says it's not a config record. Example:

2020-03-22 19:06:17,736 level=WARNING pid=11190 tid=MainThread logger=splunk_ta_aws.modinputs.config pos=__init__.py:_stream_events:344 | datainput="Config_0bd7b668-1857-4466-9977-1595a2745bf3" | message="Invalid notifications have been removed from SQS : {"Records":[{"eventVersion":"2.1","eventSource":"aws:s3","awsRegion":"us-east-2","eventTime":"2020-03-22T19:06:16.660Z","eventName":"ObjectCreated:Put","userIdentity":{"principalId":"AWS:XXXXXXXXXXXXXX"},"requestParameters":{"sourceIPAddress":"172.18.xx.xxx"},"responseElements":{"x-amz-request-id":"03FF16BD00681075","x-amz-id-2":"redacted"},"s3":{"s3SchemaVersion":"1.0","configurationId":"Put config notice","bucket":{"name":"correct bucket name for config data","ownerIdentity":{"principalId":"XXXXXXXXXXXXX"},"arn":"arn:aws:s3:::correct bucket name for config data"},"object":{"key":"aws-config/AWSLogs/436617320021/Config/ConfigWritabilityCheckFile","size":0,"eTag":"d41d8cd98f00b204e9800998ecf8427e","versionId":"M5RCgK646j6EJPn5MGNeMYhbzRbh8Bge","sequencer":"005E77B7289FBE2097"}}}]}"

I'm not sure how to diagnose from here!

[jarrell · Accepted solution]

Ah. the input type of "Config" expects me to subscribe it to an sqs fed by an sns that gets real time changes to the config data. What set up was an SQS based S3 bucket, which is a different option when you create it.