AWS CloudTrail SQS Based S3 error?

[leuorrouel]

Hi,

I tried to configure CloudTrail SQS Based S3 and I got the following message:

"Warning: This message does not have a valid SNS Signature None None doesn't match required format '^https://sns\\.[-a-z0-9]+\\.amazonaws\\.com(?:\\.cn)?/'"

Sometimes I also get:

"Failed to delete message"

I have no clue where to look in order to solve this issue. I will appreciate any help!

[danint]

Haven't looked into this too much, but I'm guessing it is related to the updates in version 5.2.1:

https://docs.splunk.com/Documentation/AddOns/released/AWS/Releasenotes

"""

Version 5.2.1 of the Splunk Add-on for AWS version contains the following new and changed features:

• Added validation of the signature of the SNS message being sent from SQS queue to Splunk. The source of the logs is validated by matching the signature of the SNS message with the signature field.

"""

The setup docs for the CloudTrail input state S3->SNS->SQS now. I don't think this was the case before, but I don't have a copy of the old docs to check.

It looks like it is using this plugin to validate. https://pypi.org/project/validate-aws-sns-message/  Note, it "Requires message be no older than one hour, the maximum lifetime of an SNS message."

Splunk, it would be much better if this was an optional feature, as it breaks our infrastructure, especially bad since this is a PATCH update, not MAJOR.

[leuorrouel]

I checked Splunkbase, but they do not have version 5.2.1. I know they had this before, but for some reason they reverted it back to version 5.2.0

[parcflyer]

We backed out of the 5.2.1 update.  5.2.0  seems to be working.

[tlanghals_il]

Same issue for ALB access log collection using the SQS based S3 input after upgrading to 5.2.1.  Not only is this not indexing the S3 logs, but it's also deleting the SQS messages so it's not appropriately handling the error.

[parcflyer]

Having the same issues in the cloud.  Was upgraded to 5.2.1 last night and all inputs are doing the exact same thing.

Pulling data from the Que,  warning, not indexing and the Que are cleared.

[splunkoptimus]

Hello I'm having the same error, did you manage to fixing it?

2023-03-15 05:26:05,915 level=WARNING pid=2768 tid=Thread-7 logger=splunk_ta_aws.modinputs.sqs_based_s3.handler pos=handler.py:_process:390 | datainput="XXXXXX" start_time=1678855832, message_id=XXXXXXXXXXXXXXX ttl=300 job_id=XXXXXXXXXXXXX | message="Warning: This message does not have a valid SNS Signature Message is too old: 2 days, 16:30:10.556919"