Getting Data In

AWS CloudTrail SQS Based S3 error

leuorrouel
Loves-to-Learn

Hi,

I tried to configure CloudTrail SQS Based S3 and I got the following message:

"Warning: This message does not have a valid SNS Signature None None doesn't match required format '^https://sns\\.[-a-z0-9]+\\.amazonaws\\.com(?:\\.cn)?/'"

Sometimes I also get:

"Failed to delete message"

I have no clue where to look in order to solve this issue. I will appreciate any help!

Labels (1)
Tags (1)
0 Karma

danint
Engager

Haven't looked into this too much, but I'm guessing it is related to the updates in version 5.2.1:

https://docs.splunk.com/Documentation/AddOns/released/AWS/Releasenotes

"""

Version 5.2.1 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Added validation of the signature of the SNS message being sent from SQS queue to Splunk. The source of the logs is validated by matching the signature of the SNS message with the signature field.

"""

The setup docs for the CloudTrail input state S3->SNS->SQS now. I don't think this was the case before, but I don't have a copy of the old docs to check.

It looks like it is using this plugin to validate. https://pypi.org/project/validate-aws-sns-message/  Note, it "Requires message be no older than one hour, the maximum lifetime of an SNS message."

 

Splunk, it would be much better if this was an optional feature, as it breaks our infrastructure, especially bad since this is a PATCH update, not MAJOR.

0 Karma

leuorrouel
Loves-to-Learn

I checked Splunkbase, but they do not have version 5.2.1. I know they had this before, but for some reason they reverted it back to version 5.2.0

0 Karma

parcflyer
Loves-to-Learn Lots

We backed out of the 5.2.1 update.  5.2.0  seems to be working.

 

0 Karma

tlanghals_il
Engager

Same issue for ALB access log collection using the SQS based S3 input after upgrading to 5.2.1.  Not only is this not indexing the S3 logs, but it's also deleting the SQS messages so it's not appropriately handling the error.

0 Karma

parcflyer
Loves-to-Learn Lots

Having the same issues in the cloud.  Was upgraded to 5.2.1 last night and all inputs are doing the exact same thing.

Pulling data from the Que,  warning, not indexing and the Que are cleared.

 

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...