Getting Data In

AWS CloudTrail SQS Based S3 error?

leuorrouel
Loves-to-Learn

Hi,

I tried to configure CloudTrail SQS Based S3 and I got the following message:

"Warning: This message does not have a valid SNS Signature None None doesn't match required format '^https://sns\\.[-a-z0-9]+\\.amazonaws\\.com(?:\\.cn)?/'"

Sometimes I also get:

"Failed to delete message"

I have no clue where to look in order to solve this issue. I will appreciate any help!

Labels (1)
Tags (1)
0 Karma

danint
Engager

Haven't looked into this too much, but I'm guessing it is related to the updates in version 5.2.1:

https://docs.splunk.com/Documentation/AddOns/released/AWS/Releasenotes

"""

Version 5.2.1 of the Splunk Add-on for AWS version contains the following new and changed features:

  • Added validation of the signature of the SNS message being sent from SQS queue to Splunk. The source of the logs is validated by matching the signature of the SNS message with the signature field.

"""

The setup docs for the CloudTrail input state S3->SNS->SQS now. I don't think this was the case before, but I don't have a copy of the old docs to check.

It looks like it is using this plugin to validate. https://pypi.org/project/validate-aws-sns-message/  Note, it "Requires message be no older than one hour, the maximum lifetime of an SNS message."

 

Splunk, it would be much better if this was an optional feature, as it breaks our infrastructure, especially bad since this is a PATCH update, not MAJOR.

0 Karma

leuorrouel
Loves-to-Learn

I checked Splunkbase, but they do not have version 5.2.1. I know they had this before, but for some reason they reverted it back to version 5.2.0

0 Karma

parcflyer
Loves-to-Learn Lots

We backed out of the 5.2.1 update.  5.2.0  seems to be working.

 

0 Karma

tlanghals_il
Engager

Same issue for ALB access log collection using the SQS based S3 input after upgrading to 5.2.1.  Not only is this not indexing the S3 logs, but it's also deleting the SQS messages so it's not appropriately handling the error.

0 Karma

parcflyer
Loves-to-Learn Lots

Having the same issues in the cloud.  Was upgraded to 5.2.1 last night and all inputs are doing the exact same thing.

Pulling data from the Que,  warning, not indexing and the Que are cleared.

 

 

Tags (1)
0 Karma

splunkoptimus
Path Finder

Hello I'm having the same error, did you manage to fixing it?

 

2023-03-15 05:26:05,915 level=WARNING pid=2768 tid=Thread-7 logger=splunk_ta_aws.modinputs.sqs_based_s3.handler pos=handler.py:_process:390 | datainput="XXXXXX" start_time=1678855832, message_id=XXXXXXXXXXXXXXX ttl=300 job_id=XXXXXXXXXXXXX | message="Warning: This message does not have a valid SNS Signature Message is too old: 2 days, 16:30:10.556919"

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...