topic Re: Unable to collect data from AWS SQS with Splunk Add-On for AWS in All Apps and Add-ons

Unable to collect data from AWS SQS with Splunk Add-On for AWS

david_emind — Mon, 28 Sep 2020 18:33:53 GMT

Hi All, Would appreciate some suggestions to a solution. Thanks!

I am unable to collect any data from AWS SQS. Brand new AWS Linux OS(yum update) with Splunk Enterprise 6.2.1 and Add-on(Version 1.0.1) and App(Version 3.0) for AWS installed. splunk user in IAM has full permissions to SQS and S3. SQS subscribed to SNS topic and is showing messages in the queue. In addition, there is a index that was manually created called aws-cloudtrail for which is required by SplunkAppforAWS.

**This is the output of my log file aws_cloudtrail.log

2015-01-03 10:09:28,865 INFO pid=30098 tid=MainThread file=aws_cloudtrail.py::413 | STARTED:
2015-01-03 10:09:28,865 DEBUG pid=30098 tid=MainThread file=aws_cloudtrail.py:stream_events:174 | Start streaming.
2015-01-03 10:09:28,865 DEBUG pid=30098 tid=MainThread file=aws_cloudtrail.py:stream_events:192 | blacklist regex for eventNames is None
2015-01-03 10:09:28,866 INFO pid=30098 tid=MainThread file=aws_cloudtrail.py:get_access_key_pwd_real:105 | get account name: splunk
2015-01-03 10:09:28,887 DEBUG pid=30098 tid=MainThread file=aws_cloudtrail.py:stream_events:206 | Connect to S3 & Sqs sucessfully
2015-01-03 10:09:28,981 CRITICAL pid=30098 tid=MainThread file=aws_cloudtrail.py:stream_events:282 | Outer catchall: ParseError: no element found: line 1, column 0
2015-01-03 10:09:28,982 INFO pid=30098 tid=MainThread file=aws_cloudtrail.py::415 | EXITED: 1

**I'm also seeing messages like this in the splunkd.log.
01-03-2015 09:17:11.556 +0000 WARN SearchOperator:inputcsv - Encountered 1 'inconsistent number of column' errors while reading input.
01-03-2015 09:18:28.428 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/aws_cloudtrail.py" ERRORno element found: line 1, column 0

Any clues why?

Re: Unable to collect data from AWS SQS with Splunk Add-On for AWS

jcoates_splunk — Sat, 03 Jan 2015 15:54:40 GMT

That usually indicates that it's pulling a message from SQS that isn't from CloudTrail. As of the latest 1.0.x it should write the message to a log and delete it, but if it doesn't have permission to delete it might get stuck on the same message.

Re: Unable to collect data from AWS SQS with Splunk Add-On for AWS

david_emind — Sat, 03 Jan 2015 18:34:35 GMT

This is a fresh install and I didn't expect this to happen. What do you think the solution could be? I can try it out and get back to you.
Thanks!

Re: Unable to collect data from AWS SQS with Splunk Add-On for AWS

jcoates_splunk — Sun, 04 Jan 2015 19:58:21 GMT

Can you try deleting the SQS message which isn't from CloudTrail?

Re: Unable to collect data from AWS SQS with Splunk Add-On for AWS

kkossery — Fri, 13 Feb 2015 14:27:17 GMT

Hello David - Were you able to find a solution to this? We see the exact problem you described.
I don't have any other SQS queue or SNS topic besides the one for CloudTrail.

Re: Unable to collect data from AWS SQS with Splunk Add-On for AWS

jcoates_splunk — Sun, 29 Mar 2015 00:13:37 GMT

with the current 1.1.0 version of the Add-on, it should log that it's seeing messages that aren't CloudTrail format and delete them from the queue so that it can proceed with the CloudTrail data.

Re: Unable to collect data from AWS SQS with Splunk Add-On for AWS

kkossery — Sun, 29 Mar 2015 17:54:51 GMT

Thanks! We were able to make it work earlier.