Splunk Add-on for Amazon Web Services - Custom Data - SQS Based

[rkeq0515]

My ultimate goal is to process csv files directly from an S3 bucket/prefix for splunk to index. I have the Splunk Add-on for Amazon Web Services installed and was attempting to create a new input Custom Data Type using SQS-Based S3. I have a bucket created and I have an SQS created. As I am filling out the AWS Input Configuration info, everything is good until I select the SQS Queue Name. I can see the SQS queue that I created, but when I select it the "Save" button turns grey and I cannot save the configuration. Also, I can no longer select the drop down for the SQS Queue Name as my mouse turns to a red circle with a line through it. I have not been able to figure this out. Any help would be appreciated.

[venkatasri]

Hi @rkeq0515 

That means you do not have enough permissions towards AWS account to list SQS Queue and other operations advised in add-on docs. Did you check the splunkd and sourcetype=aws:sqs* logs which logs the AWS error behind UI.

[patilsonali1729]

Hello, 

Did you figure out the solution for this one. Facing the same issue.

[rkeq0515]

I never got the SQS to work as I had to move forward.  I ended up going with a direct read from the S3 bucket and object, which seemed to work pretty well so far. 

Here is a sample input.conf entry.  

[aws_s3://testtype]
aws_account = s3accountname
bucket_name = testbucket
character_set = auto
ct_blacklist = ^$
host_name = s3.amazonaws.com
index = testindex
initial_scan_datetime = 2020-03-01T15:43:47Z
interval = 30
is_secure = True
key_name = csvdata1/
max_items = 100000
max_retries = 3
polling_interval = 60
recursion_depth = -1
sourcetype = aws:s3:testsrctype