Setting up SQS based S3 input!

[Stokers_23]

Hi

I am running an splunk instance within my AWS account, and i'm trying to setup an Cloudtrail SQS based S3 imput. The cloud trail logs are stored in a bucket (auditlogs) in separate account, which I access via a switch role.

I have done the following however no data appears in index I have selected

• Created an IAM policy with the required permissions
• Created the required SQS Queue, granting permissions to the auditlogs bucket to post events.
• Added an event notification on the S3 bucket to forward 'Object-created' events to my SQS Queue
• Confirmed that the SQS Queue is receiving messages
• Added a new input within the AWS Add on for splunk web, using my auto discovered IAM role
• Requested for the input sends data to my aduit index.
• Checked the logs on the splunk instance and found no errors, other issues.

Questions
- The documentation seems very unclear on the need to have an SNS topic in the middle here? Is it a requirement that SQS is updated via a subscription to an SNS topic. Specifically S3 > SNS > SQS > Splunk? Or would S3 > SQS > Splunk also work?

• My auto discovered IAM role applied to the splunk EC2 instance is in a separate account to the S3 bucket i'm trying to import data from. Is this going to cause me issues - I assume this is the issue, but there

I would appreciate any guidance here!

Thanks

[wagnerlucena]

You need to follow the second option.

2> aws cloudtrail/config > S3 bucket > Event notification trigger as SNS > SQS subscription > Splunk

Enable the cloudtrail sending notification to SNS

Create SQS and subscribe the SNS to it

In splunk, create the cloudtrail input and choose the SQS that was created in the last step.

[wagnerlucena]

don't forget upvote if the answer helped you.

[meet_vadaria]

I am also confused at this. Document is very unclear. I tried to google it. went through few blogs and videos. All are different. there are 3 ways to do this as far as I know,
1> aws cloudtrail/config > SNS notification enabled at source with S3 bucket > SQS subscription > Splunk (in this case there is a no event notification trigger in S3 bucket)

2> aws cloudtrail/config > S3 bucket > Event notification trigger as SNS > SQS subscription > Splunk

3> aws cloudtrail/config > S3 bucket > Event notification trigger as SQS > Splunk

I am confused which on to follow. So far, I have tried 3rd option. it's working but I am seeing couple of errors "Unable to parse message" from both aws cloudtrail and aws config.

Will be great if someone can explain!

[vgollapudi]

The option 2 is the best one to go for.