Getting Data In

Getting data in from S3 to Splunk through SQS based S3 method


Hi Splunkers ,


We are collecting logs from multiple devices/application and sent to one single S3 bucket and they are separated into different folders inside.

We are trying to ingest data into Splunk using the AWS Add-on via SQS based S3 input type. SQS based S3  input type will have only queue name or queue URL  to be passed.

Since we have only one bucket(with many folders having data from different source which needs to be send to different index) .Challenge we face is for creating separate inputs for each folders in the bucket and send it to different index.

Since SQS can be subscribed to only at Bucket level and not folder level(to my knowledge). 

Is there any config settings available in the Splunk AWS addon like regex or keys to read only a specific folder or skip folders so that i can a map a folder to an index.

Thanks in Advance


Labels (1)
0 Karma