Getting data in from S3 to Splunk through SQS bas...

Getting Data In

Hi Splunkers ,

We are collecting logs from multiple devices/application and sent to one single S3 bucket and they are separated into different folders inside.

We are trying to ingest data into Splunk using the AWS Add-on via SQS based S3 input type. SQS based S3 input type will have only queue name or queue URL to be passed.

Since we have only one bucket(with many folders having data from different source which needs to be send to different index) .Challenge we face is for creating separate inputs for each folders in the bucket and send it to different index.

Since SQS can be subscribed to only at Bucket level and not folder level(to my knowledge).

Is there any config settings available in the Splunk AWS addon like regex or keys to read only a specific folder or skip folders so that i can a map a folder to an index.

Thanks in Advance

Get Updates on the Splunk Community!

Getting data in from S3 to Splunk through SQS based S3 method

HTTP Event Collector

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Announcing Modern Navigation: A New Era of Splunk User Experience

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

Step into “Hunt the Insider: An Splunk ES Premier Mystery” to catch a cybercriminal ...

Join the Conversation