Getting Data In

Getting data in from S3 to Splunk through SQS based S3 method

spl_unker
Explorer

Hi Splunkers ,

 

We are collecting logs from multiple devices/application and sent to one single S3 bucket and they are separated into different folders inside.

We are trying to ingest data into Splunk using the AWS Add-on via SQS based S3 input type. SQS based S3  input type will have only queue name or queue URL  to be passed.

Since we have only one bucket(with many folders having data from different source which needs to be send to different index) .Challenge we face is for creating separate inputs for each folders in the bucket and send it to different index.

Since SQS can be subscribed to only at Bucket level and not folder level(to my knowledge). 

Is there any config settings available in the Splunk AWS addon like regex or keys to read only a specific folder or skip folders so that i can a map a folder to an index.

Thanks in Advance

 

Labels (1)
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

How to Break a Kubernetes Pod on Purpose and Let the AI Troubleshooting Agent Find ...

Let's say I've just rolled out a Kubernetes Deployment with a typo in the image tag. Nothing explodes. The pod ...

Splunk Add-on for Microsoft Windows 11.0.3: Parse Once, Search Faster

The updated Splunk Add-on for Microsoft Windows helps customers search Windows events faster by reducing ...

What's New in Splunk Enterprise Security 8.8 and Beyond

Threats move at machine speed. Keeping up should not depend on having the biggest budget, the largest team, or ...