Getting Data In

Getting data in from S3 to Splunk through SQS based S3 method

spl_unker
Explorer

Hi Splunkers ,

 

We are collecting logs from multiple devices/application and sent to one single S3 bucket and they are separated into different folders inside.

We are trying to ingest data into Splunk using the AWS Add-on via SQS based S3 input type. SQS based S3  input type will have only queue name or queue URL  to be passed.

Since we have only one bucket(with many folders having data from different source which needs to be send to different index) .Challenge we face is for creating separate inputs for each folders in the bucket and send it to different index.

Since SQS can be subscribed to only at Bucket level and not folder level(to my knowledge). 

Is there any config settings available in the Splunk AWS addon like regex or keys to read only a specific folder or skip folders so that i can a map a folder to an index.

Thanks in Advance

 

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Splunk MCP & Agentic AI: Machine Data Without Limits

  Discover how the Splunk Model Context Protocol (MCP) Server can revolutionize the way your organization ...

Finding Based Detections General Availability

Overview  We’ve come a long way, folks, but here in Enterprise Security 8.4 I’m happy to announce Finding ...

Get Your Hands Dirty (and Your Shoes Comfy): The Splunk Experience

Hands-On Learning and Technical Seminars  Sometimes, you just need to see the code. For those looking for a ...