Getting Data In

Getting data in from S3 to Splunk through SQS based S3 method

spl_unker
Explorer

Hi Splunkers ,

 

We are collecting logs from multiple devices/application and sent to one single S3 bucket and they are separated into different folders inside.

We are trying to ingest data into Splunk using the AWS Add-on via SQS based S3 input type. SQS based S3  input type will have only queue name or queue URL  to be passed.

Since we have only one bucket(with many folders having data from different source which needs to be send to different index) .Challenge we face is for creating separate inputs for each folders in the bucket and send it to different index.

Since SQS can be subscribed to only at Bucket level and not folder level(to my knowledge). 

Is there any config settings available in the Splunk AWS addon like regex or keys to read only a specific folder or skip folders so that i can a map a folder to an index.

Thanks in Advance

 

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Unlock Database Monitoring with Splunk Observability Cloud

  In today’s fast-paced digital landscape, even minor database slowdowns can disrupt user experiences and ...

Purpose in Action: How Splunk Is Helping Power an Inclusive Future for All

At Cisco, purpose isn’t a tagline—it’s a commitment. Cisco’s FY25 Purpose Report outlines how the company is ...

[Upcoming Webinar] Demo Day: Transforming IT Operations with Splunk

Join us for a live Demo Day at the Cisco Store on January 21st 10:00am - 11:00am PST In the fast-paced world ...