Splunk Search

what is the smart way to collect cloud watch logs in multiple S3 bucket?

syokota_splunk
Splunk Employee
Splunk Employee

Hi teachers,

I try to collect OS event logs in EC2 instance by using Cloudwatch logs, and archive into S3 Bucket by lambda.
S3 Bucket path is like a below image,

Amazon S3 bucket
- Server1_Linux_Secure_logs
- Server2_Linux_Secure_logs
- Server1_Windows_Security_Event_Logs
- FW_traffic_logs

I was considered that to use "Splunk Add-on for Amazon Web Services" setting.
And it seems Generic S3 input config can set sourcetype and S3 bucket path by each log source.
BTW SQS-based S3 is more scalable and recommend by admin guide.
https://docs.splunk.com/Documentation/AddOns/released/AWS/S3

But SQS-based S3 settings seems that it can not separate sourcetype and S3 bucket path.

What is the smart way to collect each data from each S3 bucket?

Thanks,
Satoshi

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...