Splunk Search

what is the smart way to collect cloud watch logs in multiple S3 bucket?

syokota_splunk
Splunk Employee
Splunk Employee

Hi teachers,

I try to collect OS event logs in EC2 instance by using Cloudwatch logs, and archive into S3 Bucket by lambda.
S3 Bucket path is like a below image,

Amazon S3 bucket
- Server1_Linux_Secure_logs
- Server2_Linux_Secure_logs
- Server1_Windows_Security_Event_Logs
- FW_traffic_logs

I was considered that to use "Splunk Add-on for Amazon Web Services" setting.
And it seems Generic S3 input config can set sourcetype and S3 bucket path by each log source.
BTW SQS-based S3 is more scalable and recommend by admin guide.
https://docs.splunk.com/Documentation/AddOns/released/AWS/S3

But SQS-based S3 settings seems that it can not separate sourcetype and S3 bucket path.

What is the smart way to collect each data from each S3 bucket?

Thanks,
Satoshi

Tags (1)
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...