Splunk Search

what is the smart way to collect cloud watch logs in multiple S3 bucket?

syokota_splunk
Splunk Employee
Splunk Employee

Hi teachers,

I try to collect OS event logs in EC2 instance by using Cloudwatch logs, and archive into S3 Bucket by lambda.
S3 Bucket path is like a below image,

Amazon S3 bucket
- Server1_Linux_Secure_logs
- Server2_Linux_Secure_logs
- Server1_Windows_Security_Event_Logs
- FW_traffic_logs

I was considered that to use "Splunk Add-on for Amazon Web Services" setting.
And it seems Generic S3 input config can set sourcetype and S3 bucket path by each log source.
BTW SQS-based S3 is more scalable and recommend by admin guide.
https://docs.splunk.com/Documentation/AddOns/released/AWS/S3

But SQS-based S3 settings seems that it can not separate sourcetype and S3 bucket path.

What is the smart way to collect each data from each S3 bucket?

Thanks,
Satoshi

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...