I need to obtain information from multiple S3 buckets since each is related to a different AWS account. Is it possible to include multiple AWS accounts in the aws.yaml file? if so - is there a specific format? I tried the following format - only the details for the account details for # 2 is getting processed. Any ideas?
s3
account details #1
s3
account details # 2
Hi, I have released a new version that takes multiple accounts , you can download it here:
https://splunkbase.splunk.com/app/1577/#/overview (version 2.0.0)
RELEASE NOTES
- Supports multiple accounts
- events are streamed in json
- file positions are tracked using yaml files, to reduce the likelihood of double ups
- all functions have been moved to classes
- boto has been updated to 2.38.0
Hi, I have released a new version that takes multiple accounts , you can download it here:
https://splunkbase.splunk.com/app/1577/#/overview (version 2.0.0)
RELEASE NOTES
- Supports multiple accounts
- events are streamed in json
- file positions are tracked using yaml files, to reduce the likelihood of double ups
- all functions have been moved to classes
- boto has been updated to 2.38.0
The format is now:
For a single account use the following style of aws.yaml:
accounts:
- account_number : 123456
billing_bucket : company-billing
aws_access_key : AAAAAAAAAAAA
aws_secret_key : AAAAAAAAAAAABBBBBBBBBBBBBBCC
For multiple accounts use the following style of aws.yaml:
accounts:
- account_number : 123456
billing_bucket : company-one-billing
aws_access_key : AAAAAAAAAAAA
aws_secret_key : AAAAAAAAAAAABBBBBBBBBBBBBBCC
- account_number : 654321
billing_bucket : company-two-billing
aws_access_key : AAAAAAAAAAAA
aws_secret_key : AAAAAAAAAAAABBBBBBBBBBBBBBCC
Hi gbacs, I am rewriting the app to handle this. There is some basic support for this but it is not sufficient and can cause duplicate data. I will let you know when i have released the new version - it is not too far away.
Thanks. gbacs
Hi there,
Any idea when the new version will be ready? Thanks
It is complete and you can download it from here https://splunkbase.splunk.com/app/1577/#/overview