All Apps and Add-ons

Splunk App for AWS: Why does creating a S3 input with nested folder structure not return any results?

ytenenbaum_splu
Splunk Employee
Splunk Employee

A Splunk customer using Splunk App for AWS and they have a problem with the S3 input. They did a few tests to check that the S3 Input is ingesting data and it was able to collect files from a single bucket (.txt files) without any issues. However, when they create an S3 Input and target it to a bucket that contains nested folder structure they're not getting any results, the logs in the buckets do not have any extension as such. They have tried this with ELB Access logs and Cloudtrail logs.

0 Karma
1 Solution

ytenenbaum_splu
Splunk Employee
Splunk Employee

They've solved it by adding a Bucket policy that allows the role to do an S3:GetObject on the Bucket. They overlooked this originally and this is what caused the issue. It appears to be working fine now and they're getting logs coming in nicely.

View solution in original post

ytenenbaum_splu
Splunk Employee
Splunk Employee

They've solved it by adding a Bucket policy that allows the role to do an S3:GetObject on the Bucket. They overlooked this originally and this is what caused the issue. It appears to be working fine now and they're getting logs coming in nicely.

View solution in original post

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.