Reading log files from Amazon S3

[grahampoulter]

We're considering Amazon Elastic Beanstalk. The application logs will get rotated to S3 buckets, so I'm asking if there is a Splunk app or other well-supported method to read logs from S3 into Splunk.

[khourihan_splun · Accepted solution]

You can try our S3 app here: http://apps.splunk.com/app/1137/

Or you can mount your S3 as a filesystem and have a Universal Forwarder monitor it like a directory. There is a good writeup here: http://www.reedmurphy.net/blog/post/splunking-through-amazon-s3-access-logs

I suggest going the s3cmd route, its a little more feature rich than the Splunk app, but I have used both with success.