How to ingest files in S3 buckets that are compressed but do not have .gz extension:
ie: s3://Blah-main/2018/04/20/16/FlowLogsLogsFirehose-us-east-1-DeliveryStream-1THLU7DASFP74-6-2018-04-20-16-10-40-68617538
Did you ever come to a solution on this?
I'm also trying to read in an S3 bucket that contains VPC Flow logs. The Splunk App for AWS doesn't seem to realize that they're GZipped data, and it loads in a bunch of garbage text into Splunk instead.
Did you find any solution for this (VPC Flow logs)?
Try to use splunk app for AWS. Refer :
https://docs.splunk.com/Documentation/AWS/5.1.0/Installation/S3
Already using a Splunk App for AWS.