I'm trying to ingest Cloudfront logs from an AWS. Currently they are dumped to an S3 bucket in the form of a .gz file.
The only data I'm getting returned is:
4/4/16
9:05:23.000 AM
#Version: 1.0
#Fields: date time x-edge-location sc-bytes c-ip cs-method cs(Host) cs-uri-stem sc-status cs(Referer) cs(User-Agent) cs-uri-query cs(Cookie) x-edge-result-type x-edge-request-id x-host-header cs-protocol cs-bytes time-taken x-forwarded-for ssl-protocol ssl-cipher x-edge-response-result-type
host = **Redacted**
source = s3://mybucketname/EL90IIKCKI7FS.2016-04-04-12.9f3c0869.gz
sourcetype = aws:cloudfront:accesslog
This is the result of the input being pointed at the root of the bucket.
However, if I point the input directly at the .gz file within the bucket, it will ingest it and i can see my access logs. This won't work long term because the logfile rolls regularly and spawns a new one.
Is there something I'm missing? Splunk seems to be aware that the .gz file exists when pointed at the root of the bucket, but it doesn't seem to be ingesting the file.
Thanks!
... View more