Since AWS CloudFront access logs use W3C extended log file format, the following works fine when ingesting them as raw files:
props.conf
[cloudfront-access]
INDEXED_EXTRACTIONS = W3C
This enables automatic header-based field extractions by Splunk at index-time.
Keep in mind this means Splunk will store all these fields in the index, thereby increasing the size of the searchable index.
However:
As of current Splunk 6.3 release, the above header-based indexed extractions is not supported with modular inputs (vs one shot or monitored files). Unfortunately, importing logs from S3 is commonly done via modular input. So I found the following configs for search-time extractions to be a reasonable alternative (with one caveat below):
props.conf:
[cloudfront-access-modinput]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRANSFORMS-1trash = filter_preamble
REPORT-1 = cloudfront_fields
transforms.conf:
# Field Extractions for CloudFront Access Logs (for modular input only; otherwise use w3c)
[cloudfront_fields]
DELIMS = "\t"
FIELDS = date,time,x-edge-location,sc-bytes,c-ip,cs-method,cs-host,cs-uri-stem,sc-status,cs-referer,cs-user-agent,cs-uri-query,cs-cookie,x-edge-result-type,x-edge-request-id,x-host-header,cs-protocol,cs-bytes,time-taken,x-forwarded-for,ssl-protocol,ssl-cipher,x-edge,response-result-type
[filter_preamble]
DEST_KEY = queue
REGEX = (?m)^\#(Version|Fields)
FORMAT = nullQueue
Caveat: the above hardcoded list of fields is based on the current CloudFront file format version 1.0, and may change the future. Something you would not have to worry about if you're using INDEXED_EXTRACTIONS = W3C
