Im not sure how to extract the x-forwaded-for out of these cloudflare logs. I would like to extract if there and take first ip and use as client.ip. Data has been scrubbed...sometimes there are 4-6 ip's in the x-forwarded-for field. This is the actual field..but included entire log for example.
headers": [{"name": "x-forwarded-for", "value": “1.1.153.26, 1.1.153.26, 2.2.2.29"}],
{"edgeResponse": {"bytes": 2984, "contentType": "text/html", "status": 200, "bodyBytes": 2373, "setCookies": null, "headers": null, "compressionRatio": 3.7}, "zonePlan": "enterprise", "origin": {"asNum": 2211123, "sslCipher": "UNK", "railgunWanError": "", "cfRailgun": "", "port": 443, "responseTime": 0, "ip": "1.1.5.xx", "sslProtocol": "unknown"}, "ownerId": 2211222, "originResponse": {"bytes": 0, "flags": 0, "httpExpires": 0, "status": 200, "bodyBytes": 0, "httpLastModified": 0, "headers": []}, "edgeRequest": {"bytes": 1560, "uri": "/", "httpHost": "mywebsite.sample.com", "httpMethod": "GET", "bodyBytes": 0, "headers": null, "keepaliveStatus": "reuseAccepted"}, "brandId": 1, "unstablePublic": null, "edge": {"pathingStatus": "nr", "cacheResponseTime": 160000000, "flServerIp": "1.17.xx.xx", "bbResult": "0", "endTimestamp": 1504663106348999936, "flServerPort": 443, "dnsResponse": {"error": "ok", "duration": 8999824, "rcode": 0, "cached": false, "overrideError": false, "errorMsg": ""}, "startTimestamp": 1504663106176000000, "enabledFlags": 0, "pathingSrc": "macro", "flServerName": "12h4210", "colo": 18, "rateLimit": {"mitigationId": null, "ruleId": 0, "sourceId": "", "processedRules": null}, "usedFlags": 0, "pathingOp": "wl"}, "client": {"srcPort": 59563, "asNum": 20940, "sslCipher": "ECDHE-ECDSA-AES128-GCM-SHA256", "sslProtocol": "TLSv1.2", "sslFlags": 1, "deviceType": "desktop", "ipClass": "noRecord", "country": "us", "ip": “1.26.2.5”}, "flags": 2, "clientRequest": {"flags": 1, "uri": "/", "bodyBytes": 0, "accept": "", "httpMethod": "GET", "bytes": 832, "httpHost": "mywebsite.sample.com", "httpProtocol": "HTTP/1.1", "userAgent": “curl)”, "referer": "", "body": null, "headers": [{"name": "x-forwarded-for", "value": “1.1.153.26, 1.1.153.26, 2.2.2.29"}], "cookies": null}, "rayId": "3443333222222222”, "zoneName": "sec.myzone.com", "securityLevel": "med", "cache": {"cacheServerName": "12c53", "cacheExternalPort": 23271, "cacheStatus": "unknown", "cacheInternalIp": "1.12.2.1”, "cacheFileKey": null, "cacheExternalIp": "1.1.1.22", "startTimestamp": 1504663106190000128, "cacheTokens": 0, "endTimestamp": 1504663106348999936, "bckType": "nc"}, "zoneId": 4533333322, "hosterId": 100, "timestamp": 1504663106176000000, "cacheResponse": {"bodyBytes": 0, "bytes": 8719, "contentType": "text/html", "status": 200, "retriedStatus": 0}, "cacheRequest": {"headers": null, "keepaliveStatus": "noReuseAccepted"}}
Try:
... | rex "x-forwarded-for\", \"value\": \"?P<client_ip>\d+\.\d+\.\d+\.\d+)"
Hopefully I got the TYPE of double quote right through that. I originally copied and pasted the string, then noticed that one of the appeared to be wrong. If you have problems, check for errors there.
Hello rgraham4,
How did you manage to pull the Cloudflare ELS logs to Splunk?
Best Regards,
Presciliano
Try:
... | rex "x-forwarded-for\", \"value\": \"?P<client_ip>\d+\.\d+\.\d+\.\d+)"
Hopefully I got the TYPE of double quote right through that. I originally copied and pasted the string, then noticed that one of the appeared to be wrong. If you have problems, check for errors there.
Thanks that did it! Just had to add the "(" in front of quote for field.