I have events like so:
{"action": {"result": true, "type": "login"}, "actor": {"email": "[email protected]", "id": "0123456789abcdef0123456789abcdef", "ip": "1.2.3.4", "type": "user"}, "id": "01234567-89ab-cdef-0123-456789abcdef", "newValue": "audit", "oldValue": "review", "owner": {"id": "fedcba9876543210fedcba9876543210"}, "when": "2023-04-21T18:52:32Z", "account_name": "test_account"}
The props.conf file is as so:
[cloudflare_audit]
NO_BINARY_CHECK=true
INDEXED_EXTRACTIONS=JSON
TIMESTAMP_FIELDS=when
disabled=false
pulldown_type=true
When I do this, I wind up with two records per event, split at that TIME_PREFIX setting, each record with the time found in "when".
Things I've tried so far, based on the above:
- Adding "KV_MODE=none" -- The event is parsed as JSON, but the time is ignored
- Adding "TIME_PREFIX=when": "" and LINE_BREAKER=}$ -- The event is split on "when", again
- Removing "INDEXED_EXTRACTIONS=true" and adding "AUTO_KV_JSON=true" -- The event is parsed as JSON, but the time is ignored
Two questions:
- How can I fix this so that it pulls in the timefield correctly, without any splitting of the JSON object?
- Why is it so difficult to ingest JSON logs?
