Valid JSON not being broken up into individual events

[BongoTheWhippet]

I've checked a number of threads about breaking JSON files and I've tried a number of offered solutions and none seem to work.

I'm running 8.1.0 and I don't remember seeing this as much of an issue in previous versions.

The snort (ids-u2json) JSON is lint-valid as follows:

{"type": "event", "event": {"msg": "ET INFO Microsoft Connection Test", "classification": "Potentially Bad Traffic", "sensor-id": 0, "event-id": 581, "event-second": 1607588446, "event-microsecond": 790456, "signature-id": 2031071, "generator-id": 1, "signature-revision": 2, "classification-id": 3, "priority": 2, "sport-itype": 63591, "dport-icode": 80, "protocol": 6, "impact-flag": 0, "impact": 0, "blocked": 0, "mpls-label": null, "vlan-id": null, "pad2": null, "source-ip": "192.168.1.125", "destination-ip": "13.107.4.52"}}
{"type": "event", "event": {"msg": "ET POLICY PE EXE or DLL Windows file download HTTP", "classification": "Potential Corporate Privacy Violation", "sensor-id": 0, "event-id": 582, "event-second": 1607588467, "event-microsecond": 769440, "signature-id": 2018959, "generator-id": 1, "signature-revision": 4, "classification-id": 33, "priority": 1, "sport-itype": 80, "dport-icode": 63676, "protocol": 6, "impact-flag": 0, "impact": 0, "blocked": 0, "mpls-label": null, "vlan-id": null, "pad2": null, "source-ip": "205.185.216.10", "destination-ip": "192.168.1.125"}}
{"type": "event", "event": {"msg": "ET INFO Packed Executable Download", "classification": "Misc activity", "sensor-id": 0, "event-id": 583, "event-second": 1607588467, "event-microsecond": 769340, "signature-id": 2014819, "generator-id": 1, "signature-revision": 1, "classification-id": 29, "priority": 3, "sport-itype": 80, "dport-icode": 63676, "protocol": 6, "impact-flag": 0, "impact": 0, "blocked": 0, "mpls-label": null, "vlan-id": null, "pad2": null, "source-ip": "205.185.216.10", "destination-ip": "192.168.1.125"}}

props.conf on the UF is as follows:

[sourcetype=json]
KV_MODE=json
AUTO_KV_JSON=true
NO_BINARY_CHECK = true
disabled = false
SHOULD_LINEMERGE = false
TIME_FORMAT = "event-second": %s, "event-microsecond": %6N
LINE_BREAKER = }}(^s)

 and props.conf on the indexer/search head as follows:

[stanza]
TZ = UTC
SHOULD_LINEMERGE = false

[_json]
DATETIME_CONFIG =
LINE_BREAKER = }}
NO_BINARY_CHECK = true
disabled = false
KV_MODE = json

[json_no_timestamp]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
disabled = false

According to what I've told the UF to do in props.conf, the JSON events should be splitting up the JSON events using the double braces LINE_BREAKER }} as follows:

{"type": "event", "event": {"msg": "ET INFO Microsoft Connection Test", "classification": "Potentially Bad Traffic", "sensor-id": 0, "event-id": 581, "event-second": 1607588446, "event-microsecond": 790456, "signature-id": 2031071, "generator-id": 1, "signature-revision": 2, "classification-id": 3, "priority": 2, "sport-itype": 63591, "dport-icode": 80, "protocol": 6, "impact-flag": 0, "impact": 0, "blocked": 0, "mpls-label": null, "vlan-id": null, "pad2": null, "source-ip": "192.168.1.125", "destination-ip": "13.107.4.52"}}

{"type": "event", "event": {"msg": "ET POLICY PE EXE or DLL Windows file download HTTP", "classification": "Potential Corporate Privacy Violation", "sensor-id": 0, "event-id": 582, "event-second": 1607588467, "event-microsecond": 769440, "signature-id": 2018959, "generator-id": 1, "signature-revision": 4, "classification-id": 33, "priority": 1, "sport-itype": 80, "dport-icode": 63676, "protocol": 6, "impact-flag": 0, "impact": 0, "blocked": 0, "mpls-label": null, "vlan-id": null, "pad2": null, "source-ip": "205.185.216.10", "destination-ip": "192.168.1.125"}}

{"type": "event", "event": {"msg": "ET INFO Packed Executable Download", "classification": "Misc activity", "sensor-id": 0, "event-id": 583, "event-second": 1607588467, "event-microsecond": 769340, "signature-id": 2014819, "generator-id": 1, "signature-revision": 1, "classification-id": 29, "priority": 3, "sport-itype": 80, "dport-icode": 63676, "protocol": 6, "impact-flag": 0, "impact": 0, "blocked": 0, "mpls-label": null, "vlan-id": null, "pad2": null, "source-ip": "205.185.216.10", "destination-ip": "192.168.1.125"}}

but it doesn't.

Instead, the UF clumps them together as a single event and only reports on the first JSON stanza. Nothing I've tried for LINE_BREAKER seems to work - the UF seems to ignore it. Many thanks

[BongoTheWhippet · Accepted solution]

Okay. This looks like a bug. There's no way the JSON events should be clumped like this.