topic Re: Parsing multiline does not working as expected in Getting Data In

Parsing multiline does not working as expected

TheEggi98 — Tue, 04 Jan 2022 14:16:21 GMT

Hello Splunkers,

i need help.
I have multiline logs looking like:

01/04/22 03:00:00 MONITOR_RAP: blah blah: blah ; blah ; blah ; blah ; blah ; 01/04/22 07:00:00 MONITOR_RAP: blah blah: blah ; blah ; blah ; blah ; blah ;

i ingest them with the following sourcetype stanza:

[mysourcetype] SHOULD_LINEMERGE = true BREAK_ONLY_BEFORE_DATE = true TRUNCATE = 1000 TIME_PREFIX = ^ MAX_TIMESTAMP_LOOKAHEAD = 17 TIME_FORMAT = %m/%d/%y %H:%M:%S

The Universal Forwarder monitors the Directory where the logs landing.
The first ingestion succeded without problems but when new logs written in the logfile of today, the parsing made multiple events out of the new logentries.

The monitor Stanza:

[monitor://<path>/*.log] disabled = 0 sourcetype = mysourcetype index = myindex

So the first couple events were parsed like it should but when new logs arrived splunk made multiple events like (the codeblocks represent one multiline event, each codeblock represents a wrong parsed event in splunk):

01/04/22 03:00:00 blah: blah ; blah ;

blah ; blah;

blah ;

What is wrong? Is it maybe a bug? I dont get it.

Re: Parsing multiline does not working as expected

richgalloway — Tue, 04 Jan 2022 16:00:01 GMT

It seems like the UF is breaking the stream in the wrong place so the indexer can't process the events correctly. Try adding these settings in props.conf on the UF.

[mysourcetype] EVENT_BREAKER_ENABLE = true EVENT_BREAKER = ([\r\n]+)\d\d\/\d\d/\d\d

Re: Parsing multiline does not working as expected

TheEggi98 — Wed, 05 Jan 2022 14:16:34 GMT

The Problem is fixed now.

It wasnt directly Splunk.
The Logs were written line by line, and that caused Splunk to rip the events apart.

Now the logs get written in a file not monitored by UF and after writing the file gets renamed so the UF monitors it.