topic Line break with multiple Linebreaker in Getting Data In

Line break with multiple Linebreaker

ss026381 — Fri, 05 Apr 2019 17:51:34 GMT

I have a log file something like this

PStart  bla bla bla bla 
PQL% bla bla bla bla
PEnd bla bla bla bla 
PQL% bla bla bla bla
bla bla blabla bla bla
PQL% bla bla bla bla

I want to break it at PStart or PQL% but if PQL% is within PStart and PEnd keep it one event.

For the above example, I want to have 3 events as follow

Event 1

PStart  bla bla bla bla 
PQL% bla bla bla bla
PEnd bla bla bla bla

Event 2

PQL% bla bla bla bla
bla bla blabla bla bla

Event 3

PQL% bla bla bla bla

The code I have is following which doesn't work properly

[sourcetype]
LINE_BREAKER = ([\r\n]+)(?:(?:PStart|(?:PQL%))) 
MUST_NOT_BREAK_BEFORE = ([\r\n]+)(?:PEnd)

Re: Line break with multiple Linebreaker

woodcock — Sun, 07 Apr 2019 14:29:21 GMT

This is a tough one. I am curious to see if there is a splunk-based solution. I've got nothing because negative-lookbehinds must be fixed-length.

Re: Line break with multiple Linebreaker

acharlieh — Wed, 30 Sep 2020 00:03:11 GMT

With the sample as given this is what I come up with:

[sourcetype]
LINE_BREAKER=([\r\n]+)(?:PStart|PEnd|PQL%)
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^PStart|^PQL%
MUST_NOT_BREAK_AFTER=^PStart
MUST_BREAK_AFTER=^PEnd

First during the line breaking stage, we break up into "lines" that start PStart, PEnd or PQL% (This is only necessary if a PEnd line may contain carrage returns or line feeds, otherwise you can get away with the default ([\r\n]+) here

Then during line merging, we only break events before a PStart or PQL% line... this gets us our rough separation of events... but then with the MUST_NOT_BREAK_AFTER / MUST_BREAK_AFTER combination, we can ensure that from PStart to PEnd is a single block. (Obviously does not support embedding, but works single level).

Re: Line break with multiple Linebreaker

ss026381 — Mon, 08 Apr 2019 13:02:20 GMT

This works, thanks...

Re: Line break with multiple Linebreaker

woodcock — Mon, 08 Apr 2019 14:38:56 GMT

Be sure to click Accept to close this question and reward the author of this MOST TERRIBLY EXCELLENT ANSWER!!!!

Re: Line break with multiple Linebreaker

ss026381 — Mon, 08 Apr 2019 15:00:01 GMT

Thank you guys, I accepted, special thanks to Charlie.