I have a log file something like this
PStart bla bla bla bla
PQL% bla bla bla bla
PEnd bla bla bla bla
PQL% bla bla bla bla
bla bla blabla bla bla
PQL% bla bla bla bla
I want to break it at PStart or PQL% but if PQL% is within PStart and PEnd keep it one event.
For the above example, I want to have 3 events as follow
Event 1
PStart bla bla bla bla
PQL% bla bla bla bla
PEnd bla bla bla bla
Event 2
PQL% bla bla bla bla
bla bla blabla bla bla
Event 3
PQL% bla bla bla bla
The code I have is following which doesn't work properly
[sourcetype]
LINE_BREAKER = ([\r\n]+)(?:(?:PStart|(?:PQL%)))
MUST_NOT_BREAK_BEFORE = ([\r\n]+)(?:PEnd)
With the sample as given this is what I come up with:
[sourcetype]
LINE_BREAKER=([\r\n]+)(?:PStart|PEnd|PQL%)
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^PStart|^PQL%
MUST_NOT_BREAK_AFTER=^PStart
MUST_BREAK_AFTER=^PEnd
First during the line breaking stage, we break up into "lines" that start PStart, PEnd or PQL% (This is only necessary if a PEnd line may contain carrage returns or line feeds, otherwise you can get away with the default ([\r\n]+)
here
Then during line merging, we only break events before a PStart or PQL% line... this gets us our rough separation of events... but then with the MUST_NOT_BREAK_AFTER / MUST_BREAK_AFTER combination, we can ensure that from PStart to PEnd is a single block. (Obviously does not support embedding, but works single level).
With the sample as given this is what I come up with:
[sourcetype]
LINE_BREAKER=([\r\n]+)(?:PStart|PEnd|PQL%)
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^PStart|^PQL%
MUST_NOT_BREAK_AFTER=^PStart
MUST_BREAK_AFTER=^PEnd
First during the line breaking stage, we break up into "lines" that start PStart, PEnd or PQL% (This is only necessary if a PEnd line may contain carrage returns or line feeds, otherwise you can get away with the default ([\r\n]+)
here
Then during line merging, we only break events before a PStart or PQL% line... this gets us our rough separation of events... but then with the MUST_NOT_BREAK_AFTER / MUST_BREAK_AFTER combination, we can ensure that from PStart to PEnd is a single block. (Obviously does not support embedding, but works single level).
This works, thanks...
Be sure to click Accept
to close this question and reward the author of this MOST TERRIBLY EXCELLENT ANSWER!!!!
Thank you guys, I accepted, special thanks to Charlie.
This is a tough one. I am curious to see if there is a splunk-based solution. I've got nothing because negative-lookbehinds must be fixed-length.