Why is line breaking not happening correctly?

[mahesh27]

Sample logs:

quotation-events~~IM~. ABC~CA~Wed Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

D0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

quotation-events~~IS~;S. ABC~CA~Tue Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

V0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

quotation-events~~IM~. ADC~BA~Sat Jan 01 13:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

B0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

quotation-events~~IM~. CCC~HA~Sun Jan 01 20:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

G0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

Output in splunk: All evets are coming as a single event and not coming completely.

D0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1
IS~;S. ABC~CA~Tue Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events
V0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1|
quotation-events~~IM~. ADC~BA~Sat Jan 01 13:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events
B0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1
quotation-events~~IM~. CCC~HA~Sun Jan 01 20:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events
quotation-events~~
quotation-events~~

G0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

props.conf
[app:logs:sourcetype]

SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n)]+w{8}~~|quotation-events~~
NO_BINARY_CHECK=true
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=75
disabled=false
TIME_FORMAT=%a %b %d %H:%M:%S %Z
TIME_PREFIX=(?:[^~]+~)~(?:[^~]+~){3}
TRUNCATE=99999
ANNOTATE_PUNCT=false

[PickleRick]

Please post your events in code block or preformatted style so they don't get wrapped.

Also your LINE_BREAKER seems to be either mistyped, mispasted or misconfigured 😉

([\r\n)] - you open normal parenthesis, then open square bracket, then close normal parenthesis, then close square bracket. Something's off here. And that's not even trying to get to the logic of the regex.

[mahesh27]

yes in the line breaker it was a typo error

LINE_BREAKER=([\r\n]+)\w{8}~~|quotation-events~~
After correct it also i am getting the same issue.

as u said it should be in preformatted i dont know what it is.
One more think when i use the above props for testing purpose via add data toll through UI even are breaking correctly, but when there injecting into splunk i am getting this issue.
can u please help on this

[PickleRick]

In the editor that you use while writing on this forum you have two options - one is the "Insert/edit code example" in the top row of the editor. The other one is the style choice input which normally says "Paragraph" - if you change it to "preformatted", you will be typing text that won't get massacred  🙂

Paste a portion of your source data this way and then what it should look like after breaking.

[mahesh27]

quotation-events~~IM~. ABC~CA~Wed Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

G0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

D0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

 quotation-events~~IS~;S. ABC~CA~Tue Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

 V0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

 quotation-events~~IM~. ADC~BA~Sat Jan 01 13:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

 B0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

 quotation-events~~IM~. CCC~HA~Sun Jan 01 20:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

[PickleRick]

OK. So what are those? Separate lines from your log? And how they should be split? I mean what the resulting events should look like?

[mahesh27]

so my events should display like below:

Basically events should breaking which are starting with quotation-events into a new line and starting with G0C5A044~~ 

1st event should  be : quotation-events~~IM~. ABC~CA~Wed Jan 02 23:24:56 EST 2023~A~0.12~0...~2345.78~SM~quotation-events

2nd event should be : B0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

3rd event: quotation-events~~IM~. CCC~HA~Sun Jan 01 20:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

4th event: G0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

[PickleRick]

Sorry, you lost me here. Out of the lines you pasted, half of the content is not even included in the events you're showing. And there is no indication whatsoever what the events should be broken on.

[mahesh27]

Hi, if u see my posts from beginning i think u will get it.

here there are 2 types of events, which need to shown in new line.
if u see my props u come to know frm where the event should break, it is very clear.