Splunk Enterprise

Why is line breaking not happening correctly?

mahesh27
Communicator

Sample logs:

quotation-events~~IM~. ABC~CA~Wed Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

D0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

quotation-events~~IS~;S. ABC~CA~Tue Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

V0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

quotation-events~~IM~. ADC~BA~Sat Jan 01 13:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

B0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

quotation-events~~IM~. CCC~HA~Sun Jan 01 20:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

G0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1


Output in splunk: All evets are coming as a single event and not coming completely.


D0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1
IS~;S. ABC~CA~Tue Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events
V0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1|
quotation-events~~IM~. ADC~BA~Sat Jan 01 13:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events
B0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1
quotation-events~~IM~. CCC~HA~Sun Jan 01 20:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events
quotation-events~~
quotation-events~~

G0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1




props.conf
[app:logs:sourcetype]

SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n)]+w{8}~~|quotation-events~~
NO_BINARY_CHECK=true
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=75
disabled=false
TIME_FORMAT=%a %b %d %H:%M:%S %Z
TIME_PREFIX=(?:[^~]+~)~(?:[^~]+~){3}
TRUNCATE=99999
ANNOTATE_PUNCT=false



0 Karma

PickleRick
SplunkTrust
SplunkTrust

Please post your events in code block or preformatted style so they don't get wrapped.

Also your LINE_BREAKER seems to be either mistyped, mispasted or misconfigured 😉

([\r\n)] - you open normal parenthesis, then open square bracket, then close normal parenthesis, then close square bracket. Something's off here. And that's not even trying to get to the logic of the regex.

0 Karma

mahesh27
Communicator

yes in the line breaker it was a typo error

LINE_BREAKER=([\r\n]+)\w{8}~~|quotation-events~~
After correct it also i am getting the same issue.

as u said it should be in preformatted i dont know what it is.
One more think when i use the above props for testing purpose via add data toll through UI even are breaking correctly, but when there injecting into splunk i am getting this issue.
can u please help on this

0 Karma

PickleRick
SplunkTrust
SplunkTrust

In the editor that you use while writing on this forum you have two options - one is the "Insert/edit code example" in the top row of the editor. The other one is the style choice input which normally says "Paragraph" - if you change it to "preformatted", you will be typing text that won't get massacred  🙂

Paste a portion of your source data this way and then what it should look like after breaking.

0 Karma

mahesh27
Communicator
quotation-events~~IM~. ABC~CA~Wed Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

G0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

D0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1
 quotation-events~~IS~;S. ABC~CA~Tue Jan 02 23:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events
 V0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1
 quotation-events~~IM~. ADC~BA~Sat Jan 01 13:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events
 B0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1
 quotation-events~~IM~. CCC~HA~Sun Jan 01 20:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

OK. So what are those? Separate lines from your log? And how they should be split? I mean what the resulting events should look like?

0 Karma

mahesh27
Communicator

so my events should display like below:


Basically events should breaking which are starting with quotation-events into a new line and starting with G0C5A044~~ 


1st event should  be : quotation-events~~IM~. ABC~CA~Wed Jan 02 23:24:56 EST 2023~A~0.12~0...~2345.78~SM~quotation-events

2nd event should be : B0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

3rd event: quotation-events~~IM~. CCC~HA~Sun Jan 01 20:24:56 EST   2023~A~0.12~0...~2345.78~SM~quotation-events

4th event: G0C5A044~~AB~DFR~Mon Jan 01 12:52:14 EST   2022~B~107.45~106.90~123.09~T~2345A1

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Sorry, you lost me here. Out of the lines you pasted, half of the content is not even included in the events you're showing. And there is no indication whatsoever what the events should be broken on.

0 Karma

mahesh27
Communicator

Hi, if u see my posts from beginning i think u will get it.

here there are 2 types of events, which need to shown in new line.
if u see my props u come to know frm where the event should break, it is very clear.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...