Cannot break for some events by lines

[NoSpaces]

Hello to everyone!
I have a strange issue with some events that come from our virtual environment.
As you can see in the screenshot, some events become multi-lined, but I don't want to observe such behavior.
I suspected the SHOULD_LINEMERGE setting, which can merge events with similar timestamps, but now I turned it off.
Below the Splunk search window I attached a result from regex101.com that shows the working of my LINE_BREAKING option.
I also tried to use the default ([\r\n]+) regex, but it did not work either.
What did I miss in my approach?
My props.conf for the desired sourcetype looks like:

[vsi_esxi_syslog]
LINE_BREAKER = ([\r\n]+)|(\n+)
SHOULD_LINEMERGE = false
TIME_PREFIX = ^<\d+>

[richgalloway]

It's looks to me like lines are wrapped by the terminal rather than newlines being added.  In that case, no special props.conf settings are needed.  You should have success with

[vsi_esxi_syslog]
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false
TIME_PREFIX = ^<\d+>

[NoSpaces]

I already tried the default sequence ([\r\n]+), as I wrote in the original post.
After your suggestion, I checked it one more time, but I still see multiline events.

[PickleRick]

1. The default ([\r\n]+) includes your (\n+), so you don't have to specify that alternative (I'm not sure how it would behave since LINE_BREAKER is supposed to contain one capture group anyway).

2. How are you getting your events? Sent by syslog? Written to a file and read with a UF?

3. Just to be on the safe side - you're not expecting already ingested events to be re-broken, right?

[NoSpaces]

1. I suggested that for some reasons events don't contain a whole pattern and tried to check only "\r".
"\r" works on the regex101. Now I changed this option to the default "([\r\n]+)".

2. I'm getting these events via Syslog. Logs come to the Indexer layer, where I apply props.conf via the Manager node, then I search on the Searchhead layer.

3. Yeah, I understand that I need to wait while indexers apply configuration and then search only events that came to Splunk after.

[PickleRick]

OK. We're getting somewhere. 🙂

2a. You have direct network inputs on indexers? That's not the best idea and calls for some re-architecting. But that shouldn't be the reason for problems with line breaking.

2b. What do you mean by "apply props.conf"? Do you push configuration bundle to the cluster from the CM or just define props.conf on the CM and just let it be? If you're pushing the configs, did you verify the effective configs on the indexer(s) receiving the events?

[NoSpaces]

2a.
You are right.
In this case Indexers ingesting logs via couple of tcp ports. We have load balancer that smears logs on all indexers.
You can advertise me some suggestions for architecture if want. I wiil glad to see some good advises 😃
When I was creating our Splunk env I decide that it is the most convenient way in our production. It's allow me to easily route events to indexes only by port.

2b.
Yes, I'm talking about applying the bundle.
I'm sure configuration was applied because I always use "splunk apply cluster-bundle -auth password --answer-yes" and wait for nodes to reboot if they decide to do it.

[PickleRick]

Syslog  often does not play well with loadbalancers. Especially when sent by tcp. So it might just be that your LB is doing something ugly to your events. With such setup I'd start with a tcpdump on the indexer to verify what is actually getting to the network input.

[NoSpaces]

We will try to check it 
Thanks for advice

[richgalloway]

Are you sure they're multi-line events?  Both screenshots in the OP showed only single-line events.  If you can, please post a screenshot of the multi-line events in Splunk.

[NoSpaces]

I'm sorry, but maybe I have the wrong understanding of what a multi-line event is.
I expect that any event that has more than one line is a multi-line event.
The event that I posted in the OP had more than one line, and what is more important, more than one timestamp.
I expected that Splunk must have broken such an event by each "\r\n".

P.S:
I attached another screenshot with the linecount field.

[richgalloway]

Thanks for the new screenshot.  It makes the situation a little clearer.  Try this line breaker

LINE_BREAKER = ()<\d+>

Is @PickleRick said, make sure the settings are on all indexers and heavy forwarders and that the instances are restarted after configurations are changed.

[NoSpaces]

Are you sure that this line breaker will be good in my situation?
As you can see in the last screenshot, the event contains "... High: <0>, Low: <0>" and I suspect that this breaker will cut events in unexpected places.

[richgalloway]

Fair point.  My goal is to break lines without regard for line ends since Splunk appears to be ignoring some of them.

Try

LINE_BREAKER = ()<\d+>20\d\d

[NoSpaces]

After validating the logs and with the help of your solution, I decided to use these settings.

SHOULD_LINEMERGE = false
TRUNCATE = 0 (because some events are already broken and don't contain a linebreaking rule).

Thank you for the advice.

[NoSpaces · Accepted solution]

Sorry, missed the main rule:

LINE_BREAKER = <\d+>\d{4}-

[PickleRick]

Yes, it should have been broken into multiple events but the question is - again - how are you ingesting those logs (and where are you applying your configurations).