topic LINE_BREAKER=([\r\n]+) not working? in Getting Data In

LINE_BREAKER=([\r\n]+) not working?

bitnapper — Fri, 03 Mar 2023 13:19:37 GMT

Hi,

I have a test instance of splunk - fresh out of the box. Only configure the essentials and imported a dump from the OpenLibrary.org. I have the problem that every single line should be one event but plenty of lines are merged together and I can't figure out why. The source data has reliable linebreaks in so the default should work.

I had the same issue in my enterprise environment and nobody could tell me why this is happening. After a while it magically disappeared and as far as I can tell it has vanished for good but I'd like to understand the reason why this is happening and prevent it from happening again.

Regards

Thorsten

Re: LINE_BREAKER=([\r\n]+) not working?

LRF — Fri, 03 Mar 2023 20:13:28 GMT

Hi @bitnapper,

Without knowing what type of logs you are working with, I would assume your issue might be related to the use of the default LINE_BREAKER ([\r\n]+) while also keeping SHOULD_LINEMERGE = true (default setting).

SHOULD_LINEMERGE explanation from props.conf file:

* When you set this to "true", Splunk software combines several lines of data
  into a single multi-line event, based on values you configure
  in the following settings.
* When you set this to "false", Splunk software does not combine lines of
  data into multiline events.
* Default: true

When SHOULD_LINEMERGE is set to true, other settings (BREAK_ONLY_BEFORE_DATE (default), BREAK_ONLY_BEFORE, MUST_BREAK_AFTER...) are used to define how Splunk software builds multi-line events merging previously broken lines.

Also, remember to restart your Splunk instance where the new parsing rules are being applied (tipically HF/INDEXER or all-in-one architecture)

Hope this helps, have a nice day,

Fabrizio

Re: LINE_BREAKER=([\r\n]+) not working?

bitnapper — Sat, 04 Mar 2023 10:18:33 GMT

Hi Fabrizio,

thanks for you input. I had linemerge to false to avoid exactly that. Also I configure a one-time import via web. I know that splunk likes restarts like a windows 3.11 but there is no point at which one can restart splunk. I goes directly after applying to importing the data.