topic Re: Line breaking odd issue in Splunk Search

Line breaking odd issue

jwhughes58 — Wed, 31 Jul 2024 16:33:55 GMT

I'm working with a 9.1.2 UF on Linux. This is the props.conf

[stanza] # # Input-time operation on Forwarders # LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true SHOULD_LINEMERGE = false TRUNCATE = 999 DATETIME_CONFIG = CURRENT

This is the contents of the file

Splunk Reporting Hosts as of 07/31/2024 12:05:01 UTC host hostname1 hostname2 hostname3 hostname4 ... hostname1081

There are 1,083 lines in the file. I used od -cx to verify there is \n at the end of each line. For some reason, the last entry from a search consists of the first 257 lines from the file, and then the remaining lines are individual entries. I didn't have DATETIME_CONFIG in the stanza, so I thought that might be the issue. It is now, and it is still an issue. I'm out of ideas. Anyone see this before or have an idea on how to resolve this?

TIA,

Joe

Re: Line breaking odd issue

yuanliu — Wed, 31 Jul 2024 17:01:45 GMT

I had a similar problem and the answer is in Line breaking. See Why are REST API receivers/simple breaks input unexpectedly in Getting Data In.

Re: Line breaking odd issue

jwhughes58 — Wed, 31 Jul 2024 17:11:10 GMT

@yuanliuSo this section of the props.conf spec

MAX_EVENTS = <integer> * The maximum number of input lines to add to any event. * Splunk software breaks after it reads the specified number of lines. * Default: 256

takes precedence over the LINE_BREAKER?

Re: Line breaking odd issue

yuanliu — Wed, 31 Jul 2024 18:06:24 GMT

It doesn't take precedence. It just limits how many lines are allowed in each event. Splunk has a good reason to use 256 as default. I just wish they name the property with better clarity:-) You mentioned that you had 1083 lines. Raise MAX_EVENTS to 2000 for this sourcetype and you should be good. (You made a very astute observation about line count in your events from the very beginning. I wish I had that insight so I wouldn't have been stuck for years.)

Re: Line breaking odd issue

PickleRick — Wed, 31 Jul 2024 18:19:16 GMT

Wait a second. You're talking about an UF? And those props are where? On the UF or on the idx/HF? Do you use EVENT_BREAKER?

Re: Line breaking odd issue

jwhughes58 — Wed, 31 Jul 2024 20:50:12 GMT

@PickleRickThat was the issue. I was only pushing to the UF and not the indexers. Sometimes I forget that props.conf has parts that go to the indexer and parts go to the search heads.