splunk ignoring LINE_BREAKER

[phemmer]

I've configured a source type in props.conf with LINE_BREAKER = (\n+) to remove the \r from the default value. This was done so that we can send multi-line events using \r as the delimiter between lines, and \n as the delimiter between events.
The problem however is that splunk is still breaking messages into multiple events on the \r character.

Any ideas why?

Here's an example.
This screenshot is the tcpdump output of a 2-line message, where the second line should be this is line 2.

Here's the event as shown by the search app.

Here's the active sourcetype configuration:

The etc/system/local/props.conf on the indexer nodes:

[syslog]
LINE_BREAKER = (\n+)

[phemmer · Accepted solution]

Figured it out.
We had an app which was doing transforms on syslog sources. However the regex it was using used .* to match all data, but . doesn't match carriage return (by default).
I tweaked the regex the app uses to add (?s) to set the regex flag letting it match.

[s2_splunk]

The sourcetype definition in your screenshot does not match what you list under etc/system/local/props.conf on the indexer nodes!? Looks like you have defined the sourcetype in the app called "AppsBrowser", can you share the props.conf file from etc/apps/appsBrowser/local, please?

[phemmer]

@alemarzu sorry, looks like the screenshots broke. I uploaded them to an image host.

@ssievert: I don't know why that shows "AppsBrowser". I've never touched that. There is no etc/appsbrowser/local, and etc/appsbrowser/default has no props.conf.

@richgalloway: That is the default, so I just left it alone for now. However if SHOULD_LINEMERGE were causing a problem, there would be too many lines in a single event, which is the opposite of my problem.

[richgalloway]

The Admin manual says LINE_BREAKER should only be used when SHOULD_LINEMERGE=false. It also says BREAK_ONLY_BEFORE only applies if SHOULD_LINEMERGE=true so it doesn't make sense to have both LINE_BREAKER and BREAK_ONLY_BEFORE in the same sourcetype. Also, while LINE_BREAKER requires a capturing group there is no explicit requirement for a capturing group in BREAK_ONLY_BEFORE.

[alemarzu]

Hi there phemmer,

Would you mind to paste the sample data ?