New sourcetype linebreaking issues

[gelica]

Hi,

I'm a new Splunk user and I'm trying to define a new source type.

My log file looks like:

-----------
Name1
-----------
Info1.1: one
info1.2: two
-----------
Name2
-----------
Info2.1: one
Info2.1: two
two
Info2.3: Three
etc..

I want to have a sourcetype that breaks my file before every name, but the real names aren't as nicely formatted as the ones in my example, they often consist of multiple words and sometimes some special character as well.

I manage to break before each --------- but then the names and the info gets separated. I also tried setting the LINE_BREAKER but then the pattern matching my regex is removed.

I tried some different regexes, similar to (-{2,50}[^:]+-{2,50}[^:]+:) and (?<=--)([^-:]+)(?=--) but they don't work.
It seems like the newlines aren't recognized correctly..?

Does anyone have any idea of what I'm doing wrong?

Thanks!

[lguinn2 · Accepted solution]

Try this in your props.conf

[yoursourcetype]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = (?m)-{10}.*-{10}

I know that my regex is probably not what you need - but substitute whatever you used for the LINE_BREAKER. Using BREAK_ONLY_BEFORE will have a similar effect, but it doesn't remove the matching pattern.

[lguinn2]

Try replacing the word "LINE_BREAKER" with "BREAK_ONLY_BEFORE" and see what happens.

[gelica]

I replaced the regex with ([\r\n]+)(-{2,50})([\r\n]+)([^:]+)([\r\n]+)(-{2,50}[^:]+:) and also tried removing the ([\r\n]+)'s from my regex.
I played around with LINE_BREAKER some more and got this to work in the data preview:
LINE_BREAKER = ([\r\n]+)(-{2,50})([\r\n]+)([^:]+)([\r\n]+)(-{2,50}[^:]+:)
MAX_EVENTS = 10000
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
pulldown_type = 1
For some reason the text isn't removed by this linebreaker pattern. Everything looks OK in the preview but when I add the data for real, it gets messed up..