Why am I seeing inconsistent behavior with BREAK_ONLY_BEFORE with my sourcetype configuration?

[BP9906]

I have a sourcetype of j_out that breaks the lines properly for jboss java log file.

The event breaks here:

60487.098: [Full GC (Ergonomics)  there's more after this

My j_out sourcetype configuration is this:

MAX_TIMESTAMP_LOOKAHEAD=30
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=(?i)^\d\d:\d\d:\d\d,\d\d\d|(?i)^\d+\.\d\d\d
TRUNCATE=0
MAX_EVENTS=5000

The logs all start with either one of these:

15:41:41,136 ...
116.624: [Full GC (Metadata GC Threshold) ...

The first is a real time stamp, the second is a second counter since java was started. This would explain my regex above.

Any idea why it would randomly break incorrectly? (inconsistent)

[somesoni2]

I would suggest to try following configurations

props.conf on Indexers/Heavy forwarders

[YourSourceType]
MAX_TIMESTAMP_LOOKAHEAD=30
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?=(\d+:\d+:\d+,\d+)|(\d+\.\d+))
TRUNCATE=0
MAX_EVENTS=5000

Also, I don't see any proper/fix timestamp for the events, so you can use current time for the events, by adding following attribute

DATETIME_CONFIG=CURRENT

[BP9906]

Why Line_breaker instead? It would truncate the time/second values doing it that way.

[BP9906]

I added this and recycled the indexer and I'm still seeing the behavior.

9/18/15 1:00:08.000 AM 86333.133: [Full GC

9/18/15 1:00:12.000 AM [PSYoungGen: 7690K->0K(228352K)] [ParOldGen: 839127K->336242K(1280000K)] 846818K->336242K(1508352K) [PSPermGen: 229705K->211702K(441856K)], 3.9622840 secs] [Times: user=7.84 sys=0.48, real=3.96 secs]

j.out file on the server shows:
86333.133: [Full GC [PSYoungGen: 7690K->0K(228352K)] [ParOldGen: 839127K->336242K(1280000K)] 846818K->336242K(1508352K) [PSPermGen: 229705K->211702K(441856K)], 3.9622840 secs] [Times: user=7.84 sys=0.48, real=3.96 secs]

[somesoni2]

This will not truncate as I've put lookup-ahead regex ('?='). Did you get a chance to test it? You can check that in Preview to start with.

[BP9906]

Ok, thanks. Im attempting your suggestion. Will wait a day to see if it happens on this indexer and get back to you.