I have a sourcetype of j_out that breaks the lines properly for jboss java log file.
The event breaks here:
60487.098: [Full GC (Ergonomics) there's more after this
My j_out sourcetype configuration is this:
MAX_TIMESTAMP_LOOKAHEAD=30
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=(?i)^\d\d:\d\d:\d\d,\d\d\d|(?i)^\d+\.\d\d\d
TRUNCATE=0
MAX_EVENTS=5000
The logs all start with either one of these:
15:41:41,136 ...
116.624: [Full GC (Metadata GC Threshold) ...
The first is a real time stamp, the second is a second counter since java was started. This would explain my regex above.
Any idea why it would randomly break incorrectly? (inconsistent)
I would suggest to try following configurations
props.conf on Indexers/Heavy forwarders
[YourSourceType]
MAX_TIMESTAMP_LOOKAHEAD=30
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?=(\d+:\d+:\d+,\d+)|(\d+\.\d+))
TRUNCATE=0
MAX_EVENTS=5000
Also, I don't see any proper/fix timestamp for the events, so you can use current time for the events, by adding following attribute
DATETIME_CONFIG=CURRENT
Why Line_breaker instead? It would truncate the time/second values doing it that way.
I added this and recycled the indexer and I'm still seeing the behavior.
9/18/15 1:00:08.000 AM 86333.133: [Full GC
9/18/15 1:00:12.000 AM [PSYoungGen: 7690K->0K(228352K)] [ParOldGen: 839127K->336242K(1280000K)] 846818K->336242K(1508352K) [PSPermGen: 229705K->211702K(441856K)], 3.9622840 secs] [Times: user=7.84 sys=0.48, real=3.96 secs]
j.out file on the server shows:
86333.133: [Full GC [PSYoungGen: 7690K->0K(228352K)] [ParOldGen: 839127K->336242K(1280000K)] 846818K->336242K(1508352K) [PSPermGen: 229705K->211702K(441856K)], 3.9622840 secs] [Times: user=7.84 sys=0.48, real=3.96 secs]
This will not truncate as I've put lookup-ahead regex ('?='). Did you get a chance to test it? You can check that in Preview to start with.
Ok, thanks. Im attempting your suggestion. Will wait a day to see if it happens on this indexer and get back to you.