How to edit my configuration to line break events at every "= ID:" in my sample log file?

[sshres5]

Some of the events are not being broken down. It works most of the time, but will not break lines couple of times, each time the log gets ingested.

Moreover, the config works fine in my test environment. And I repeat, there is no issue over there. However, when I deploy it on prod, it is failing couple of times in each log.

Log sample

= ID: 453608, XXXXXXXXX: **MonitorAll YYYYYYYYYYYYYYY YYYYYY aYYYYYYYYY: N/A, Target: N/A, Filename: N/A, Blocked: XXXXX, Endpoint: ??????????????? = ID: 453604, XXXXXXXXX: **MonitorAll -YYYYYYYY YYYYY vYYYYvYYYY N/A, Target: N/A, Filename: N/A, Blocked: XXXXX, Endpoint: ????????????? = ID: 453605, XXXXXXXX: **MonitorAll -YYYYYYY eYYYYYYY CYYYYYYYYYYY N/A, Target: N/A, Filename: N/A, Blocked: XXXXX, Endpoint: ????????????????

I have been trying to start a new line every time, I see = ID:

Both the configs work most of the time, but there is always some event, just like above, that has hiccups.

KV_MODE = none
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE = ^\=\sID:\s 

KV_MODE = none
SHOULD_LINEMERGE = false
LINE_BREAKER=([\n\r]+(\=\sID:\s+))

[somesoni2 · Accepted solution]

Try this

[Yoursourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)(?=\=\sID:\s)

OR

[Yoursourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\=\sID:\s)

[sshres5]

I have already tried both of them, does not work on my production system. Works great on my test box though. And it is just few lines on the log that it skips, but rest of them works fine.

[somesoni2]

Then there is must be something different between your test box and Production. Where are you putting this props.conf for line breaking (it should be on Indexer OR heavy forwarder whichever comes first in the data flow from source). Often test boxes are standalone Splunk (acts as both Search Head and Indexer) so when migrating to PROD with distributed environment, it should be configured on Indexer/HF and Splunk should be restarted.

[sshres5]

I have it under Indexer.

[sshres5]

So yes, the location of the props.conf was the issue. Once I moved it to the forwarder TA in HF, it works as a charm.

Thanks @somesoni2

This document has details on it.
http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

So if you are using HF, parsing needs to be done on HF.