Why does linebreaking regex have no capturing groups?

[jackin]

Hi

Need help to fix the below error

 My Props :

Sample events:

[richgalloway]

As the message says, the LINE_BREAKER attribute must contain a capture group (a set of parenthses).  Try LINE_BREAKER = ()^\{

You only need to specify LINE_BREAKER once in a stanza.

[jackin]

Thanks for reply @richgalloway 

When applying a linebreaker, all logs fall under a single line.

It is showing like Failed to parse timestamp Defaulting to file modtime

[richgalloway]

Failing to parse timestamps is a different problem.  Please post a new question so this one can focus on the line breaking problem.

What do you mean by "all logs fall under a single line"?  The sample events appear to be multi-line.

[jackin]

Hi

after using the below props configuration , the same error as mentioned above is coming ..

SHOULD_LINEMERGE=false
LINE BREAKER=([\r\n]+){
NO BINARY CHECK-true
BREAK ONLY_BEFORE=^\{
CHARSET=UTF-8
disabled=false
KV MODE=json
MAX TIMESTAMP LOOKAHEAD=70
TIME PREFIX="(timeStamplevtime)"\s*: \s*"
TIME FORMAT=%Y-%m-%dT%I:%M:%S
TRUNCATE=999999

[richgalloway]

Remove the BREAK_ONLY_BEFORE setting.

[jackin]

If I remove it. Logs are not breaking properly.

[richgalloway]

You shouldn't have both BREAK_ONLY_BEFORE and LINE_BREAKER in the same stanza.  Choose one or the other.  If you don't use LINE_BREAKER then SHOULD_LINEMERGE should be set to true.  See https://docs.splunk.com/Documentation/Splunk/9.0.4/Data/Configureeventlinebreaking for details.

[lorinj62]

The documentation that you refer to has them both in the same stanza, in steps 3 and 4:

Break and reassemble the data stream into events

This method oftentimes simplifies the configuration process, as it gives you access to several settings that you can use to define line-merging rules.

You must perform these steps on the heavy forwarder that you have designated to send data to your Splunk Cloud Platform instance.

1. On the forwarder that is to send data to your Splunk Cloud Platform instance, use a text editor to open $SPLUNK_HOME/etc/system/local/props.conf for editing.
2. In this file, specify a stanza in the props.conf configuration file that represents the stream of data you want to break and reassemble into events.
3. In that stanza, configure the LINE_BREAKER setting with a regular expression that breaks the data stream into multiple lines.
4. Add the SHOULD_LINEMERGE setting to the stanza, and set its value to true.
5. Configure additional line-merging settings, such as BREAK_ONLY_BEFORE and others, to specify how the forwarder is to reassemble the lines into events. For more information on the line-merging settings, see Attributes that apply only when the SHOULD_LINEMERGE setting is true later in this topic.

If your data conforms well to the default LINE_BREAKER value, which is any number of newlines and carriage returns, you don't need to change the LINE_BREAKER setting. Instead, set SHOULD_LINEMERGE=true and use the line-merging settings to reassemble the data.

[richgalloway]

I contend the documentation is incorrect.  LINE_BREAKER and BREAK_ONLY_BEFORE are contradictory and shouldn't be used together.  At the very least, great care should be used to ensure the two settings work properly.

[PickleRick]

Actually, I believe the docs are correct since BREAK_ONLY_BEFORE applies to the line-merging stage which - if enabled - happens after line breaking.

Anyway, @jackin  unless you have a very, very peculiar use case, as a rule of thumb you should never enable line-merging. It is resource-intensive and most often you can achieve the same result by simply chosing a proper line breaker.

So, how I would approach this - I'd firstly try to use the default ([\r\n]+) linebreaker and check if the stream gets broken into separate lines (disable SHOULD_LINEMERGE!).

If it does, you can start searching how to anchor the breaker to the opening bracket.

If it doesn't, that means you have some other characters in your data stream and you have to check what it is.