Getting Data In

Why does linebreaking regex have no capturing groups?

jackin
Path Finder

Hi

Need help to fix the below error

jackin_0-1683885511356.png

 My Props :

jackin_1-1683885571420.png


Sample events:

jackin_2-1683885624631.png

 

Labels (3)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

As the message says, the LINE_BREAKER attribute must contain a capture group (a set of parenthses).  Try LINE_BREAKER = ()^\{

You only need to specify LINE_BREAKER once in a stanza.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jackin
Path Finder

Thanks for reply @richgalloway 

When applying a linebreaker, all logs fall under a single line.

It is showing like Failed to parse timestamp Defaulting to file modtime

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Failing to parse timestamps is a different problem.  Please post a new question so this one can focus on the line breaking problem.

What do you mean by "all logs fall under a single line"?  The sample events appear to be multi-line.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jackin
Path Finder

Hi

after using the below props configuration , the same error as mentioned above is coming ..

SHOULD_LINEMERGE=false
LINE BREAKER=([\r\n]+){
NO BINARY CHECK-true
BREAK ONLY_BEFORE=^\{
CHARSET=UTF-8
disabled=false
KV MODE=json
MAX TIMESTAMP LOOKAHEAD=70
TIME PREFIX="(timeStamplevtime)"\s*: \s*"
TIME FORMAT=%Y-%m-%dT%I:%M:%S
TRUNCATE=999999

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Remove the BREAK_ONLY_BEFORE setting.

---
If this reply helps you, Karma would be appreciated.
0 Karma

jackin
Path Finder

If I remove it. Logs are not breaking properly.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You shouldn't have both BREAK_ONLY_BEFORE and LINE_BREAKER in the same stanza.  Choose one or the other.  If you don't use LINE_BREAKER then SHOULD_LINEMERGE should be set to true.  See https://docs.splunk.com/Documentation/Splunk/9.0.4/Data/Configureeventlinebreaking for details.

---
If this reply helps you, Karma would be appreciated.

lorinj62
Engager

The documentation that you refer to has them both in the same stanza, in steps 3 and 4:

Break and reassemble the data stream into events

This method oftentimes simplifies the configuration process, as it gives you access to several settings that you can use to define line-merging rules.

You must perform these steps on the heavy forwarder that you have designated to send data to your Splunk Cloud Platform instance.

  1. On the forwarder that is to send data to your Splunk Cloud Platform instance, use a text editor to open $SPLUNK_HOME/etc/system/local/props.conf for editing.
  2. In this file, specify a stanza in the props.conf configuration file that represents the stream of data you want to break and reassemble into events.
  3. In that stanza, configure the LINE_BREAKER setting with a regular expression that breaks the data stream into multiple lines.
  4. Add the SHOULD_LINEMERGE setting to the stanza, and set its value to true.
  5. Configure additional line-merging settings, such as BREAK_ONLY_BEFORE and others, to specify how the forwarder is to reassemble the lines into events. For more information on the line-merging settings, see Attributes that apply only when the SHOULD_LINEMERGE setting is true later in this topic.

If your data conforms well to the default LINE_BREAKER value, which is any number of newlines and carriage returns, you don't need to change the LINE_BREAKER setting. Instead, set SHOULD_LINEMERGE=true and use the line-merging settings to reassemble the data.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I contend the documentation is incorrect.  LINE_BREAKER and BREAK_ONLY_BEFORE are contradictory and shouldn't be used together.  At the very least, great care should be used to ensure the two settings work properly.

---
If this reply helps you, Karma would be appreciated.
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Actually, I believe the docs are correct since BREAK_ONLY_BEFORE applies to the line-merging stage which - if enabled - happens after line breaking.

Anyway, @jackin  unless you have a very, very peculiar use case, as a rule of thumb you should never enable line-merging. It is resource-intensive and most often you can achieve the same result by simply chosing a proper line breaker.

So, how I would approach this - I'd firstly try to use the default ([\r\n]+) linebreaker and check if the stream gets broken into separate lines (disable SHOULD_LINEMERGE!).

If it does, you can start searching how to anchor the breaker to the opening bracket.

If it doesn't, that means you have some other characters in your data stream and you have to check what it is.

0 Karma
Get Updates on the Splunk Community!

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Announcing the General Availability of Splunk Enterprise Security 8.1!

We are pleased to announce the general availability of Splunk Enterprise Security 8.1. Splunk becomes the only ...

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...