Splunk Enterprise

Issue with event Line Breaking

SplunkExplorer
Contributor

Hi Splunkers, I have an inssue with a line breaking use case. I know it is very simple to fix, but I still have the problem, so there is something I'm not getting in the right way. 
First, a little bit of info about env.

  • Log source: custom application
  • Input type: File monitor Input
  • File monitoring: via UF, so a deployed app has been deployed with a DS
  • Final flow: Log Source with UF -> HF -> Splunk Cloud
  • Data are ingested? Yes.

Issue: once log are collected, we got a unique big log. So, we need to separate logs in different events. So I thought: Ok fine, I did a lot of custom addon, I know how do do it. By the way, I did not performed initial configuration about UF, so I check related deployed app and logs . That's the summary:

  • Single event ends with "platform":"ArcodaSAT"}
  • UF deployed app is very simple: it has an app.conf, an inputs,.conf and a props.conf.
  • inputs.conf file works fine due logs are ingested from the right source
  • Below, settings in I found in props.conf:

            [<sourcetype_name>]
            CHARSET=AUTO
            LINE_BREAKER = (\"platform\"\:\"ArcodaSAT\"\})
            SHOULD_LINEMERGE = true

Observation:

  • Regex is fine; I tested it on regex101 with a log sample and it catch fine.
  • I tried, in the LINE_BREAKER, both using round brackets - cause documentation say that parameter use the capture group to check where new log starts - and without. Same result.
  • SHOULD_LINEMERGE has be set both as true and false: same result

Let me say again: I know this is some nonsense I'm missing, but I can't find it.

Labels (2)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

The LINE_BREAKER attribute requires at least one capture group and the text that matches the first capture group will be discarded and replaced with a event break.  Knowing this and that an empty capture group is allowed, try these settings:

 

[<sourcetype_name>]
CHARSET=AUTO
LINE_BREAKER = "platform":"ArcodaSAT"\}()
SHOULD_LINEMERGE = false

 

 

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

The LINE_BREAKER attribute requires at least one capture group and the text that matches the first capture group will be discarded and replaced with a event break.  Knowing this and that an empty capture group is allowed, try these settings:

 

[<sourcetype_name>]
CHARSET=AUTO
LINE_BREAKER = "platform":"ArcodaSAT"\}()
SHOULD_LINEMERGE = false

 

 

---
If this reply helps you, Karma would be appreciated.

marnall
Builder

The props.conf file should be on the machine that is parsing your logs. If your log path is UF->HF->Cloud, then likely the HF machine is the one doing the parsing, and it should have the props.conf file, not the UF.

Also, keep in mind that the first capture group of LINE_BREAKER is discarded. It is intended to capture the filler characters that occur between distinct events. If you would like to keep "platform":"ArcodaSAT"} as part of the first event, then it should not be in a capture group. Try this:

LINE_BREAKER = \"platform\"\:\"ArcodaSAT\"\}()

For SHOULD_LINEMERGE, this would be better set as FALSE unless you would like events to be recombined to make bigger events. If your LINE_BREAKER above works well to separate distinct events, then SHOULD_LINEMERGE should be false

SHOULD_LINEMERGE = false

0 Karma
Get Updates on the Splunk Community!

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...

New This Month - SLO Capabilities, APM Advanced Filtering & Usage Analytics Plus ...

More for SLO Management We’re continuing to expand the built-in SLO management experience in Splunk ...

Enterprise Security Content Update (ESCU) | New Releases

In June, the Splunk Threat Research Team had 2 releases of new security content via the Enterprise Security ...