Getting Data In

How to avoid default LINE_BREAKING from splitting up the following log into multiple lines?

dnavara
Explorer

We see that the following log lines are always split into multiple events. I've tried multiple variations of LINE_BREAKER,  BREAK_ONLY_BEFORE and  MUST_NOT_BREAK_AFTER  but nothing worked. Does anyone know how I could go about this? 

--------------------------------------------------
FlowFile Properties
Key: 'entryDate'
	Value: 'Wed Jan 04 16:14:58 UTC 2023'
Key: 'lineageStartDate'
	Value: 'Wed Jan 04 16:14:58 UTC 2023'
Key: 'fileSize'
	Value: '180'
FlowFile Attribute Map Content
--------------------------------------------------

 

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @dnavara,

the answer of @richgalloway is correct, remember to add 

SHOULD_LINEMERGE = True

Ciao.

Giuseppe

0 Karma

richgalloway
SplunkTrust
SplunkTrust

At the risk of repeating one of the "multiple variations", have you tried

LINE_BREAKER = -{50}([\r\n]+)
---
If this reply helps you, Karma would be appreciated.

dnavara
Explorer

Hi, thanks for the answer. I've tried this and it works when I upload test data manually but for some reason it doesn't work when I upload it to the HF. Is there any way these logs could already be broken up before they arrive? I've noticed that the logs have slightly different timestamps in the JSON format ie. 
2023-01-05T08:56:20.916403009Z and 2023-01-05T08:56:20.916400199Z. I am not sure if this is because they arrived at different times or some processing time on HF.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

If the data is processed by a HF then the LINE_BREAKER must be set on the HF.

---
If this reply helps you, Karma would be appreciated.
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

From Data to Insight: Announcing the Winners of the Splunk Dashboard Contest

Hi Splunkers, First off, thank you to everyone who participated in our very first From Data to Insight: The ...

Splunk Developers: Construct Your Future at the .conf26 Builder Bar

Calling all Splunk architects, platform admins, and app developers: the site is open, and the blueprints are ...

Quick connection discovery mode for forwarders

When a Splunk forwarder loses connectivity to its indexers, it does not always reconnect immediately. In many ...