Getting Data In

Log file with CRLF not producing multiple indexed events

shocko
Contributor

I'm using Splunk Enterprise 9 on Windows Server 2019 and monitoring a simple log file that has CRLF lines endings and is encoded as UTF8. My inputs stanza is as follows:

 

[monitor://c:\windows\debug\test.log]
disabled = 0
sourcetype = my_sourcetype
index=test

 

Consider two consectuive lines in the log file

 

Some data 1
Some data 2

 

When indexed this creates a single event rather than my expectation of 2 events.

Where am I going wrong?

 

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What are the props.conf settings for [mysourcetype]?

---
If this reply helps you, Karma would be appreciated.
0 Karma

shocko
Contributor

I dont have one as I didn't think I needed one for something this simple. I have tried just now though adding this to no avail

 

[my_sourcetype]
SHOULD_LINEMERGE = FALSE
LINE_BREAKER = ([\r\n]+)

 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Hi
This should work. If you are looking spec file for props.conf you see that SHOULD_LINEMERGE = true for unknown reason? It should be false for almost 100% of cases.
r. Ismo
0 Karma

shocko
Contributor

Is there any tooling (btool perhaps) that would tell me what props/transfroms are being applied to my sourcetype? Even if I drop the sourcetype form my inputs.conf the issue perists 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

You could use btool to look what is applied to your sourcetype, BUT if there is also apply to source or host something those will override sourcetype definitions. Unfortunately I don't know if there is any tool which can show to you which of those are applied ;-(

0 Karma

shocko
Contributor

So I spun up a new Splunk instance in Podman (completely clean) and ingested the same file and the behaviourt is the same with no line breaking! This is with UTF8 encoding and CRLF or LF endings. So I went into the UI and created a sourcetype for it:

 

[netlogon]
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
pulldown_type = 1

 


Now working on 9.2.0.1 but not on 9.1.2 😞

0 Karma
Get Updates on the Splunk Community!

Splunk Classroom Chronicles: Training Tales and Testimonials (Episode 2)

Welcome to the "Splunk Classroom Chronicles" series, created to help curious, career-minded learners get ...

Index This | I am a number but I am countless. What am I?

January 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  Happy New Year! We’re ...

What’s New in Splunk Enterprise 9.4: Tools for Digital Resilience

PLATFORM TECH TALKS What’s New in Splunk Enterprise 9.4: Tools for Digital Resilience Thursday, February 27, ...