Getting Data In

Log file with CRLF not producing multiple indexed events

shocko
Contributor

I'm using Splunk Enterprise 9 on Windows Server 2019 and monitoring a simple log file that has CRLF lines endings and is encoded as UTF8. My inputs stanza is as follows:

 

[monitor://c:\windows\debug\test.log]
disabled = 0
sourcetype = my_sourcetype
index=test

 

Consider two consectuive lines in the log file

 

Some data 1
Some data 2

 

When indexed this creates a single event rather than my expectation of 2 events.

Where am I going wrong?

 

 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What are the props.conf settings for [mysourcetype]?

---
If this reply helps you, Karma would be appreciated.
0 Karma

shocko
Contributor

I dont have one as I didn't think I needed one for something this simple. I have tried just now though adding this to no avail

 

[my_sourcetype]
SHOULD_LINEMERGE = FALSE
LINE_BREAKER = ([\r\n]+)

 

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Hi
This should work. If you are looking spec file for props.conf you see that SHOULD_LINEMERGE = true for unknown reason? It should be false for almost 100% of cases.
r. Ismo
0 Karma

shocko
Contributor

Is there any tooling (btool perhaps) that would tell me what props/transfroms are being applied to my sourcetype? Even if I drop the sourcetype form my inputs.conf the issue perists 

0 Karma

isoutamo
SplunkTrust
SplunkTrust

You could use btool to look what is applied to your sourcetype, BUT if there is also apply to source or host something those will override sourcetype definitions. Unfortunately I don't know if there is any tool which can show to you which of those are applied ;-(

0 Karma

shocko
Contributor

So I spun up a new Splunk instance in Podman (completely clean) and ingested the same file and the behaviourt is the same with no line breaking! This is with UTF8 encoding and CRLF or LF endings. So I went into the UI and created a sourcetype for it:

 

[netlogon]
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
pulldown_type = 1

 


Now working on 9.2.0.1 but not on 9.1.2 😞

0 Karma
Get Updates on the Splunk Community!

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Get Inspired! We’ve Got Validation that Your Hard Work is Paying Off

We love our Splunk Community and want you to feel inspired by all your hard work! Eric Fusilero, our VP of ...

What's New in Splunk Enterprise 9.4: Features to Power Your Digital Resilience

Hey Splunky People! We are excited to share the latest updates in Splunk Enterprise 9.4. In this release we ...