topic Re: Log file with CRLF not producing multiple indexed events in Getting Data In

Log file with CRLF not producing multiple indexed events

shocko — Wed, 13 Mar 2024 15:24:48 GMT

I'm using Splunk Enterprise 9 on Windows Server 2019 and monitoring a simple log file that has CRLF lines endings and is encoded as UTF8. My inputs stanza is as follows:

[monitor://c:\windows\debug\test.log] disabled = 0 sourcetype = my_sourcetype index=test

Consider two consectuive lines in the log file

Some data 1 Some data 2

When indexed this creates a single event rather than my expectation of 2 events.

Where am I going wrong?

Re: Log file with CRLF not producing multiple indexed events

richgalloway — Wed, 13 Mar 2024 16:41:06 GMT

What are the props.conf settings for [mysourcetype]?

Re: Log file with CRLF not producing multiple indexed events

shocko — Wed, 13 Mar 2024 20:07:23 GMT

I dont have one as I didn't think I needed one for something this simple. I have tried just now though adding this to no avail

[my_sourcetype] SHOULD_LINEMERGE = FALSE LINE_BREAKER = ([\r\n]+)

Re: Log file with CRLF not producing multiple indexed events

isoutamo — Wed, 13 Mar 2024 21:27:16 GMT

Hi
This should work. If you are looking spec file for props.conf you see that SHOULD_LINEMERGE = true for unknown reason? It should be false for almost 100% of cases.
r. Ismo

Re: Log file with CRLF not producing multiple indexed events

shocko — Thu, 14 Mar 2024 12:12:44 GMT

Is there any tooling (btool perhaps) that would tell me what props/transfroms are being applied to my sourcetype? Even if I drop the sourcetype form my inputs.conf the issue perists

Re: Log file with CRLF not producing multiple indexed events

isoutamo — Thu, 14 Mar 2024 15:49:49 GMT

You could use btool to look what is applied to your sourcetype, BUT if there is also apply to source or host something those will override sourcetype definitions. Unfortunately I don't know if there is any tool which can show to you which of those are applied ;-(

Re: Log file with CRLF not producing multiple indexed events

shocko — Wed, 20 Mar 2024 12:21:29 GMT

So I spun up a new Splunk instance in Podman (completely clean) and ingested the same file and the behaviourt is the same with no line breaking! This is with UTF8 encoding and CRLF or LF endings. So I went into the UI and created a sourcetype for it:

[netlogon] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true SHOULD_LINEMERGE = false category = Custom pulldown_type = 1

Now working on 9.2.0.1 but not on 9.1.2 😞