topic Re: multiline log: break on return char, not timestamp in Dashboards & Visualizations

multiline log: break on return char, not timestamp

alexmartinez — Thu, 16 Nov 2017 14:02:16 GMT

Sorry new to Splunk...I've a single logfile with entries that look like this:

"15/11/2017 20:20:59","0","1803.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:00","0","1260.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:00","0","2415.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:01","0","134.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:01","0","808.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:02","0","261.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:02","0","646.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:03","0","1157.xml","Copied to Amazon S3",5,"O"

Splunk is breaking this into events by timestamp (field 1) but because the above entries have repeating timestamps I only get the first event for each date.

How can I insure that EACH line gets its own event?

Re: multiline log: break on return char, not timestamp

niketn — Thu, 16 Nov 2017 18:44:10 GMT

@alexmartinez, what is your current props.conf file settings for this sourcetype?

If you want to break events on every line you should turn off line merge setting:

SHOULD_LINEMERGE=false

Refer to Splunk Documentation: https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureeventlinebreaking

Re: multiline log: break on return char, not timestamp

alexmartinez — Tue, 29 Sep 2020 16:53:41 GMT

In my edit Source TYpe/advanced settings on the console for Splunk Enterprrse:

BREAK_ONLY_BEFORE ([\n]+)
FIELD_NAMES logtimestamp,is_control_message,filename_message,status,file_status,error_type
INDEXED_EXTRACTIONS csv
NO_BINARY_CHECK true
SHOULD_LINEMERGE true
TIMESTAMP_FIELDS logtimestamp
TIME_FORMAT %d/%m/%Y %H:%M:%S
category Structured
disabled false
pulldown_type true

I tried:
SHOULD_LINEMERGE = false as an admin user in the console but it reverts back to true! I also tried removing the attribute BREAK_ONLY_BEFORE but it won't let me. Can't I edit Advanced settings?

Re: multiline log: break on return char, not timestamp

alexmartinez — Tue, 29 Sep 2020 16:49:54 GMT

Thanks niketnilay

I had to add a new sourcetype with
1. SHOULD_LINEMERGE = false
2 removed BREAK_ONLY_BEFORE attribute

...then pointed our data to this new sourcetype. Worked a treat.

Alex

Re: multiline log: break on return char, not timestamp

alexmartinez — Tue, 29 Sep 2020 16:53:52 GMT

Thanks @niketn that worked when i configured a new sourcetype with that attribute and also removed the BREAK_ONLY_BEFORE attribute . Then I pointed the datasource to this new sourcetype. Its all working now as expected.

Re: multiline log: break on return char, not timestamp

niketn — Fri, 17 Nov 2017 14:43:43 GMT

@alexmartinez, I have converted to answer, please accept.

Re: multiline log: break on return char, not timestamp

niketn — Fri, 24 Nov 2017 04:34:41 GMT

@alexmartinez, please accept the answer if your issue is resolved.