Dashboards & Visualizations

multiline log: break on return char, not timestamp

alexmartinez
New Member

Sorry new to Splunk...I've a single logfile with entries that look like this:

"15/11/2017 20:20:59","0","1803.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:00","0","1260.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:00","0","2415.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:01","0","134.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:01","0","808.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:02","0","261.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:02","0","646.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:03","0","1157.xml","Copied to Amazon S3",5,"O"

Splunk is breaking this into events by timestamp (field 1) but because the above entries have repeating timestamps I only get the first event for each date.

How can I insure that EACH line gets its own event?

Tags (1)
0 Karma

niketn
Legend

@alexmartinez, what is your current props.conf file settings for this sourcetype?

If you want to break events on every line you should turn off line merge setting:

SHOULD_LINEMERGE=false

Refer to Splunk Documentation: https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureeventlinebreaking

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

alexmartinez
New Member

Thanks @niketn that worked when i configured a new sourcetype with that attribute and also removed the BREAK_ONLY_BEFORE attribute . Then I pointed the datasource to this new sourcetype. Its all working now as expected.

0 Karma

niketn
Legend

@alexmartinez, I have converted to answer, please accept.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

@alexmartinez, please accept the answer if your issue is resolved.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

alexmartinez
New Member

Thanks niketnilay

I had to add a new sourcetype with
1. SHOULD_LINEMERGE = false
2 removed BREAK_ONLY_BEFORE attribute

...then pointed our data to this new sourcetype. Worked a treat.

Alex

0 Karma

alexmartinez
New Member

In my edit Source TYpe/advanced settings on the console for Splunk Enterprrse:

BREAK_ONLY_BEFORE ([\n]+)
FIELD_NAMES logtimestamp,is_control_message,filename_message,status,file_status,error_type
INDEXED_EXTRACTIONS csv
NO_BINARY_CHECK true
SHOULD_LINEMERGE true
TIMESTAMP_FIELDS logtimestamp
TIME_FORMAT %d/%m/%Y %H:%M:%S
category Structured
disabled false
pulldown_type true

I tried:
SHOULD_LINEMERGE = false as an admin user in the console but it reverts back to true! I also tried removing the attribute BREAK_ONLY_BEFORE but it won't let me. Can't I edit Advanced settings?

0 Karma
Get Updates on the Splunk Community!

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Unleash Unified Security and Observability with Splunk Cloud Platform

     Now Available on Microsoft AzureThursday, March 27, 2025  |  11AM PST / 2PM EST | Register NowStep boldly ...

Splunk AppDynamics with Cisco Secure Application

Web applications unfortunately present a target rich environment for security vulnerabilities and attacks. ...