Dashboards & Visualizations

multiline log: break on return char, not timestamp

alexmartinez
New Member

Sorry new to Splunk...I've a single logfile with entries that look like this:

"15/11/2017 20:20:59","0","1803.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:00","0","1260.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:00","0","2415.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:01","0","134.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:01","0","808.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:02","0","261.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:02","0","646.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:03","0","1157.xml","Copied to Amazon S3",5,"O"

Splunk is breaking this into events by timestamp (field 1) but because the above entries have repeating timestamps I only get the first event for each date.

How can I insure that EACH line gets its own event?

Tags (1)
0 Karma

niketn
Legend

@alexmartinez, what is your current props.conf file settings for this sourcetype?

If you want to break events on every line you should turn off line merge setting:

SHOULD_LINEMERGE=false

Refer to Splunk Documentation: https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureeventlinebreaking

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

alexmartinez
New Member

Thanks @niketn that worked when i configured a new sourcetype with that attribute and also removed the BREAK_ONLY_BEFORE attribute . Then I pointed the datasource to this new sourcetype. Its all working now as expected.

0 Karma

niketn
Legend

@alexmartinez, I have converted to answer, please accept.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

@alexmartinez, please accept the answer if your issue is resolved.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

alexmartinez
New Member

Thanks niketnilay

I had to add a new sourcetype with
1. SHOULD_LINEMERGE = false
2 removed BREAK_ONLY_BEFORE attribute

...then pointed our data to this new sourcetype. Worked a treat.

Alex

0 Karma

alexmartinez
New Member

In my edit Source TYpe/advanced settings on the console for Splunk Enterprrse:

BREAK_ONLY_BEFORE ([\n]+)
FIELD_NAMES logtimestamp,is_control_message,filename_message,status,file_status,error_type
INDEXED_EXTRACTIONS csv
NO_BINARY_CHECK true
SHOULD_LINEMERGE true
TIMESTAMP_FIELDS logtimestamp
TIME_FORMAT %d/%m/%Y %H:%M:%S
category Structured
disabled false
pulldown_type true

I tried:
SHOULD_LINEMERGE = false as an admin user in the console but it reverts back to true! I also tried removing the attribute BREAK_ONLY_BEFORE but it won't let me. Can't I edit Advanced settings?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...