topic Re: How to split multiple lines of data into individual lines? in Getting Data In

How to split multiple lines of data into individual lines?

otman01 — Mon, 06 Jul 2015 14:13:32 GMT

Hello every one,

I have some data in my Splunk server that is not separated correctly. I want to split this data into lines. Below a sample of my data:

06/07/15 15:39:11,000       zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=251;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=03;time=614;mark=Iphone;zone CL=Product;AC=resp;MID=AS5952 ;code=00;time=506;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=274;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=00;time=892;mark=Iphone;zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=256;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=00;time=623;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=281;mark=samsung;

so what I want to do is to split this log like :

06/07/15 15:39:11,000  zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=251;mark=samsung;
06/07/15 15:39:11,000  zone CL=Product;AC=resp;MID=AS5952 ;code=03;time=614;mark=Iphone;
06/07/15 15:39:11,000   zone CL=Product;AC=resp;MID=AS5952 ;code=00;time=506;mark=samsung;
06/07/15 15:39:11,000   zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=274;mark=samsung;

any help please. thank you .

Re: How to split multiple lines of data into individual lines?

woodcock — Mon, 06 Jul 2015 17:15:45 GMT

I am assuming that what you posted was one event and that you are succesfully sending these "bunched" events into Splunk already; if so, use something like this when you need to break them apart at search time:

...  | rex max_match=0 field=_raw "(?<lineData>zone.*?mark=[^;]+)" | mvexpand lineData | fields lineData

If you have your timestmaping working correctly, each event will have the correct timestamp.

Re: How to split multiple lines of data into individual lines?

bmacias84 — Tue, 29 Sep 2020 06:36:10 GMT

If his application is not line breaking the event, essentially printing multiple line into a single line they would merge. If this is the case you could use BREAK_ONLY_BEFORE in the props.conf.

Re: How to split multiple lines of data into individual lines?

otman01 — Tue, 29 Sep 2020 06:36:15 GMT

how can I use this parameter BREAK_ONLY_BEFORE based on my log ?
thank you

Re: How to split multiple lines of data into individual lines?

otman01 — Tue, 29 Sep 2020 06:36:18 GMT

I used the parameter with this configuration
BREAK_ONLY_BEFORE=zone
but it doesn't work

Re: How to split multiple lines of data into individual lines?

otman01 — Tue, 07 Jul 2015 10:26:15 GMT

Can I reindex my data without delete it? because I can't delete data present in the server

Re: How to split multiple lines of data into individual lines?

woodcock — Tue, 07 Jul 2015 14:06:26 GMT

He is saying at index-time, you can configure Splunk to break events into multiple events so that you do not have to do it at search-time. The "problem" with this approach is that each line does not have it's own timestmamp so this will cause Splunk to issue a warning in the log for each sub-event after the first in a clump with text like this:

WARN  DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event.

It is harmless, but since it does the correct thing (use previous event's timestamp), it will work for you.

Re: How to split multiple lines of data into individual lines?

peter_krammer — Thu, 11 Feb 2016 16:00:07 GMT

We also just had this problem. I added SHOULD_LINEMERGE = false to my props.conf but all earlier events were naturally still mashed together. I wrote an all purpose query to split the lines based on previous answer, but with added bonus that all field extractions for the sourcetype work fine after putting the splitted data into the _raw field.

| rex max_match=0 field=_raw "(?<lineData>[^\n]+)" | mvexpand lineData | eval _raw=lineData

Re: How to split multiple lines of data into individual lines?

RichaSingh — Mon, 30 May 2016 19:17:24 GMT

This one was such a saviour !

Thanks for sharing this...

Re: How to split multiple lines of data into individual lines?

woodcock — Mon, 30 May 2016 23:44:56 GMT

Yes, but you will have to clear the fishbucket first.

Re: How to split multiple lines of data into individual lines?

MuS — Tue, 31 May 2016 00:17:29 GMT

Just in case the next question will be How can I clear the fishbucket?

Please find the docs on how to remove a file from the fishbucket using btprobe here http://docs.splunk.com/Documentation/Splunk/6.4.0/Troubleshooting/CommandlinetoolsforusewithSupport#btprobe or How to clean the fishbucket here http://docs.splunk.com/Documentation/Splunk/6.4.0/Indexer/RemovedatafromSplunk#Remove_data_from_one_or_all_indexes

cheers, MuS

Re: How to split multiple lines of data into individual lines?

nick405060 — Fri, 27 Jul 2018 18:43:24 GMT

Thanks this worked!!