Hello every one,
I have some data in my Splunk server that is not separated correctly. I want to split this data into lines. Below a sample of my data:
06/07/15 15:39:11,000 zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=251;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=03;time=614;mark=Iphone;zone CL=Product;AC=resp;MID=AS5952 ;code=00;time=506;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=274;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=00;time=892;mark=Iphone;zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=256;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=00;time=623;mark=samsung;zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=281;mark=samsung;
so what I want to do is to split this log like :
06/07/15 15:39:11,000 zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=251;mark=samsung;
06/07/15 15:39:11,000 zone CL=Product;AC=resp;MID=AS5952 ;code=03;time=614;mark=Iphone;
06/07/15 15:39:11,000 zone CL=Product;AC=resp;MID=AS5952 ;code=00;time=506;mark=samsung;
06/07/15 15:39:11,000 zone CL=Product;AC=resp;MID=AS5952 ;code=57;time=274;mark=samsung;
any help please. thank you .
I am assuming that what you posted was one event and that you are succesfully sending these "bunched" events into Splunk already; if so, use something like this when you need to break them apart at search time:
... | rex max_match=0 field=_raw "(?<lineData>zone.*?mark=[^;]+)" | mvexpand lineData | fields lineData
If you have your timestmaping
working correctly, each event will have the correct timestamp.
We also just had this problem. I added SHOULD_LINEMERGE = false to my props.conf but all earlier events were naturally still mashed together. I wrote an all purpose query to split the lines based on previous answer, but with added bonus that all field extractions for the sourcetype work fine after putting the splitted data into the _raw field.
| rex max_match=0 field=_raw "(?<lineData>[^\n]+)" | mvexpand lineData | eval _raw=lineData
Thanks this worked!!
I am assuming that what you posted was one event and that you are succesfully sending these "bunched" events into Splunk already; if so, use something like this when you need to break them apart at search time:
... | rex max_match=0 field=_raw "(?<lineData>zone.*?mark=[^;]+)" | mvexpand lineData | fields lineData
If you have your timestmaping
working correctly, each event will have the correct timestamp.
Can I reindex my data without delete it? because I can't delete data present in the server
Yes, but you will have to clear the fishbucket
first.
Just in case the next question will be How can I clear the fishbucket?
Please find the docs on how to remove a file from the fishbucket using btprobe
here http://docs.splunk.com/Documentation/Splunk/6.4.0/Troubleshooting/CommandlinetoolsforusewithSupport#... or How to clean the fishbucket
here http://docs.splunk.com/Documentation/Splunk/6.4.0/Indexer/RemovedatafromSplunk#Remove_data_from_one_...
cheers, MuS
This one was such a saviour !
Thanks for sharing this...
If his application is not line breaking the event, essentially printing multiple line into a single line they would merge. If this is the case you could use BREAK_ONLY_BEFORE in the props.conf.
I used the parameter with this configuration
BREAK_ONLY_BEFORE=zone
but it doesn't work
He is saying at index-time
, you can configure Splunk to break events into multiple events so that you do not have to do it at search-time
. The "problem" with this approach is that each line does not have it's own timestmamp so this will cause Splunk to issue a warning in the log for each sub-event after the first in a clump with text like this:
WARN DateParserVerbose - Failed to parse timestamp. Defaulting to timestamp of previous event.
It is harmless, but since it does the correct thing (use previous event's timestamp), it will work for you.
how can I use this parameter BREAK_ONLY_BEFORE based on my log ?
thank you