topic Re: How to split multi-line events at search time? in Splunk Search

How to split multi-line events at search time?

aa123s — Wed, 14 Feb 2018 17:26:52 GMT

I have events that look like this:

[abc] logline1
[def] logline 2
[ghi] logline 3

and I would like to split those events at search time into 3 single line events.
Is that possible?

Thanks!

P.S.
I Know this should be done at Indexer / Heavy Forwarder level using LINE_BREAKER, but that's not an option at this time.

Re: How to split multi-line events at search time?

adonio — Wed, 14 Feb 2018 17:49:09 GMT

hello there,

maybe try the mvexpand command
check i tout:
https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Mvexpand

Re: How to split multi-line events at search time?

aa123s — Wed, 14 Feb 2018 19:14:34 GMT

Before posting I tried this:
| rex mode=sed "s/([\r\n]+)/##LF##/g" | makemv _raw delim="##LF##" | mvexpand _raw

but I couldn't make it work. Events are joined in a long string separated by ##LF##, but then those lines don't split back into separate events

Re: How to split multi-line events at search time?

somesoni2 — Wed, 14 Feb 2018 19:20:21 GMT

Try like this. The mvexpand command doesn't seem to work with fields starting with underscore.

your base search | rex mode=sed "s/([\r\n]+)/##LF##/g" | makemv _raw delim="##LF##" | rename _raw as raw | mvexpand raw | rename raw as _raw

Re: How to split multi-line events at search time?

micahkemp — Wed, 14 Feb 2018 19:36:14 GMT

Referring to your previous question:

https://answers.splunk.com/answers/618398/why-is-splunk-not-breaking-each-log-line-into-sing.html#answer-619402

I strongly suggest working now to get these logs indexed properly instead of trying to solve this problem at search time. You will end up being frustrated time and time again if your events are not indexed properly.

Re: How to split multi-line events at search time?

aa123s — Wed, 14 Feb 2018 19:49:56 GMT

Super! Its almost working: the remaining problem is that lines are being re-grouped in reverse order... Could that be fixed? Thanks!

Re: How to split multi-line events at search time?

aa123s — Wed, 14 Feb 2018 19:54:02 GMT

I got that, Thanks. We are already working to add correct indexing at forwarder level. In the mean time, however, we need this workaround.

Re: How to split multi-line events at search time?

aa123s — Wed, 14 Feb 2018 19:58:36 GMT

I'm sorry my comment was incomplete. I meant rows are being re-grouped in reverse order when I pipe the output of your solution to transaction ...
It normally doesn't happen

Re: How to split multi-line events at search time?

aa123s — Wed, 14 Feb 2018 20:05:00 GMT

I ended up adding | reverse at the end... go figure why that happens!...
Thanks a lot!

Re: How to split multi-line events at search time?

msquicc — Wed, 25 May 2022 15:28:18 GMT

here's what I came up with. seems to work pretty well without modifying the data: