topic How to combine two consecutive events into one based on the content of the first event? in Splunk Search

How to combine two consecutive events into one based on the content of the first event?

DavidHourani — Fri, 04 Dec 2015 14:11:46 GMT

Hello,

I would like to combine 2 events into one based on the content of the first one.

So every time I find an event containing the word "Banana" I wanna combine it with the line that follows regardless of what the following line is.

Could you please help out?

Thank you.
David

Re: How to combine two consecutive events into one based on the content of the first event?

jplumsdaine22 — Fri, 04 Dec 2015 14:20:11 GMT

Can you put some sample data here? The closer to your original data the better

Re: How to combine two consecutive events into one based on the content of the first event?

stephanefotso — Fri, 04 Dec 2015 14:20:22 GMT

Hello. You can do it through configuration files (props.conf and transforms.conf). Read this:
http://docs.splunk.com/Documentation/Splunk/6.3.1511/Data/Configureeventlinebreaking

Thanks

Re: How to combine two consecutive events into one based on the content of the first event?

sundareshr — Fri, 04 Dec 2015 15:30:46 GMT

You could try the transaction command. Something like this could work.

index=* | transaction startswith="banana" maxevents=2

Having said this, keep in mind the sort order in splunk may not be the same as what you are thinking. So what you think as the "next" event may not be what splunk considers to be the "next" event.

Re: How to combine two consecutive events into one based on the content of the first event?

DavidHourani — Fri, 04 Dec 2015 19:05:11 GMT

I think that should work. I did it with |transaction _time startswith="banana" endwith="the keyword in the next even" since the "Next" event was in chronological order worked fine for me ^^

Re: How to combine two consecutive events into one based on the content of the first event?

woodcock — Sat, 05 Dec 2015 19:51:36 GMT

You should try to avoid using transaction whenever you can. Try this instead (faster and more robust):

... "banana" OR "the keyword in the next event" | reverse | streamstats count(eval(searchmatch("banana"))) AS SessionID | reverse | stats list(_raw) AS events by SessionID

Re: How to combine two consecutive events into one based on the content of the first event?

DavidHourani — Mon, 07 Dec 2015 09:24:42 GMT

I'm getting 0 results with that search. I agree with you that transactions are slow and yes I think a better method would be to try to avoid it. How exactly does the Reverse command works ?

Re: How to combine two consecutive events into one based on the content of the first event?

woodcock — Tue, 08 Dec 2015 16:28:43 GMT

We need reverse so that as we work backwards through the list from top-to-bottom, we process the oldest events first, meaning that whenever we see a banana event, it marks the beginning of a new "session".

I made a mistake in that I used stats instead of streamstats. I have correct this in my original answer; try again.

Re: How to combine two consecutive events into one based on the content of the first event?

DavidHourani — Wed, 09 Dec 2015 12:10:40 GMT

its still not working for some reason... apparently SessionID is always empty...

Re: How to combine two consecutive events into one based on the content of the first event?

sundareshr — Wed, 09 Dec 2015 14:27:05 GMT

Try this

... "banana" OR "the keyword in the next event" | reverse | eval x=if(searchmatch("banana"), 1, 0) | streamstats sum(x) AS SessionID | reverse | stats list(_raw) AS events by SessionID

Re: How to combine two consecutive events into one based on the content of the first event?

woodcock — Wed, 09 Dec 2015 20:41:02 GMT

ARGH! I blew it again! I used count(searchmatch("banana")) instead of count(eval(searchmatch("banana"))). I have updated my answer again. If you care to retry, I am sure it will work this time!