topic Re: How to join multiple log streams together in Splunk Search

How to join multiple log streams together

robgreen — Sun, 11 Dec 2011 23:27:11 GMT

We have our logs always generate a sessionid but each host has a separate sessionid with a link to the original as parentsessionid. I am trying to get a splunk query for when a session id is entered it will return back all the logs for all the hosts following the child parent relationship.

something like this

hostA: sessionid=aaa parentsessionid=null name=value name2=value2
hostB: sessionid=bbb parentsessionid=aaa name=valuexx name2=value2
hostC: sessionid=ccc parentsessionid=bbb name=valueyy name3=value3

if someone enters aaa i would like all three sessions to be returned in the query in order (as the session is running on all 3 at roughly the same time and in general one event on one host causes another event on a different host)

rob

Re: How to join multiple log streams together

Takajian — Sun, 11 Dec 2011 23:38:26 GMT

Transaction command may be help for your case. This command can group events into transactions. Please try following command and confirm if this work or not.

sourcetype= | transaction sessionid parentsessionid maxspan=1m

Re: How to join multiple log streams together

robgreen — Sun, 11 Dec 2011 23:43:59 GMT

when i do
search sessionid=aaa i get 30 events, when i do
search sessionid=bbb i get 20 events, when i do
search sessionid=ccc i get 10 events. all good so far.

when i do the above i get 1 event when i am expecting 60 events.

Re: How to join multiple log streams together

Takajian — Sun, 11 Dec 2011 23:51:48 GMT

If you expect 60 events in your case, transaction command will not help although I thought it may help. Transaction command group events into transaction. What do you mean "how to join multiple log streams together"? I thought you want to group multiple events into a transaction.

Re: How to join multiple log streams together

robgreen — Mon, 12 Dec 2011 00:03:56 GMT

even though the user only specified to search for "aaa" i want to include all events that are descended from it. ie ccc has no reference to aaa directly but bbb references both aaa and ccc. i am fine if it can only be its direct ancestor.. originally i thought i could do something like
search sessionid=aaa | join sessionid [search parentsessionid=]
but i dont see a way to reference a field from a previous search in the pipeline..

rob

Re: How to join multiple log streams together

Takajian — Mon, 12 Dec 2011 00:09:52 GMT

Following command will work? I still think transaction command will help.

sourcetype= | transaction sessionid parentsessionid maxspan=1m startswith=hostA endswith=hostC