Splunk Grouping not works always

[nkannan1984]

I am using the following splunk query

to combine the events in to one transaction based on the referenceid.
It works fine for most of the transactions ... but some transactions are grouped not based on matching referenceid
Kindly let me know what i am missing here..
Thanks in advance

index="xxxx" host="yyyy.local" referenceid | transaction startswith="SplunkLoggingStart" endswith="SplunkLoggingEnd" | convert ctime(_time) as TransactionTime

26 Jul 2017 11:16:00,833 [WARN ] {Route Executor 0} (Route.50.Filter.39) SplunkLoggingStart ReferenceId: f680f964-7057-3b9b-a5d6-f07767238e91

26 Jul 2017 11:16:00,955 [WARN ] {Route Executor 7} (Route.50.Filter.39) Step 1 InputRequest ReferenceId: f680f964-7057-3b9b-a5d6-f07767238e91

6 Jul 2017 11:15:58,127 [WARN ] {Route Executor 7} (Route.50.Filter.46) Step 4 OutputResponse ReferenceId: f680f964-7057-3b9b-a5d6-f07767238e91

26 Jul 2017 11:16:03,086 [WARN ] {Route Executor 3} (Route.50.Filter.46) SplunkLoggingEnd ReferenceId: f680f964-7057-3b9b-a5d6-f07767238e91

[cpetterborg]

If this worked for you, please accept the answer so that others finding this question will know what will work for them.

Thanks!

[cpetterborg · Accepted solution]

If your ReferenceId numbers are unique, then you don't have to use startswith and endswith. You can just use the following search:

index="xxxx" host="yyyy.local" referenceid |rex "ReferenceId:\s+(?<referenceid>.*)" | transaction referenceid | table duration, referenceid, _time

That will give you the time of the transaction and the duration. You are including all events that have "referenceid" which is the string it is looking for, so you get all of them that have ReferenceId, too, but you probably aren't getting them field extracted (which I do in the rex in my search), then you tell it to group the events together by the referenceid that was extracted. The _time and, if you need it, the duration of the transaction. Transaction wants to know what to group together, not just the startswith and endswith, so you tell it the referenceid.

[nkannan1984]

Thanks this was very helpful .I ended up with something like this

 index="xxxx" host="yyyy.local" referenceid | rex field=_raw "(?<ReferenceID>\b[a-f0-9]{8}(?:-[a-f0-9]{4}){3}-[a-f0-9]{12}\b)" | transaction ReferenceID