Grouping the request and response as a transaction

[Jananee_iNautix]

We have a log which can be grouped as a transaction. The transaction will have the following events:
2014/08/07 10:17:18.947 High Messaging.Message.MessageIgnored CoreId(ci1407421015292.1078088@rs22_te) RefToCoreId(ci1407420962070.1077899@rs22_te)
2014/08/07 10:16:55.758 High Messaging.Message.MessageReceived CoreId(ci1407421015292.1078088@rs22_te) RefToCoreId(ci1407420962070.1077899@rs22_te)
2014/08/07 10:16:55.754 High Messaging.Message.ResponseDelivered CoreId(ci1407420962070.1077899@rs22_te) RefToCoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:55.298 High Messaging.Message.ResponseSent CoreId(ci1407420962070.1077899@rs22_te) RefToCoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:31.137 High Messaging.Message.PayloadDelivered CoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:28.836 High Messaging.Message.MessageSent CoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:28.542 Low Messaging.Message.MessagePackaged.Response CoreId(ci1407420962070.1077899@rs22_te) RefToCoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:02.298 Low Messaging.Message.MessagePackaged.Request CoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:02.003 Low Messaging.Message.MessageUnpackaged.Request CoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:15:33.333 High Messaging.Message.MessageReceived CoreId(ci1407420933327.1077789@rs22_te)

In the above transaction, some are releated to request details and some are related to response details. Request details are referred uniquely by "CoreId". Whereas for the corresponding Response, "RefToCoreId" will have the value of CoreId from Request details and a new "CoreId" will be generated.
In the example above, for Request details the CoreId is "ci1407420933327.1077789@rs22_te" and for Response the CoreId is "ci1407420962070.1077899@rs22_te" and RefToCoreId is "ci1407420933327.1077789@rs22_te".

Problem: TO uniquely identify this as a transaction, we cannot use the CoreId alone. Because the CoreId for Request and corresponding Response are different. But CoreId in Request and RefToCoreId in Response are of same value.We have to somehow group the request ans response events together.Could you please let us know how to do this?

I'm attaching a sample full event for reference that contains both Request and Response details :
2014/08/07 10:16:55.298 High Messaging.Message.ResponseSent CoreId(ci1407420962234) MessageId(1407420988182) RefToCoreId(ci1407420933327) SenderPartyName(ABC) ReceiverPartyName(GFT) SenderRoutingId(01043000112) ReceiverRoutingId(01002206234) Direction(Outbound) MessageState(Produced) FinalState(false) MessageSize(2830) BackupFilename(/opt/enterprise/ci1407420962234_te_Produced_5113692) PeerAddress(http://newdomain.com:8080/enterprise/as2) SignatureDigest(SHA1) Protocol(AS2) ProtocolVersion(1.0) SendAttempt(1) MaxSendAttempts(3) ResendInterval(3600000) Transport(HTTP)

2014/08/07 10:16:28.836 High Messaging.Message.MessageSent CoreId(ci1407420933327.1077789@rs22_te) SenderPartyName(Eastman Kodak) ReceiverPartyName(Mellon Financial) SenderRoutingId(01002206234) ReceiverRoutingId(01043000112) Direction(Inbound) MessageState(Produced) FinalState(false) MessageSize(254) BackupFilename(/opt/enterprise/common/data/backup/rs22_te/month08/d07/h10/ci1407420933327.1077789@rs22_te_Produced_5113689) PeerAddress(/opt/enterprise/inbound/DMVRDATA) ProductionFilename(z24b4f654_258e2) MimeType(application/PDF) Protocol(RAW) ProtocolVersion(1.0) SendAttempt(1) MaxSendAttempts(3) ResendInterval(3600000) Transport(FILESYSTEM) DocumentId(000001514)

[felipetesta · Accepted solution]

I would use the coalesce command:

    | eval TempID=coalesce(RefToCoreId, CoreId) | transaction TempID

Mind that your "sample full events" have different values for CoreId:

ResponseSent RefToCoreId(ci1407420933327)
MessageSent CoreId(ci1407420933327.1077789@rs22_te)

[Jananee_iNautix]

There is a slight modification. Instead of coalesce when it is |eval TempID = if(isnull(RefCoreID),CoreID,RefCoreID)|transaction TempID it works fine..Thanks for the idea anyways...

[NGRhodes]

The trick is to evaluate a field that holds a common value between requests and responses.
As you only get RefToCoreId in select events, I would do 2 transactions, first by coreid and then by the generated field to pull the requests and responses together.

e.g. after your initial search (not sytax checked) add the following:

|
eval TempID=if(searchmatch("Messaging.Message.MessageReceived"), CoreId, if(searchmatch("Messaging.Message.MessagePackaged.Response"), RefToCoreId, NULL) | 
transaction CoreId | 
transaction TempID