Splunk Search

Grouping the request and response as a transaction

Jananee_iNautix
Explorer

We have a log which can be grouped as a transaction. The transaction will have the following events:
2014/08/07 10:17:18.947 High Messaging.Message.MessageIgnored CoreId(ci1407421015292.1078088@rs22_te) RefToCoreId(ci1407420962070.1077899@rs22_te)
2014/08/07 10:16:55.758 High Messaging.Message.MessageReceived CoreId(ci1407421015292.1078088@rs22_te) RefToCoreId(ci1407420962070.1077899@rs22_te)
2014/08/07 10:16:55.754 High Messaging.Message.ResponseDelivered CoreId(ci1407420962070.1077899@rs22_te) RefToCoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:55.298 High Messaging.Message.ResponseSent CoreId(ci1407420962070.1077899@rs22_te) RefToCoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:31.137 High Messaging.Message.PayloadDelivered CoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:28.836 High Messaging.Message.MessageSent CoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:28.542 Low Messaging.Message.MessagePackaged.Response CoreId(ci1407420962070.1077899@rs22_te) RefToCoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:02.298 Low Messaging.Message.MessagePackaged.Request CoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:16:02.003 Low Messaging.Message.MessageUnpackaged.Request CoreId(ci1407420933327.1077789@rs22_te)
2014/08/07 10:15:33.333 High Messaging.Message.MessageReceived CoreId(ci1407420933327.1077789@rs22_te)

In the above transaction, some are releated to request details and some are related to response details. Request details are referred uniquely by "CoreId". Whereas for the corresponding Response, "RefToCoreId" will have the value of CoreId from Request details and a new "CoreId" will be generated.
In the example above, for Request details the CoreId is "ci1407420933327.1077789@rs22_te" and for Response the CoreId is "ci1407420962070.1077899@rs22_te" and RefToCoreId is "ci1407420933327.1077789@rs22_te".

Problem: TO uniquely identify this as a transaction, we cannot use the CoreId alone. Because the CoreId for Request and corresponding Response are different. But CoreId in Request and RefToCoreId in Response are of same value.We have to somehow group the request ans response events together.Could you please let us know how to do this?

I'm attaching a sample full event for reference that contains both Request and Response details :
2014/08/07 10:16:55.298 High Messaging.Message.ResponseSent CoreId(ci1407420962234) MessageId(1407420988182) RefToCoreId(ci1407420933327) SenderPartyName(ABC) ReceiverPartyName(GFT) SenderRoutingId(01043000112) ReceiverRoutingId(01002206234) Direction(Outbound) MessageState(Produced) FinalState(false) MessageSize(2830) BackupFilename(/opt/enterprise/ci1407420962234_te_Produced_5113692) PeerAddress(http://newdomain.com:8080/enterprise/as2) SignatureDigest(SHA1) Protocol(AS2) ProtocolVersion(1.0) SendAttempt(1) MaxSendAttempts(3) ResendInterval(3600000) Transport(HTTP)

2014/08/07 10:16:28.836 High Messaging.Message.MessageSent CoreId(ci1407420933327.1077789@rs22_te) SenderPartyName(Eastman Kodak) ReceiverPartyName(Mellon Financial) SenderRoutingId(01002206234) ReceiverRoutingId(01043000112) Direction(Inbound) MessageState(Produced) FinalState(false) MessageSize(254) BackupFilename(/opt/enterprise/common/data/backup/rs22_te/month08/d07/h10/ci1407420933327.1077789@rs22_te_Produced_5113689) PeerAddress(/opt/enterprise/inbound/DMVRDATA) ProductionFilename(z24b4f654_258e2) MimeType(application/PDF) Protocol(RAW) ProtocolVersion(1.0) SendAttempt(1) MaxSendAttempts(3) ResendInterval(3600000) Transport(FILESYSTEM) DocumentId(000001514)

Tags (1)
0 Karma
1 Solution

felipetesta
Path Finder

I would use the coalesce command:

    | eval TempID=coalesce(RefToCoreId, CoreId) | transaction TempID

Mind that your "sample full events" have different values for CoreId:

ResponseSent RefToCoreId(ci1407420933327)
MessageSent CoreId(ci1407420933327.1077789@rs22_te)

View solution in original post

0 Karma

felipetesta
Path Finder

I would use the coalesce command:

    | eval TempID=coalesce(RefToCoreId, CoreId) | transaction TempID

Mind that your "sample full events" have different values for CoreId:

ResponseSent RefToCoreId(ci1407420933327)
MessageSent CoreId(ci1407420933327.1077789@rs22_te)

0 Karma

Jananee_iNautix
Explorer

There is a slight modification. Instead of coalesce when it is |eval TempID = if(isnull(RefCoreID),CoreID,RefCoreID)|transaction TempID it works fine..Thanks for the idea anyways...

0 Karma

NGRhodes
Explorer

The trick is to evaluate a field that holds a common value between requests and responses.
As you only get RefToCoreId in select events, I would do 2 transactions, first by coreid and then by the generated field to pull the requests and responses together.

e.g. after your initial search (not sytax checked) add the following:

|
eval TempID=if(searchmatch("Messaging.Message.MessageReceived"), CoreId, if(searchmatch("Messaging.Message.MessagePackaged.Response"), RefToCoreId, NULL) | 
transaction CoreId | 
transaction TempID
0 Karma
Get Updates on the Splunk Community!

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...