Splunk Search

How to set up an alert if an ack message is not available for a particular req?

prashanthberam
Explorer

Hi,
i have messages like this how to setup an alert if ack message is not available in the logs for particular req.
and between req and rsp is more than 30 sec i need to setup an one more alert.

my logs like this:

2017-03-10 15:56:42.056 [WMQJCAResourceAdapter : 1] [INFO ] [DCN 0201706380692310C] SplunkLog - CorrelationID=000001806003698150190841, DCN=0201706380692310C, TransactionTimestamp=2017-03-10 15:56:37.742, GroupNumber =000Y69HB3, ServiceLinecount=4, SectionNumber=0008, CorporateEntityCode=OK1, ClaimType=0, VendorName=VERSCEND, VendorCode=CVP, TransactionCode=RSP, UtilizationAmount=3.75

2017-03-10 15:56:39.003 [WMQJCAResourceAdapter : 6] [INFO ] [DCN 0201706380692310C] SplunkLog - CorrelationID=000001806003698150190841, DCN=0201706380692310C, TransactionTimestamp=2017-03-10 15:56:39.002, GroupNumber =000Y69HB3, ServiceLinecount=4, SectionNumber=0008, CorporateEntityCode=OK1, ClaimType=0, VendorName=VERSCEND, VendorCode=CVP, TransactionCode=ACK, OutCome=C, Messagetext=ACCEPTED

2017-03-10 15:56:36.939 [WMQJCAResourceAdapter : 1] [INFO ] [DCN 0201706380692310C] SplunkLog - CorrelationID=000001806003698150190841, DCN=0201706380692310C, TransactionTimestamp=2017-03-10 15:56:36.939, GroupNumber =000Y69HB3, ServiceLinecount=4, SectionNumber=0008, CorporateEntityCode=OK1, ClaimType=0, VendorName=VERSCEND, VendorCode=CVP, TransactionCode=REQ
0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Assuming there is a unique transaction ID available in log for each req-ack-rsp combination, you could do like this (assuming CorrelationID is the unique identifier, if there are multiple columns add them to stats's by clause)

Updated mv funtion

Alert when there is no ACK event for a transaction

your base search fetching all records
| stats min(_time) as StartTime  max(_time) as EndTime values(TransactionCode) as TransactionCodes by CorrelationID
| eval _time=StartTime | where isnull(mvfilter(match(TransactionCodes,"ACK")))

Alert when transaction duration is more than 30 sec

your base search fetching all records
| stats min(_time) as StartTime  max(_time) as EndTime values(TransactionCode) as TransactionCodes by CorrelationID
| eval _time=StartTime  | eval duration=EndTime-StartTime | where mvcount(TransactionCodes)=3 AND duration>30

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Assuming there is a unique transaction ID available in log for each req-ack-rsp combination, you could do like this (assuming CorrelationID is the unique identifier, if there are multiple columns add them to stats's by clause)

Updated mv funtion

Alert when there is no ACK event for a transaction

your base search fetching all records
| stats min(_time) as StartTime  max(_time) as EndTime values(TransactionCode) as TransactionCodes by CorrelationID
| eval _time=StartTime | where isnull(mvfilter(match(TransactionCodes,"ACK")))

Alert when transaction duration is more than 30 sec

your base search fetching all records
| stats min(_time) as StartTime  max(_time) as EndTime values(TransactionCode) as TransactionCodes by CorrelationID
| eval _time=StartTime  | eval duration=EndTime-StartTime | where mvcount(TransactionCodes)=3 AND duration>30
0 Karma

prashanthberam
Explorer

while am searching first query am getting this Error in 'where' command: The arguments to the 'mvfind' function are invalid. what it means. may i know the reason.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Oops. Used wrong function. Just updated the query to use correct function.

0 Karma

prashanthberam
Explorer

Thanks for the help Somesoni2 . Now it's working ..

0 Karma

woodcock
Esteemed Legend

Like this:

Your Base Search Here | stats count list(_time) AS times range(_time) AS duration list(TransactionCode) AS TransactionCode BY  CorrelationID | search TransactionCode="REQ" AND NOT TransactionCode="ACK"

And this:

Your Base Search Here | stats count list(_time) AS times range(_time) AS duration list(TransactionCode) AS TransactionCode BY CorrelationID | search duration > 30 AND TransactionCode="REQ" AND TransactionCode="RSP"
0 Karma

woodcock
Esteemed Legend

I am done editing; sorry for the churn; I did not notice the 2nd part of the question.

0 Karma

prashanthberam
Explorer

ohh Np woodcock. thanks for helping.

0 Karma

woodcock
Esteemed Legend

I lied; I had an extra NOT in my 2nd answer. It is all good now.

0 Karma

prashanthberam
Explorer

ya i haven't noticed that one. thanks woodcock

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...