topic Re: Counting a field for number of messages per unique value in Splunk Search

Counting a field for number of messages per unique value

sysprg1 — Tue, 25 Sep 2012 21:18:10 GMT

I have transactions being logged to Splunk, but I get multiple messages per transaction.
We are in the middle tier and the 4 messages are as follows:
Point 1 - We receive a request from a requester with a unique identifier.
Point 2 - We send the request to a backend system
Point 3 - We receive a response from a backend system
Point 4 - We send the response to the requester.

Having all 4 points would indicate a successful transaction. If we don't get all 4 points, then it is a failed request. What I've got is the count of the points based on unique identifier, but I need a Total Count of transaction (total # of unique IDs), count of Success and count of failures.

Thanks in advance for your help.

Re: Counting a field for number of messages per unique value

Ayn — Tue, 25 Sep 2012 22:17:02 GMT

You don't say where in the process you're stuck, but in this answer I'm assuming the following:

The unique identifier is present in all points 1-4 so that they can be identified correctly.
The unique identifier is extracted to some field in Splunk (let's call it txnid in this example)

If you've got that far, this is my suggestion on how to proceed. Run transaction to group the events together. transaction will, among other things it does, create two fields called eventcount and duration. eventcount is exactly what it says - the number of events in each transaction. So you could build your transactions, then check which ones don't have exactly 4 events in them, and the ones that do have it. This can then be used in your stats calculation.

... | transaction txnid | stats dc(txnid) as total_txn_count, count(eval(eventcount=4)) as txn_success, count(eval(eventcount!=4)) as txn_fail

Re: Counting a field for number of messages per unique value

sysprg1 — Wed, 26 Sep 2012 13:00:18 GMT

That worked perfectly. Thanks.