topic Re: Transaction - Related Events in Splunk Search

Transaction - Related Events

shonky — Wed, 14 Nov 2012 05:22:22 GMT

I've been working for a while at extracting fields from joined events. At the moment I have (a simplified version):

2012-11-14 04:23:58.7730 (bzdate=20121113) txnid=1234 amount=$31.25 method = ABC

index=XYZ method = ABC | fields bzdate txnid amount

However before this I have:

012-11-14 04:23:58.4755 (bzdate=20121113) txnid=1234 XML: <?xml version="1.0" encoding="UTF-8"?>

31.25
true

I think I need to use xmlkv as well as the transaction command to extract these. But I'm unsure how to "link" the 2 transactions.

I tried something like:

index=XYZ method = ABC | transaction txnid | xmlkv | fields bzdate txnid amount amountAuthorised

But no avail thus far. Unfortunately I can't be sure exactly how far apart the 2 events will be (they won't be a large amount apart) and there will be a lot of "noise" around these entries.

Thanks for any assistance

Re: Transaction - Related Events

martin_mueller — Wed, 14 Nov 2012 07:48:22 GMT

Your second search filters by method=ABC - however, your XML event does not appear to contain such a field. As a result your transaction command only gets to see half the story.

Re: Transaction - Related Events

shonky — Wed, 14 Nov 2012 07:50:21 GMT

Thank you for this. That makes sense. Is there a way, then, to define that I want to search for that method=ABC but also connect the 2 events based on the matching txnid?

Re: Transaction - Related Events

martin_mueller — Wed, 14 Nov 2012 07:56:19 GMT

You could apply the method filter after the transaction command. Whether this yields the result you have in mind depends on your data. Something like this:

index=foo | transaction bar | search method=baz | ...

Re: Transaction - Related Events

Ayn — Wed, 14 Nov 2012 09:11:51 GMT

Just rearrange your search a bit. First create a transaction based on the txnid, THEN search for transactions that have method=ABC.

index=XYZ | transaction txnid | xmlkv | search method=ABC | fields bzdate txnid amount amountAuthorised

EDIT: Or, because transaction is kind of a resource intensive command, you could use a subsearch that filters out all txnid's that have method=ABC and THEN call transaction.

* [search index=XYZ method=ABC | fields index txnid] | transaction txnid | xmlkv | fields bzdate txnid amount amountAuthorised

Re: Transaction - Related Events

shonky — Wed, 14 Nov 2012 17:09:10 GMT

I think both will work, but you're right, the 1st method is killing me on speed as Transaction is slowing it down so much.

The 2nd one looks interesting but am seeing

"Search operation 'index' is unknown. You might not have permission to run this operation."

Am guessing I may need to talk to our Splunk admins to see what needs to be enabled.

Re: Transaction - Related Events

Ayn — Wed, 14 Nov 2012 18:18:51 GMT

No, that sounds like you didn't enter the search correctly. You need to start off with specifying the "search" command in subsearches. So, [index=XYZ ...] will not work but [search index=XYZ ...] will.