Why does the Transaction command return multiple results?

[andrewtrobec]

Hello!
I am working with the transaction command. I am passing a field and using startswith and endswith definition options. When I run it, though, the output produces two results per transaction. The first contains all events in the transaction while the second, the one I'm looking for, contains the events specified in the definition options. To provide a simplified example, the events in a transaction are as follows:

_time,FIELD,MESSAGE
28/04/2017 00:00:01,FIELD1,Starting Message
28/04/2017 00:00:02,FIELD1,Intermediate Message 1
28/04/2017 00:00:03,FIELD1,Intermediate Message 2
28/04/2017 00:00:04,FIELD1,Intermediate Message 3
28/04/2017 00:00:05,FIELD1,Ending Message
28/04/2017 00:00:11,FIELD2,Starting Message
28/04/2017 00:00:12,FIELD2,Intermediate Message 1
28/04/2017 00:00:13,FIELD2,Intermediate Message 2
28/04/2017 00:00:14,FIELD2,Intermediate Message 3
28/04/2017 00:00:15,FIELD2,Ending Message

The search is then:

index="my_index" | transaction FIELD startswith=eval(MESSAGE="Starting Message") endswith=eval(MESSAGE="Ending Message") | table FIELD, eventcount

Which produces the following table:

FIELD,eventcount
FIELD1,5
FIELD1,2
FIELD2,5
FIELD2,2

Instead, I was expecting

FIELD,eventcount
FIELD1,2
FIELD2,2

How do I ensure that only the transaction containing the startswith and endswith events is returned?

Thank you!

Andrew

[somesoni2]

If you don't care about intermediate transaction events, why don't you exclude them from base search. Give this a try

index="my_index" MESSAGE="Starting Message" OR MESSAGE="Ending Message"| transaction FIELD startswith=eval(MESSAGE="Starting Message") endswith=eval(MESSAGE="Ending Message") | table FIELD, eventcount

[andrewtrobec]

Thanks for the response. This does work, and is a good solution. Maybe I'm not understanding the transaction function properly, but I thought that it would filter out the events automatically using the startswith and endswith definition options.

EDIT: Now that I've run the solution I get the following:

FIELD,eventcount
FIELD1,2
FIELD1,2
FIELD2,2
FIELD2,2

So it still produces a double transaction for each FIELD value...

[somesoni2]

If you just run your base search, how many events you get?

index="my_index" MESSAGE="Starting Message" OR MESSAGE="Ending Message"

[DalJeanis]

Remember, in splunk, unless you specify otherwise, events naturally come out with the MOST RECENT FIRST.

So, splunk is finding the end of a transaction with a value of FIELD1, then it's finding the beginning of what it thinks is ANOTHER transaction with value FIELD1.

add |reverse or |sort 0 _time immediately before the transaction command. Bonus points for |sort 0 _time FIELD, since it's effectively going to have to do that anyway.

[andrewtrobec]

Thanks for your input. Just like to add: if I run the command as index="my_index" | transaction FIELD then it creates the transactions and puts them in chronological order. I figured that using startswith and endswith would do the same thing, but exclude all evens in between. Is this the wrong understanding of the command?

[DalJeanis]

I believe on faith that there may be someone who fully understands the transaction command, but I haven't met him or her yet. My experience here tells me that it is reckless to post any "solution" code containing the verb transaction unless you have mocked up test data, because the verb is terribly finnicky and people's real data is even more squirrelly than you can mock up in a reasonable amount of time.

Key points i do know...

• startswith does not mean the earliest _time, it means the first transaction encountered by splunk. Apply the reverse logic for endswith.

• maxspan and maxpause do not work the way I would expect. When I attempt to use them both, maxspan appears to be limited to the length of maxpause Or I may be hallucinating that, but I can't get it to act with, say, transactions that last 2 hours but can't pause more than 10 minutes between events.