topic Transactions using different identifying fields in Splunk Search

Transactions using different identifying fields

himynamesdave — Mon, 28 Sep 2020 18:56:08 GMT

Attached is some data that you should be able to use to reproduce what I am trying to achieve.

Events.csv – extract of raw_field and sourcetype
Field extractions.txt – extract of field extractions from props.conf

I'm trying to to follow the flow of transactions using Splunk.

Transactions use different identifiers as they progress through which are: ORDER_NUMBER, CAR_PDR, CAR_PCR, PFM_PDR, PFM_PCR

I an using joins to make sense of the results:

chain=* | join PFM_PCR type=outer [search PFM_PCR=* PFM_PDR=] | join CAR_PCR type=outer [search CAR_PCR= CAR_PDR=] | join PFM_PDR type=outer [search ORDER_NUMBER= PFM_PDR=] | join CAR_PDR type=outer [search ORDER_NUMBER= CAR_PDR=*]| transaction ORDER_NUMBER

I thought I didn’t need the joins and could do the following instead:

chain=* | transaction ORDER_NUMBER CAR_PDR CAR_PCR PFM_PDR PFM_PCR

but this had the effect of creating transactions that had all the keys as the tuple for the transaction ID. So we got a transactions for (order1,car_pdr1,…), (order1,car_pdr2,…) etc

is there a better way of doing the transaction on ORDER_NUMBER that avoids all those messy joins?

Re: Transactions using different identifying fields

emiller42 — Wed, 11 Feb 2015 17:35:14 GMT

So transaction should be working exactly as you're expecting here. Consider:

event=1 field1=foo
event=2 field1=foo
event=3 field1=foo field2=bar
event=4 field2=bar
event=5 field2=bar

If you run |transaction field1 field2 you'll actually get a single event based containing 1 through 5. This is because it's looking for transitive relationships, and as long as there is at least one event where fields overlap, it'll consider them joined.

However, that ONLY works if you have some overlap connecting events.

Your sample data doesn't seem to include everything needed to test this. For example, there are no events that meet the EXTRACT-chain,PFM_PDR,File_name, EXTRACT-chain (PcR finished), EXTRACT-PFM_PcR,PFM_PcR_type, EXTRACT-PFM_PDR,PFM_PcR,chain,Product_name extractions from the PSM_FILE sourcetype.

Re: Transactions using different identifying fields

jbjerke_splunk — Mon, 28 Sep 2020 18:54:05 GMT

Hi david

While not exactly what you are asking for, you might be able to get around the problem by using the stats command and list() by ORDER_NUMBER

| stats count as eventcount list(_raw) as events,list(PFM_PCR) as PFM_PCR,list(CAR_PCR) as CAR_PCR,list(PFM_PDR) as PFM_PDR,list(CAR_PDR) as CAR_PDR, range(_time) as duration by ORDER_NUMBER

After this command you can apply some conditional searching to narrow down the results to fit your outer join "criterias"