topic Re: Is there a sort option for the transaction command in Splunk Search

Is there a sort option for the transaction command

jwhughes58 — Fri, 06 Mar 2020 19:58:50 GMT

I'm working with ForeScout Audit Policy events. Some of them have this in the message, Part (1/n), Part (2/n), and so on. I'm using the transaction command below to join the parts.

index=network sourcetype="forescout:audit" partOf=*
| transaction fields=partOf maxspan=1s
| search eventtype=fs_policy_change
| append [search index=network sourcetype=forescout:audit NOT partOf=* eventtype=fs_policy_change]
| sort - _time

The field partOf is set in default/transforms.conf

[fs_get_parts]
REGEX = \|\sPart\s\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)\s\|

The append adds the single event policy changes. The issue is the order is sometimes correct and other times not. For example I will get Part (4/4), Part (2/4), Part (1/4), and Part (3/4) for some of the transactions and others in the correct order. I didn't see anything in the transaction command to allow me to sort the partOf. Any ideas?

Splunk Enterprise 7.2.5.1

TIA,
Joe

Re: Is there a sort option for the transaction command

woodcock — Fri, 06 Mar 2020 20:19:17 GMT

Ditch transaction; try this:

index="network" AND sourcetype="forescout:audit" AND "partOf"="*"
| rex "Part \((?<ThisPart>\d+)\/"
| sort 0 ThisPart partOf
| stats min(_time) AS _time count range(_time) AS duration list(_raw) AS events values(eventtype) AS eventtype BY partOf
| search eventtype="fs_policy_change"
| sort 0 - _time

I don't know what the rest is supposed to do but whatever it is, don't do it with append. Also, NEVER use sort without a number after it; otherwise it will truncate your results set.

Re: Is there a sort option for the transaction command

jwhughes58 — Fri, 06 Mar 2020 20:50:35 GMT

This won't work. This is the output I get

Feb 28 23:10:05 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  4 17:00:36 hostname CounterACT[16202]: | Part (2/2) |  ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  5 18:40:57 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  5 18:40:57 hostname CounterACT[16202]: | Part (1/2) | ***
Mar  4 17:00:36 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:10:05 hostname CounterACT[16202]: | Part (1/2) | ***

It needs to be

    Mar  5 18:40:57 hostname CounterACT[16202]: | Part (1/2) | ***
    Mar  5 18:40:57 hostname CounterACT[16202]: | Part (2/2) | ***
    Mar  4 17:00:36 hostname CounterACT[16202]: | Part (1/2) | ***
    Mar  4 17:00:36 hostname CounterACT[16202]: | Part (2/2) | ***
    Feb 28 23:11:28 hostname CounterACT[16202]: | Part (1/2) | ***
    Feb 28 23:11:28 hostname CounterACT[16202]: | Part (2/2) | ***
    Feb 28 23:10:05 hostname CounterACT[16202]: | Part (1/2) | ***
    Feb 28 23:10:05 hostname CounterACT[16202]: | Part (2/2) | ***

Where each 1/2 pair is a separate event. For example one event using my original search

Mar  6 20:32:51 hostname CounterACT[16202]: | Part (1/2) | Message part 1 of 2
Mar  6 20:32:51 hostname CounterACT[16202]: | Part (2/2) | Message part 2 of 2

I need the transaction to group the multi-part event into one event.

Re: Is there a sort option for the transaction command

woodcock — Fri, 06 Mar 2020 22:50:35 GMT

I had my sort wrong. I edited and fixed it, try again. I also added some _time stuff.

Re: Is there a sort option for the transaction command

to4kawa — Sat, 07 Mar 2020 00:29:52 GMT

sample:

| makeresults 
| eval _raw="Time,Host,Couter,Part,Message
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:31,hostname,CounterACT[16202]: | ,Part  ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***" 
| multikv forceheader=1 
| table Time,Host,Couter,Part,Message 
| rex field=Part "\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)" 
| eval _time=strptime(Time,"%B %d %T") 
| sort 0 _time numPart partOf 
| transaction fields=partOf maxspan=1s keeporphans=t

Recommend:

index=network sourcetype="forescout:audit" eventtype=fs_policy_change
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
| reverse

Re: Is there a sort option for the transaction command

jwhughes58 — Mon, 09 Mar 2020 19:28:08 GMT

Thanks. I made one minor change

index=network sourcetype="forescout:audit" partOf=*
| sort 0 _time -numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t

and am getting the events I'm looking for along with the part order being correct.