Splunk Search

Is there a sort option for the transaction command

jwhughes58
Contributor

I'm working with ForeScout Audit Policy events. Some of them have this in the message, Part (1/n), Part (2/n), and so on. I'm using the transaction command below to join the parts.

index=network sourcetype="forescout:audit" partOf=*
| transaction fields=partOf maxspan=1s
| search eventtype=fs_policy_change
| append [search index=network sourcetype=forescout:audit NOT partOf=* eventtype=fs_policy_change]
| sort - _time

The field partOf is set in default/transforms.conf

[fs_get_parts]
REGEX = \|\sPart\s\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)\s\|

The append adds the single event policy changes. The issue is the order is sometimes correct and other times not. For example I will get Part (4/4), Part (2/4), Part (1/4), and Part (3/4) for some of the transactions and others in the correct order. I didn't see anything in the transaction command to allow me to sort the partOf. Any ideas?

Splunk Enterprise 7.2.5.1

TIA,
Joe

0 Karma
1 Solution

to4kawa
Ultra Champion

sample:

| makeresults 
| eval _raw="Time,Host,Couter,Part,Message
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:31,hostname,CounterACT[16202]: | ,Part  ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***" 
| multikv forceheader=1 
| table Time,Host,Couter,Part,Message 
| rex field=Part "\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)" 
| eval _time=strptime(Time,"%B %d %T") 
| sort 0 _time numPart partOf 
| transaction fields=partOf maxspan=1s keeporphans=t

Recommend:

index=network sourcetype="forescout:audit" eventtype=fs_policy_change
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
| reverse

View solution in original post

0 Karma

to4kawa
Ultra Champion

sample:

| makeresults 
| eval _raw="Time,Host,Couter,Part,Message
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:31,hostname,CounterACT[16202]: | ,Part  ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***" 
| multikv forceheader=1 
| table Time,Host,Couter,Part,Message 
| rex field=Part "\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)" 
| eval _time=strptime(Time,"%B %d %T") 
| sort 0 _time numPart partOf 
| transaction fields=partOf maxspan=1s keeporphans=t

Recommend:

index=network sourcetype="forescout:audit" eventtype=fs_policy_change
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
| reverse
0 Karma

jwhughes58
Contributor

Thanks. I made one minor change

index=network sourcetype="forescout:audit" partOf=*
| sort 0 _time -numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t

and am getting the events I'm looking for along with the part order being correct.

0 Karma

woodcock
Esteemed Legend

Ditch transaction; try this:

index="network" AND sourcetype="forescout:audit" AND "partOf"="*"
| rex "Part \((?<ThisPart>\d+)\/"
| sort 0 ThisPart partOf
| stats min(_time) AS _time count range(_time) AS duration list(_raw) AS events values(eventtype) AS eventtype BY partOf
| search eventtype="fs_policy_change"
| sort 0 - _time

I don't know what the rest is supposed to do but whatever it is, don't do it with append. Also, NEVER use sort without a number after it; otherwise it will truncate your results set.

0 Karma

jwhughes58
Contributor

This won't work. This is the output I get

Feb 28 23:10:05 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  4 17:00:36 hostname CounterACT[16202]: | Part (2/2) |  ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  5 18:40:57 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  5 18:40:57 hostname CounterACT[16202]: | Part (1/2) | ***
Mar  4 17:00:36 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:10:05 hostname CounterACT[16202]: | Part (1/2) | ***

It needs to be

    Mar  5 18:40:57 hostname CounterACT[16202]: | Part (1/2) | ***
    Mar  5 18:40:57 hostname CounterACT[16202]: | Part (2/2) | ***
    Mar  4 17:00:36 hostname CounterACT[16202]: | Part (1/2) | ***
    Mar  4 17:00:36 hostname CounterACT[16202]: | Part (2/2) | ***
    Feb 28 23:11:28 hostname CounterACT[16202]: | Part (1/2) | ***
    Feb 28 23:11:28 hostname CounterACT[16202]: | Part (2/2) | ***
    Feb 28 23:10:05 hostname CounterACT[16202]: | Part (1/2) | ***
    Feb 28 23:10:05 hostname CounterACT[16202]: | Part (2/2) | ***

Where each 1/2 pair is a separate event. For example one event using my original search

Mar  6 20:32:51 hostname CounterACT[16202]: | Part (1/2) | Message part 1 of 2
Mar  6 20:32:51 hostname CounterACT[16202]: | Part (2/2) | Message part 2 of 2

I need the transaction to group the multi-part event into one event.

0 Karma

woodcock
Esteemed Legend

I had my sort wrong. I edited and fixed it, try again. I also added some _time stuff.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...