I'm working with ForeScout Audit Policy events. Some of them have this in the message, Part (1/n), Part (2/n), and so on. I'm using the transaction command below to join the parts.
index=network sourcetype="forescout:audit" partOf=*
| transaction fields=partOf maxspan=1s
| search eventtype=fs_policy_change
| append [search index=network sourcetype=forescout:audit NOT partOf=* eventtype=fs_policy_change]
| sort - _time
The field partOf is set in default/transforms.conf
[fs_get_parts]
REGEX = \|\sPart\s\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)\s\|
The append adds the single event policy changes. The issue is the order is sometimes correct and other times not. For example I will get Part (4/4), Part (2/4), Part (1/4), and Part (3/4) for some of the transactions and others in the correct order. I didn't see anything in the transaction command to allow me to sort the partOf. Any ideas?
Splunk Enterprise 7.2.5.1
TIA,
Joe
sample:
| makeresults
| eval _raw="Time,Host,Couter,Part,Message
Mar 5 18:40:57,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar 5 18:40:57,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Mar 4 17:00:36,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar 4 17:00:36,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:31,hostname,CounterACT[16202]: | ,Part ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***"
| multikv forceheader=1
| table Time,Host,Couter,Part,Message
| rex field=Part "\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)"
| eval _time=strptime(Time,"%B %d %T")
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
Recommend:
index=network sourcetype="forescout:audit" eventtype=fs_policy_change
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
| reverse
sample:
| makeresults
| eval _raw="Time,Host,Couter,Part,Message
Mar 5 18:40:57,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar 5 18:40:57,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Mar 4 17:00:36,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar 4 17:00:36,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:31,hostname,CounterACT[16202]: | ,Part ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***"
| multikv forceheader=1
| table Time,Host,Couter,Part,Message
| rex field=Part "\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)"
| eval _time=strptime(Time,"%B %d %T")
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
Recommend:
index=network sourcetype="forescout:audit" eventtype=fs_policy_change
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
| reverse
Thanks. I made one minor change
index=network sourcetype="forescout:audit" partOf=*
| sort 0 _time -numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
and am getting the events I'm looking for along with the part order being correct.
Ditch transaction
; try this:
index="network" AND sourcetype="forescout:audit" AND "partOf"="*"
| rex "Part \((?<ThisPart>\d+)\/"
| sort 0 ThisPart partOf
| stats min(_time) AS _time count range(_time) AS duration list(_raw) AS events values(eventtype) AS eventtype BY partOf
| search eventtype="fs_policy_change"
| sort 0 - _time
I don't know what the rest is supposed to do but whatever it is, don't do it with append
. Also, NEVER use sort
without a number after it; otherwise it will truncate your results set.
This won't work. This is the output I get
Feb 28 23:10:05 hostname CounterACT[16202]: | Part (2/2) | ***
Mar 4 17:00:36 hostname CounterACT[16202]: | Part (2/2) | ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (2/2) | ***
Mar 5 18:40:57 hostname CounterACT[16202]: | Part (2/2) | ***
Mar 5 18:40:57 hostname CounterACT[16202]: | Part (1/2) | ***
Mar 4 17:00:36 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:10:05 hostname CounterACT[16202]: | Part (1/2) | ***
It needs to be
Mar 5 18:40:57 hostname CounterACT[16202]: | Part (1/2) | ***
Mar 5 18:40:57 hostname CounterACT[16202]: | Part (2/2) | ***
Mar 4 17:00:36 hostname CounterACT[16202]: | Part (1/2) | ***
Mar 4 17:00:36 hostname CounterACT[16202]: | Part (2/2) | ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (2/2) | ***
Feb 28 23:10:05 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:10:05 hostname CounterACT[16202]: | Part (2/2) | ***
Where each 1/2 pair is a separate event. For example one event using my original search
Mar 6 20:32:51 hostname CounterACT[16202]: | Part (1/2) | Message part 1 of 2
Mar 6 20:32:51 hostname CounterACT[16202]: | Part (2/2) | Message part 2 of 2
I need the transaction to group the multi-part event into one event.
I had my sort
wrong. I edited and fixed it, try again. I also added some _time
stuff.