Splunk Search

Is there a sort option for the transaction command

jwhughes58
Communicator

I'm working with ForeScout Audit Policy events. Some of them have this in the message, Part (1/n), Part (2/n), and so on. I'm using the transaction command below to join the parts.

index=network sourcetype="forescout:audit" partOf=*
| transaction fields=partOf maxspan=1s
| search eventtype=fs_policy_change
| append [search index=network sourcetype=forescout:audit NOT partOf=* eventtype=fs_policy_change]
| sort - _time

The field partOf is set in default/transforms.conf

[fs_get_parts]
REGEX = \|\sPart\s\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)\s\|

The append adds the single event policy changes. The issue is the order is sometimes correct and other times not. For example I will get Part (4/4), Part (2/4), Part (1/4), and Part (3/4) for some of the transactions and others in the correct order. I didn't see anything in the transaction command to allow me to sort the partOf. Any ideas?

Splunk Enterprise 7.2.5.1

TIA,
Joe

0 Karma
1 Solution

to4kawa
Ultra Champion

sample:

| makeresults 
| eval _raw="Time,Host,Couter,Part,Message
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:31,hostname,CounterACT[16202]: | ,Part  ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***" 
| multikv forceheader=1 
| table Time,Host,Couter,Part,Message 
| rex field=Part "\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)" 
| eval _time=strptime(Time,"%B %d %T") 
| sort 0 _time numPart partOf 
| transaction fields=partOf maxspan=1s keeporphans=t

Recommend:

index=network sourcetype="forescout:audit" eventtype=fs_policy_change
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
| reverse

View solution in original post

0 Karma

to4kawa
Ultra Champion

sample:

| makeresults 
| eval _raw="Time,Host,Couter,Part,Message
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  5 18:40:57,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Mar  4 17:00:36,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:11:28,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***
Feb 28 23:11:31,hostname,CounterACT[16202]: | ,Part  ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (1/2) ,| ***
Feb 28 23:10:05,hostname,CounterACT[16202]: | ,Part (2/2) ,| ***" 
| multikv forceheader=1 
| table Time,Host,Couter,Part,Message 
| rex field=Part "\((?<numPart>\d{1,3})\/(?<partOf>\d{1,3})\)" 
| eval _time=strptime(Time,"%B %d %T") 
| sort 0 _time numPart partOf 
| transaction fields=partOf maxspan=1s keeporphans=t

Recommend:

index=network sourcetype="forescout:audit" eventtype=fs_policy_change
| sort 0 _time numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t
| reverse
0 Karma

jwhughes58
Communicator

Thanks. I made one minor change

index=network sourcetype="forescout:audit" partOf=*
| sort 0 _time -numPart partOf
| transaction fields=partOf maxspan=1s keeporphans=t

and am getting the events I'm looking for along with the part order being correct.

0 Karma

woodcock
Esteemed Legend

Ditch transaction; try this:

index="network" AND sourcetype="forescout:audit" AND "partOf"="*"
| rex "Part \((?<ThisPart>\d+)\/"
| sort 0 ThisPart partOf
| stats min(_time) AS _time count range(_time) AS duration list(_raw) AS events values(eventtype) AS eventtype BY partOf
| search eventtype="fs_policy_change"
| sort 0 - _time

I don't know what the rest is supposed to do but whatever it is, don't do it with append. Also, NEVER use sort without a number after it; otherwise it will truncate your results set.

0 Karma

jwhughes58
Communicator

This won't work. This is the output I get

Feb 28 23:10:05 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  4 17:00:36 hostname CounterACT[16202]: | Part (2/2) |  ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  5 18:40:57 hostname CounterACT[16202]: | Part (2/2) | ***
Mar  5 18:40:57 hostname CounterACT[16202]: | Part (1/2) | ***
Mar  4 17:00:36 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:11:28 hostname CounterACT[16202]: | Part (1/2) | ***
Feb 28 23:10:05 hostname CounterACT[16202]: | Part (1/2) | ***

It needs to be

    Mar  5 18:40:57 hostname CounterACT[16202]: | Part (1/2) | ***
    Mar  5 18:40:57 hostname CounterACT[16202]: | Part (2/2) | ***
    Mar  4 17:00:36 hostname CounterACT[16202]: | Part (1/2) | ***
    Mar  4 17:00:36 hostname CounterACT[16202]: | Part (2/2) | ***
    Feb 28 23:11:28 hostname CounterACT[16202]: | Part (1/2) | ***
    Feb 28 23:11:28 hostname CounterACT[16202]: | Part (2/2) | ***
    Feb 28 23:10:05 hostname CounterACT[16202]: | Part (1/2) | ***
    Feb 28 23:10:05 hostname CounterACT[16202]: | Part (2/2) | ***

Where each 1/2 pair is a separate event. For example one event using my original search

Mar  6 20:32:51 hostname CounterACT[16202]: | Part (1/2) | Message part 1 of 2
Mar  6 20:32:51 hostname CounterACT[16202]: | Part (2/2) | Message part 2 of 2

I need the transaction to group the multi-part event into one event.

0 Karma

woodcock
Esteemed Legend

I had my sort wrong. I edited and fixed it, try again. I also added some _time stuff.

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...