How do I send events to nullqueue?

[stevenbutterwor]

I am currently ingesting AWS VPC Flow logs from our AWS tenant. Most of the logs are internal traffic between ec2 instances. I'd like to send these events to nullqueue as they are not much use to us, we are only concerned with inbound and outbound traffic to the VPC. I think the best way to drop these events is to insert some regex into transforms.conf? Could someone help with this?

I wish to exclude events that have both srcaddr starting with 10.x and dstaddr starting with 10.x

Any help would be appreciated

[stevenbutterwor · Accepted solution]

The correct regex is as follows

^\d\s\d*\seni-.*\s10.\d{1,3}.\d{1,3}.\d{1,3}\s10.\d{1,3}.\d{1,3}.\d{1,3}

[mayurr98]

try this regex:

eni-\d{8}\s+10\d\.\d{1,3}\.\d{1,3}.\.\d{1,3}\s+10\d\.\d{1,3}\.\d{1,3}\.\d{1,3}

[jawaharas]

You can try below RegEx.

[setnull]
REGEX = ^\d\s\d*\seni-\d{2}\s10\.\d{1,3}\.\d{1,3}\.\d{1,3}\s10\.\d{1,3}\.\d{1,3}\.\d{1,3}
DEST_KEY = queue
FORMAT = nullQueue

[stevenbutterwor]

I have attached an image. The 2 field names are srcaddr and dstaddr. It's when both of these start with 10 I want to send to nullQueue

[sandyIscream]

@stevenbutterworth You have to setup two files. props.conf and transforms.conf

Please have a loot at the example below.

props.conf
[srctype1]
TRANSFORMS-set = setnull, logstoCapture

transforms.conf
[logstoCapture]
REGEX =
DEST_KEY = queue
FORMAT = indexQueue

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[stevenbutterwor]

Thanks for your answer - it's regex part I need help with

[sandyIscream]

Could you please send the sample log file. Then I might be able to help you write the regex query

[sandyIscream]

@stevenbutterworth Did you try the regex that people have posted here ? If that helps then could you please accept the answer.