Getting Data In

How do I send events to nullqueue?

stevenbutterwor
Path Finder

I am currently ingesting AWS VPC Flow logs from our AWS tenant. Most of the logs are internal traffic between ec2 instances. I'd like to send these events to nullqueue as they are not much use to us, we are only concerned with inbound and outbound traffic to the VPC. I think the best way to drop these events is to insert some regex into transforms.conf? Could someone help with this?

I wish to exclude events that have both srcaddr starting with 10.x and dstaddr starting with 10.x

Any help would be appreciated

0 Karma
1 Solution

stevenbutterwor
Path Finder

The correct regex is as follows

^\d\s\d*\seni-.*\s10.\d{1,3}.\d{1,3}.\d{1,3}\s10.\d{1,3}.\d{1,3}.\d{1,3}

View solution in original post

0 Karma

stevenbutterwor
Path Finder

The correct regex is as follows

^\d\s\d*\seni-.*\s10.\d{1,3}.\d{1,3}.\d{1,3}\s10.\d{1,3}.\d{1,3}.\d{1,3}

0 Karma

mayurr98
Super Champion

try this regex:

eni-\d{8}\s+10\d\.\d{1,3}\.\d{1,3}.\.\d{1,3}\s+10\d\.\d{1,3}\.\d{1,3}\.\d{1,3}
0 Karma

jawaharas
Motivator

You can try below RegEx.

[setnull]
REGEX = ^\d\s\d*\seni-\d{2}\s10\.\d{1,3}\.\d{1,3}\.\d{1,3}\s10\.\d{1,3}\.\d{1,3}\.\d{1,3}
DEST_KEY = queue
FORMAT = nullQueue
0 Karma

stevenbutterwor
Path Finder

alt text

I have attached an image. The 2 field names are srcaddr and dstaddr. It's when both of these start with 10 I want to send to nullQueue

0 Karma

sandyIscream
Communicator

@stevenbutterworth You have to setup two files. props.conf and transforms.conf

Please have a loot at the example below.

props.conf
[srctype1]
TRANSFORMS-set = setnull, logstoCapture

transforms.conf
[logstoCapture]
REGEX =
DEST_KEY = queue
FORMAT = indexQueue

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

0 Karma

stevenbutterwor
Path Finder

Thanks for your answer - it's regex part I need help with

0 Karma

sandyIscream
Communicator

Could you please send the sample log file. Then I might be able to help you write the regex query

0 Karma

sandyIscream
Communicator

@stevenbutterworth Did you try the regex that people have posted here ? If that helps then could you please accept the answer.

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...