Getting Data In

Filtration of logs during ingestion

anandhalagaras1
Communicator

Hi All,

I want to filter out the logs during the ingesting time itself so that if the keyword "GET / - 80"  is present in the logs then it should not be ingested into Splunk but the rest of the logs should be ingested into Splunk.  I will place the props and transforms in the Heavy Forwarder server so that during parsing it can filter out those logs.

Sample logs:

2021-02-15 13:04:28 xxx.xx.xxx.x GET / - 80 - xxx.xx.xx.x - - xxx x x xx

2021-02-15 13:04:27 xxx.xx.xxx.x GET / - 443 - xxx.xx.xx.x - - xxx x x xx

where "x" represents number IP's . So kindly help with the props and transforms.

The sourcetype is "abc".

 

Labels (2)
0 Karma
1 Solution

manjunathmeti
Champion

You can match and send the logs containing  "GET / - 80" to null queue. 
transforms.conf

[nullq]
DEST_KEY = queue
REGEX = GET\s\/\s-\s80
FORMAT = nullQueue


props.conf

[sourcetype_name]
TRANSFORMS-nullq = nullq

 

If this reply helps you, an upvote/like would be appreciated.

View solution in original post

0 Karma

manjunathmeti
Champion

You can match and send the logs containing  "GET / - 80" to null queue. 
transforms.conf

[nullq]
DEST_KEY = queue
REGEX = GET\s\/\s-\s80
FORMAT = nullQueue


props.conf

[sourcetype_name]
TRANSFORMS-nullq = nullq

 

If this reply helps you, an upvote/like would be appreciated.

0 Karma

anandhalagaras1
Communicator

@manjunathmeti ,

Thank you it worked like a charm.

0 Karma
Get Updates on the Splunk Community!

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...