Getting Data In

Filtration of logs during ingestion

anandhalagaras1
Communicator

Hi All,

I want to filter out the logs during the ingesting time itself so that if the keyword "GET / - 80"  is present in the logs then it should not be ingested into Splunk but the rest of the logs should be ingested into Splunk.  I will place the props and transforms in the Heavy Forwarder server so that during parsing it can filter out those logs.

Sample logs:

2021-02-15 13:04:28 xxx.xx.xxx.x GET / - 80 - xxx.xx.xx.x - - xxx x x xx

2021-02-15 13:04:27 xxx.xx.xxx.x GET / - 443 - xxx.xx.xx.x - - xxx x x xx

where "x" represents number IP's . So kindly help with the props and transforms.

The sourcetype is "abc".

 

Labels (2)
0 Karma
1 Solution

manjunathmeti
Champion

You can match and send the logs containing  "GET / - 80" to null queue. 
transforms.conf

[nullq]
DEST_KEY = queue
REGEX = GET\s\/\s-\s80
FORMAT = nullQueue


props.conf

[sourcetype_name]
TRANSFORMS-nullq = nullq

 

If this reply helps you, an upvote/like would be appreciated.

View solution in original post

0 Karma

manjunathmeti
Champion

You can match and send the logs containing  "GET / - 80" to null queue. 
transforms.conf

[nullq]
DEST_KEY = queue
REGEX = GET\s\/\s-\s80
FORMAT = nullQueue


props.conf

[sourcetype_name]
TRANSFORMS-nullq = nullq

 

If this reply helps you, an upvote/like would be appreciated.

0 Karma

anandhalagaras1
Communicator

@manjunathmeti ,

Thank you it worked like a charm.

0 Karma
Get Updates on the Splunk Community!

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...