Getting Data In

Filtration of logs during ingestion

anandhalagaras1
Communicator

Hi All,

I want to filter out the logs during the ingesting time itself so that if the keyword "GET / - 80"  is present in the logs then it should not be ingested into Splunk but the rest of the logs should be ingested into Splunk.  I will place the props and transforms in the Heavy Forwarder server so that during parsing it can filter out those logs.

Sample logs:

2021-02-15 13:04:28 xxx.xx.xxx.x GET / - 80 - xxx.xx.xx.x - - xxx x x xx

2021-02-15 13:04:27 xxx.xx.xxx.x GET / - 443 - xxx.xx.xx.x - - xxx x x xx

where "x" represents number IP's . So kindly help with the props and transforms.

The sourcetype is "abc".

 

Labels (2)
0 Karma
1 Solution

manjunathmeti
Champion

You can match and send the logs containing  "GET / - 80" to null queue. 
transforms.conf

[nullq]
DEST_KEY = queue
REGEX = GET\s\/\s-\s80
FORMAT = nullQueue


props.conf

[sourcetype_name]
TRANSFORMS-nullq = nullq

 

If this reply helps you, an upvote/like would be appreciated.

View solution in original post

0 Karma

manjunathmeti
Champion

You can match and send the logs containing  "GET / - 80" to null queue. 
transforms.conf

[nullq]
DEST_KEY = queue
REGEX = GET\s\/\s-\s80
FORMAT = nullQueue


props.conf

[sourcetype_name]
TRANSFORMS-nullq = nullq

 

If this reply helps you, an upvote/like would be appreciated.

0 Karma

anandhalagaras1
Communicator

@manjunathmeti ,

Thank you it worked like a charm.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...