Getting Data In

avoid some events

roopeshetty
Path Finder
Hi Team,
We are collecting data from Alibaba cloud through a heavy forwarder (using Alibaba add-ons) and pushing the data to our splunk cloud. But what we are seeing is its collecting all data from the Alibaba cloud which is huge in size, and upon validating it we realized that below events are making 80% of the whole events and it is not required to us. So we want to exclude below events (rule_result=pass and status=200) from being collected. We know this can be done by editing Props.conf File, but we have been trying it from long for it but we are not successful. Can someone please advise us how to edit this Props.conf file and get these below events (rule_result=pass and status=200) excluded from the heavy forwarder.
 
index= alibaba source="alibaba:cloudfirewall" rule_result=pass
index=alibaba source="alibaba:waf" status=200
 
 
Labels (3)
0 Karma
1 Solution

aasabatini
Motivator

hi @roopeshetty 

try this on the heavy forwarder

transforms.conf

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = result\=200
DEST_KEY = queue
FORMAT = indexQueue


[cloudnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[cloudparsing]
REGEX = result\=pass
DEST_KEY = queue
FORMAT = indexQueue

 

props.conf

[alibaba:cloudfirewall]
TRANSFORMS-set= cloudnull,cloudparsing

[alibaba:waf]
TRANSFORMS-set= setnull,setparsing
“The answer is out there, Neo, and it’s looking for you, and it will find you if you want it to.”

View solution in original post

Tags (1)

aasabatini
Motivator

hi @roopeshetty 

try this on the heavy forwarder

transforms.conf

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = result\=200
DEST_KEY = queue
FORMAT = indexQueue


[cloudnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[cloudparsing]
REGEX = result\=pass
DEST_KEY = queue
FORMAT = indexQueue

 

props.conf

[alibaba:cloudfirewall]
TRANSFORMS-set= cloudnull,cloudparsing

[alibaba:waf]
TRANSFORMS-set= setnull,setparsing
“The answer is out there, Neo, and it’s looking for you, and it will find you if you want it to.”
Tags (1)

roopeshetty
Path Finder

thanks a lot aasabatini.  Its working now as expected. Thanks again.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...