topic Re: avoid some events in Getting Data In

avoid some events

roopeshetty — Tue, 21 Dec 2021 09:31:30 GMT

Hi Team,

We are collecting data from Alibaba cloud through a heavy forwarder (using Alibaba add-ons) and pushing the data to our splunk cloud. But what we are seeing is its collecting all data from the Alibaba cloud which is huge in size, and upon validating it we realized that below events are making 80% of the whole events and it is not required to us. So we want to exclude below events (rule_result=pass and status=200) from being collected. We know this can be done by editing Props.conf File, but we have been trying it from long for it but we are not successful. Can someone please advise us how to edit this Props.conf file and get these below events (rule_result=pass and status=200) excluded from the heavy forwarder.

index= alibaba source="alibaba:cloudfirewall" rule_result=pass

index=alibaba source="alibaba:waf" status=200

Re: avoid some events

aasabatini — Tue, 21 Dec 2021 10:25:05 GMT

hi @roopeshetty

try this on the heavy forwarder

transforms.conf

[setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue [setparsing] REGEX = result\=200 DEST_KEY = queue FORMAT = indexQueue [cloudnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue [cloudparsing] REGEX = result\=pass DEST_KEY = queue FORMAT = indexQueue

props.conf

[alibaba:cloudfirewall] TRANSFORMS-set= cloudnull,cloudparsing [alibaba:waf] TRANSFORMS-set= setnull,setparsing

Re: avoid some events

roopeshetty — Tue, 21 Dec 2021 11:50:23 GMT

thanks a lot aasabatini. Its working now as expected. Thanks again.