topic Re: Why is my Splunk 6.2.5 Heavy Forwarder not filtering out events as expected? in Getting Data In

Why is my Splunk 6.2.5 Heavy Forwarder not filtering out events as expected?

tony_luu — Thu, 01 Oct 2015 17:21:48 GMT

My Heavy Forwarder forwards data to the indexer fine, however, I wanted to filter out some events before being forwarded using props.conf and transforms.conf, but the indexer still receives everything.

props.conf:

   [source::/var/log/vsftpd.log]
   TRANSFORMS-null = setnull

transforms.conf:

   [setnull]
   REGEX = 220
   DEST_KEY = queue
   FORMAT = nullQueue

for testing, I just simplified the REGEX to filter out all events containing "220"
I even tried REGEX = . (to filter out everything) but still had no effect.

What am I missing?
I'm using Splunk 6.2.5 BTW.

Re: Why is my Splunk 6.2.5 Heavy Forwarder not filtering out events as expected?

somesoni2 — Thu, 01 Oct 2015 18:12:47 GMT

Try like this (changes to transforms.conf, keep the same props.conf)

transforms.conf

[setnull]
REGEX = (220)
DEST_KEY = queue
FORMAT = nullQueue

Also, ensure to restart the heavy forwarder after change.

Re: Why is my Splunk 6.2.5 Heavy Forwarder not filtering out events as expected?

tony_luu — Tue, 29 Sep 2020 07:28:13 GMT

same results.
just for testing, i brought the props.conf/transforms.conf to the indexer and it filtered as expected.
so it must be something on the forwarder side.

not sure if it helps, but the "cmd btool" dumped this info:

% /opt/splunk/bin/splunk cmd btool transforms list setnull
[setnull]
CAN_OPTIMIZE = True
CLEAN_KEYS = True
DEFAULT_VALUE =
DEST_KEY = queue
FORMAT = nullQueue
KEEP_EMPTY_VALS = False
LOOKAHEAD = 4096
MV_ADD = False
REGEX = 220
SOURCE_KEY = _raw
WRITE_META = False

Re: Why is my Splunk 6.2.5 Heavy Forwarder not filtering out events as expected?

tony_luu — Mon, 05 Oct 2015 15:54:53 GMT

just open a new case with splunk support.
will post the results when the case is resolved.

Re: Why is my Splunk 6.2.5 Heavy Forwarder not filtering out events as expected?

tony_luu — Thu, 08 Oct 2015 19:08:03 GMT

Mystery solved!

per Splunk support recommendation, i reinstalled a fresh splunk 6.2.5 and everything worked as expected.

I guess the problem is that my previous 6.2.5 installation was an upgrade from 6.0.1
Shouldn't have to do that but hey it works now.