Getting Data In

Why is my Splunk 6.2.5 Heavy Forwarder not filtering out events as expected?

tony_luu
Path Finder

My Heavy Forwarder forwards data to the indexer fine, however, I wanted to filter out some events before being forwarded using props.conf and transforms.conf, but the indexer still receives everything.

props.conf:

   [source::/var/log/vsftpd.log]
   TRANSFORMS-null = setnull

transforms.conf:

   [setnull]
   REGEX = 220
   DEST_KEY = queue
   FORMAT = nullQueue

for testing, I just simplified the REGEX to filter out all events containing "220"
I even tried REGEX = . (to filter out everything) but still had no effect.

What am I missing?
I'm using Splunk 6.2.5 BTW.

0 Karma
1 Solution

tony_luu
Path Finder

Mystery solved!

per Splunk support recommendation, i reinstalled a fresh splunk 6.2.5 and everything worked as expected.

I guess the problem is that my previous 6.2.5 installation was an upgrade from 6.0.1
Shouldn't have to do that but hey it works now.

View solution in original post

0 Karma

tony_luu
Path Finder

Mystery solved!

per Splunk support recommendation, i reinstalled a fresh splunk 6.2.5 and everything worked as expected.

I guess the problem is that my previous 6.2.5 installation was an upgrade from 6.0.1
Shouldn't have to do that but hey it works now.

0 Karma

tony_luu
Path Finder

just open a new case with splunk support.
will post the results when the case is resolved.

0 Karma

somesoni2
Revered Legend

Try like this (changes to transforms.conf, keep the same props.conf)

transforms.conf

[setnull]
REGEX = (220)
DEST_KEY = queue
FORMAT = nullQueue

Also, ensure to restart the heavy forwarder after change.

0 Karma

tony_luu
Path Finder

same results.
just for testing, i brought the props.conf/transforms.conf to the indexer and it filtered as expected.
so it must be something on the forwarder side.

not sure if it helps, but the "cmd btool" dumped this info:

% /opt/splunk/bin/splunk cmd btool transforms list setnull
[setnull]
CAN_OPTIMIZE = True
CLEAN_KEYS = True
DEFAULT_VALUE =
DEST_KEY = queue
FORMAT = nullQueue
KEEP_EMPTY_VALS = False
LOOKAHEAD = 4096
MV_ADD = False
REGEX = 220
SOURCE_KEY = _raw
WRITE_META = False

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...