topic Forwarder-side filter not working in Getting Data In

Forwarder-side filter not working

dantonag — Tue, 26 Jun 2012 16:29:04 GMT

Hello,
we've set up a transforms.conf filter in one of our forwarders (to lower indexed volume per day), but the configuration is not working (events are not filtered out).

Log lines (events) we want to exclude look like this:

26/06/2012 18:24:31 | ReadLstOfValTable | INFO | Numero di record estratti [1]

We have configured transforms.conf and props.conf like this:

props.conf

[SiebelDataADP]

TRANSFORMS-null= SiebelDataADPFiltro1

transforms.conf

[SiebelDataADPFiltro1]

REGEX = .*Numero\sdi\srecord\sestratti\s.*

DEST_KEY = queue

FORMAT = nullQueue

Any hints?

Thanks

Gabriele

Re: Forwarder-side filter not working

Ayn — Tue, 26 Jun 2012 16:39:45 GMT

This kind of filtering cannot be done on light forwarders (Universal Forwarder or its older counterpart, the Lightweight Forwarder). props.conf settings on light forwarders are largely ignored because the features they relate to (such as segmenting data that is read into individual events) don't exist there. This kind of filtering can only be performed on an indexer or a heavy forwarder.

Re: Forwarder-side filter not working

dantonag — Tue, 26 Jun 2012 17:59:59 GMT

Thanks.

So we have either to:

Configure filtering server-side ($SPLUNK_HOME/etc/system/local/props.conf and transforms.conf)
Use an "heavy" forwarder (i.e. Splunk itself)

Is this right?

Re: Forwarder-side filter not working

Ayn — Tue, 26 Jun 2012 19:13:57 GMT

Yes, that's correct.