topic Re: filter events on indexer in Splunk Search

filter events on indexer

fernandoandre — Thu, 05 Jan 2012 12:21:46 GMT

I want to filter some types of events at my indexer, that are received from several universal forwarders.

I try something similar like this:

http://splunk-base.splunk.com/answers/96/how-do-i-exclude-some-events-from-being-indexed-by-splunk

However I'm not receiving any events on my index.

Am I missing some configuration or some config file?
How do filter and index only those events on the regex?
The target index only needs to be on the univ forwarder inputs.conf file right?
The dropped events sent to nullQueue are never indexed correct? They are dropped before the index process..?

Any help is much appreciated.

My configuration files are the following:

INDEXER:

props.conf

[WinEventLog:Security]

TRANSFORMS-set= setnull,setparsing

transforms.conf

[setnull]

REGEX = .

DEST_KEY = queue

FORMAT = nullQueue

[setparsing]

REGEX = (?msi)^EventCode=552|538|576|528|529).*

DEST_KEY = queue

FORMAT = indexQueue (I don't fully understand the purpose of this field! Should it have the target index?)

Univ. Forwarder

inputs.conf

[WinEventLog:Security]

disabled = 0

index = myIndex

start_from = oldest

outputs.conf

[tcpout]

defaultGroup=(myServer)_9997

[tcpout:(myServer)_9997]

server=(myServer):9997

useACK = true

[tcpout-server://(myServer):9997]

useACK = true

Re: filter events on indexer

BobM — Thu, 05 Jan 2012 13:17:41 GMT

There are a couple of problems with this.

First the regex in setparsing is wrong. You should start the list of numbers with an open brace and don't need the terminating .*

REGEX = (?msi)^EventCode=(552|538|576|528|529)

Next, you have your regex in setnull to match everything. This will send it all to the null queue. Then you pull back (if the regex is fixed) the ones you want. This is inefficient so you want to create a regex to catch only what you want to throw away.

I would do this as follows:-

On the indexer

props.conf

[WinEventLog:Security]
TRANSFORMS-set= setnull

transforms.conf

[setnull]
REGEX = (?msi)^EventCode=(?!(552|538|576|528|529)\b)
DEST_KEY = queue
FORMAT = nullQueue

The ?! in this regex is sayng NOT what is in this bracket and the \b is making sure it doesn't match 5221, 5385 etc.

By the way, the FORMAT = indexQueue is telling splunk to put the event back in the indexing queue and it will use whatever index was defined in inputs.conf

Bob

Re: filter events on indexer

fernandoandre — Thu, 05 Jan 2012 16:08:27 GMT

I have understand more clearly the mechanism with your explanation than with documentation and other posts.

I have tested for 1hour now and it works perfectly.

Thank you for your assistance.

Re: filter events on indexer

amit2301 — Tue, 26 Sep 2017 07:41:52 GMT

Hi ,
I am trying to nullify the windows event log which are of "type=Information" and i want only the events having errors and warnings.
i am not able to achieve kindly help .
I am using below:

props.conf

[WinEventLog:Application]
TRANSFORMS-set= setnull

transforms.conf

[setnull]
REGEX = "Type=Information"
DEST_KEY = queue
FORMAT = nullQueue