topic help with event filtering - excluding events before indexing in Getting Data In

help with event filtering - excluding events before indexing

ebailey — Mon, 07 Oct 2013 18:36:12 GMT

I have an overload of events no one wants and are eating up our license so I did the following and it is not working.

I am trying to drop the following message:

"Terminating on fatal IPC exception"

I am running this off of a heavy forwarder:

Here is my props

[source::/opt/logs/all_logs]
TRANSFORMS-null= setnull

Here is my transforms

[setnull]
REGEX = /Terminating on fatal IPC exception/
DEST_KEY = queue
FORMAT = nullQueue

The above is working so well now no events are being forwarded to the indexers. Any idea what I am doing wrong?

Thanks

Re: help with event filtering - excluding events before indexing

lukejadamec — Mon, 07 Oct 2013 19:01:15 GMT

The config looks good. Somehow you're regex is matching everything. I've used something very similar in the past, but on the indexer. Never tried it on a heavy forwarder.

Re: help with event filtering - excluding events before indexing

dwaddle — Mon, 07 Oct 2013 22:30:46 GMT

I don't think the REGEX needs the / characters around it unless that is in the actual event. This should be fine:

REGEX = Terminating on fatal IPC exception

But, a stanza name as generic as "setnull" may already existing within configs. I might suggest naming it something more specific like:

[source::/opt/logs/all_logs]
TRANSFORMS-null= setnull-fatalIPCexception

[setnull-fatalIPCexception]
REGEX = Terminating on fatal IPC exception
DEST_KEY = queue
FORMAT = nullQueue

Re: help with event filtering - excluding events before indexing

ebailey — Mon, 07 Oct 2013 23:48:54 GMT

I made your suggested change and no joy. Now all events are flowing from the heavy forwarder to the indexers. Thanks for your effort.

Re: help with event filtering - excluding events before indexing

ebailey — Mon, 07 Oct 2013 23:49:51 GMT

BTW - the actual full event is

Oct 7 23:49:04 xxxhostnamexxx lsassd[9246]: 0x3fcc8b90:Terminating on fatal IPC exception

Re: help with event filtering - excluding events before indexing

lukejadamec — Mon, 07 Oct 2013 23:54:48 GMT

Is this a multi-line event? You could try adding (?msi) to the beginning of the regex. You can test the regex in a search string which saves restarting the forwarder.

Re: help with event filtering - excluding events before indexing

ebailey — Tue, 08 Oct 2013 13:59:10 GMT

this is not a multi-line event - good idea using a search string to test the regex. Thanks