topic Re: What's the best way to filter this sample entry to nullQueue? in Getting Data In

What's the best way to filter this sample entry to nullQueue?

agentguerry — Thu, 26 Mar 2015 01:24:31 GMT

Can filters be set up to accomplish this?

Scenario:

4 servers (server001.domain, server002.domain, server003.domain, server004.domain)
on each of those servers is a /var/log/"hostname"a.out and /var/log/"hostname"b.out file

I want to filter out on all 4 servers, in both of the files ("a.out" and "b.out")
ex. var/log/server001a.out and /var/log/server001b.out
/var/log/server002a.out and /var/log/server002b.out

com.site.ws.util.jaxws.SOAPMessageContextUtil - Could not instantiate EfficientStreamMessage java.lang.NoSuchFieldException: envelopeTag

On my Splunk "master" I have:

props.conf

[source::/var/log/server*.out]
TRANSFORMS-null = setnull

transforms.conf

[setnull]
REGEX = (?ism).*java.lang.NoSuchFieldException: envelopeTag.*
DEST_KEY = queue
FORMAT = nullQueue

Restart Splunk.
But the Splunk Web interface still shows them being logged.
Does this syntax/setup look correct?
Can you set a source to be multiple files with a * or wildcards?

Re: What's the best way to filter this sample entry to nullQueue?

musskopf — Thu, 26 Mar 2015 01:46:11 GMT

Have a look on this link: http://answers.splunk.com/answers/96/how-do-i-exclude-some-events-from-being-indexed-by-splunk.html

Also, are you doing that only on your master? You might need to configure every forwarder.

Re: What's the best way to filter this sample entry to nullQueue?

agentguerry — Thu, 26 Mar 2015 01:54:12 GMT

This would be on the master (server that hosts the indexer, web GUI)

my master inputs.conf file has this and has been pushed to all forwarders.

[monitor:///var/log/*svc*.out]
index = services
source = Services
disabled = 0
sourcetype = ServerOutLogs

From what I understand, to configure every forwarder (the 4 server00x.domain boxes) I need to install a "heavy forwarder" on those. Hoping for a way to do this on the "master"

Re: What's the best way to filter this sample entry to nullQueue?

musskopf — Thu, 26 Mar 2015 02:30:11 GMT

Ok, so if you're running Universal Forwarders it need to be done at Master. Have a look on that link, try to replicate that example and once work adjust for your needs.

Re: What's the best way to filter this sample entry to nullQueue?

musskopf — Thu, 26 Mar 2015 02:47:21 GMT

Another thing.. in your Regexp, you have a "." (dot) at the end... is that really there? You might wanna remove it.

Re: What's the best way to filter this sample entry to nullQueue?

agentguerry — Thu, 26 Mar 2015 02:51:31 GMT

This is what I had set. Should show 2 asterisks, but the site didn't render it properly

REGEX=(?ism).*java.lang.NoSuchFieldException: envelopeTag.*

Re: What's the best way to filter this sample entry to nullQueue?

woodcock — Mon, 10 Aug 2015 15:08:49 GMT

If your props.conf and transforms.conf are in the same scope as your inputs.conf (i.e. exist inside identical path/directory), this is a correct configuration PROVIDED there are no other stanzas that get processed later that move the same data back out of the nullQueue AND that the files have been distributed to the Indexers and Heavy Forwarders and all Splunk instances there have been restarted.

Re: What's the best way to filter this sample entry to nullQueue?

yannK — Mon, 10 Aug 2015 18:16:27 GMT

Beware, the stanza name "setnull" is often used and reused on different apps, so can double check with a btool that this is thew good one applied .

./splunk cmd btool transforms list setnull

I recommend to create your own stanza name, that explains what is does and is unique,
example : [SendJavaExceptionToNullQueue]