topic Re: nullQueue difficulty in Getting Data In

nullQueue difficulty

cgnoel — Mon, 28 Sep 2020 12:52:38 GMT

I am trying configure Splunk to ignore certain events using the standard nullQueue configuration described in the docs as well as on this site but am not getting any desired results. I have even gone as far as to configure an isolated sample sourcetype to ignore every event using the provided [send_to_nullqueue]. I am certain that I am missing something minor with my configuration. What are the most common oversights that might be the source of my issue?

props.conf
[splunk_mv_test-too_small]
TRANSFORMS-null=send_to_nullqueue

transforms.conf
[send_to_nullqueue]
DEST_KEY = queue
REGEX = .
FORMAT = nullQueue

Re: nullQueue difficulty

Drainy — Wed, 28 Nov 2012 15:21:22 GMT

Could you paste your config? we might be able to spot the error!

Re: nullQueue difficulty

Ayn — Wed, 28 Nov 2012 15:55:42 GMT

Where are you doing this, on the indexer or on a forwarder?

Re: nullQueue difficulty

cgnoel — Wed, 28 Nov 2012 15:59:02 GMT

Indexer. I'm attempting to apply this to existing indexes (restarting Splunk).

Re: nullQueue difficulty

cgnoel — Wed, 28 Nov 2012 17:01:06 GMT

How likely is it that something is overriding this setting? What would be a good way to troubleshoot this?

Re: nullQueue difficulty

jbsplunk — Wed, 28 Nov 2012 20:10:27 GMT

Are you sure that you've got data coming in with this sourcetype? You could also look at btool to make sure splunk sees the config. From $SPLUNK_HOME/bin/ run 'splunk cmd btool transforms list --debug'

Re: nullQueue difficulty

lguinn2 — Thu, 29 Nov 2012 01:29:30 GMT

I think the problem may be this (I am doing some guessing here):

You are monitoring a directory and have Splunk doing automatic sourcetyping - which is appropriate. I expect that nothing has overriden this setting. When Splunk can't figure out a sourcetype for an input because there is insufficient data, it defaults the sourcetype to "somename-too_small".

Your configuration specifies that all data of this sourcetype is to be ignored.

But there isn't really any data that has been assigned this sourcetype - it's an artifact of Splunk's parsing. I think it should work anyway, but it doesn't seem to. I don't think you will find any help with btool because of this.

BUT - I think there are better ways to solve this problem. Assigning data to the null queue is relatively inefficient and should be a last resort anyway. Here are some other ideas

1 - If you don't want any data from this file, blacklist it in the inputs. Find the input that is collecting the directory where this file lives, and put in a blacklist entry for the name of the file. More info about blacklisting an input

2 - If you want the data, but just don't want it to show up as "somename-too_small", override the sourcetype name in props.conf More info on overriding the automatic sourcetype