Getting Data In

nullQueue difficulty

cgnoel
Explorer

I am trying configure Splunk to ignore certain events using the standard nullQueue configuration described in the docs as well as on this site but am not getting any desired results. I have even gone as far as to configure an isolated sample sourcetype to ignore every event using the provided [send_to_nullqueue]. I am certain that I am missing something minor with my configuration. What are the most common oversights that might be the source of my issue?

props.conf
[splunk_mv_test-too_small]
TRANSFORMS-null=send_to_nullqueue

transforms.conf
[send_to_nullqueue]
DEST_KEY = queue
REGEX = .
FORMAT = nullQueue

Tags (1)
0 Karma
1 Solution

lguinn2
Legend

I think the problem may be this (I am doing some guessing here):

You are monitoring a directory and have Splunk doing automatic sourcetyping - which is appropriate. I expect that nothing has overriden this setting. When Splunk can't figure out a sourcetype for an input because there is insufficient data, it defaults the sourcetype to "somename-too_small".

Your configuration specifies that all data of this sourcetype is to be ignored.

But there isn't really any data that has been assigned this sourcetype - it's an artifact of Splunk's parsing. I think it should work anyway, but it doesn't seem to. I don't think you will find any help with btool because of this.

BUT - I think there are better ways to solve this problem. Assigning data to the null queue is relatively inefficient and should be a last resort anyway. Here are some other ideas

1 - If you don't want any data from this file, blacklist it in the inputs. Find the input that is collecting the directory where this file lives, and put in a blacklist entry for the name of the file. More info about blacklisting an input

2 - If you want the data, but just don't want it to show up as "somename-too_small", override the sourcetype name in props.conf More info on overriding the automatic sourcetype

View solution in original post

lguinn2
Legend

I think the problem may be this (I am doing some guessing here):

You are monitoring a directory and have Splunk doing automatic sourcetyping - which is appropriate. I expect that nothing has overriden this setting. When Splunk can't figure out a sourcetype for an input because there is insufficient data, it defaults the sourcetype to "somename-too_small".

Your configuration specifies that all data of this sourcetype is to be ignored.

But there isn't really any data that has been assigned this sourcetype - it's an artifact of Splunk's parsing. I think it should work anyway, but it doesn't seem to. I don't think you will find any help with btool because of this.

BUT - I think there are better ways to solve this problem. Assigning data to the null queue is relatively inefficient and should be a last resort anyway. Here are some other ideas

1 - If you don't want any data from this file, blacklist it in the inputs. Find the input that is collecting the directory where this file lives, and put in a blacklist entry for the name of the file. More info about blacklisting an input

2 - If you want the data, but just don't want it to show up as "somename-too_small", override the sourcetype name in props.conf More info on overriding the automatic sourcetype

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

Are you sure that you've got data coming in with this sourcetype? You could also look at btool to make sure splunk sees the config. From $SPLUNK_HOME/bin/ run 'splunk cmd btool transforms list --debug'

0 Karma

cgnoel
Explorer

How likely is it that something is overriding this setting? What would be a good way to troubleshoot this?

0 Karma

cgnoel
Explorer

Indexer. I'm attempting to apply this to existing indexes (restarting Splunk).

0 Karma

Ayn
Legend

Where are you doing this, on the indexer or on a forwarder?

0 Karma

Drainy
Champion

Could you paste your config? we might be able to spot the error!

.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!