Does Splunk process events before sending to nullqueue?

[dsmc_adv]

We have configured a default null queue to discard all events that we don't want to allow to be indexed without authorization. Our transforms have the first filter in transform to send to the null_qeue_filter.

The topic for this question is because I have events in our splunkd.log from a source which looks to be exceeding the linecount:

2/23/16 
5:34:11.169 PM  
02-23-2016 17:34:11.169 +0100 WARN  AggregatorMiningProcessor - Breaking event because limit of 256 has been exceeded 

But when I look at this specific sourcetype and data source, there are no events with more than one line, so my assumption is that those events exceeding the limits are those which are being discarded, in other words sent to the null queue. Is this the correct behavior ?

[yannK · Accepted solution]

yes, if you look at the pipeline order , the nullqueue is happening after the parsing.
But before the license count.

Parsing (linebreak, encoding) -> aggregating (timestamp detection, event grouping )-> typing (regex, nullqueues, sedcmd ...) -> indexing (write to disk, count on license or forwarder on heavy forwarders)

[dsmc_adv]

is not Splunk in this case wasting resources parsing events that will be sent to trash ?

[yannK]

correct. the best method is not avoid monitoring the bad event in the first place.

But we cannot filter an event before it has been break down to a proper event, you do not want to delete a whole file if only one event match the filter 🙂