Is there an internal log in Splunk with the number of events that were sent to null queue?

[jhigginsmq]

Hi.

We have recently been inadvertently sending some events to the null queue, due to a new data source that matches a greedy regex pattern specified in transforms.conf on the indexer. We can correct the regex easily, but as I understand it, the events are lost for good as there is no copy of the raw data anywhere.

My question: is there maybe a log in Splunk that will advise on the number of events sent to the null queue? It would be good to know the fraction of incoming events being discarded, although some of these will be from a legitimate, intended match to the regex in transforms.conf.

[somesoni2]

Give this a try

index=_internal sourcetype=splunkd source=*metrics.log group=pipeline processor=nullqueue | stats sum(executes) as NullQueueInvocations

[twinspop · Accepted solution]

index=_internal component=metrics processor=nullqueue group=pipeline sourcetype=splunkd

[jhigginsmq]

Thanks guys, I used up my '2 posts a day' yesterday so just responding now... I've never really looked at metrics.log before but been reading up on it, looks to be quite useful, cheers.

[koshyk]

Are you dropping at "heavy forwarders" or at "indexers" ?

[jhigginsmq]

Hi Koshyk, this would be at the indexers.