Getting Data In

Is there an internal log in Splunk with the number of events that were sent to null queue?

Path Finder

Hi.

We have recently been inadvertently sending some events to the null queue, due to a new data source that matches a greedy regex pattern specified in transforms.conf on the indexer. We can correct the regex easily, but as I understand it, the events are lost for good as there is no copy of the raw data anywhere.

My question: is there maybe a log in Splunk that will advise on the number of events sent to the null queue? It would be good to know the fraction of incoming events being discarded, although some of these will be from a legitimate, intended match to the regex in transforms.conf.

0 Karma
1 Solution

Influencer
index=_internal component=metrics processor=nullqueue group=pipeline sourcetype=splunkd

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

Give this a try

index=_internal sourcetype=splunkd source=*metrics.log group=pipeline processor=nullqueue | stats sum(executes) as NullQueueInvocations
0 Karma

Influencer
index=_internal component=metrics processor=nullqueue group=pipeline sourcetype=splunkd

View solution in original post

0 Karma

Path Finder

Thanks guys, I used up my '2 posts a day' yesterday so just responding now... I've never really looked at metrics.log before but been reading up on it, looks to be quite useful, cheers.

0 Karma

Super Champion

Are you dropping at "heavy forwarders" or at "indexers" ?

0 Karma

Path Finder

Hi Koshyk, this would be at the indexers.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!