topic Re: Persistent Queues and index clustering in Getting Data In

Persistent Queues and index clustering

ckurtz — Mon, 28 Sep 2020 14:40:33 GMT

I've recently increased queue sizes on our indexers in our index cluster manually (editing the inputs.conf on the indexers themselves instead of on the index master, as a test) following the docs (http://docs.splunk.com/Documentation/Splunk/5.0.4/Data/Usepersistentqueues):

queueSize = 5MB

persistentQueueSize = 100MB

But when I add those same queue values on our index master and attempt to apply the bundle I get errors:

In handler 'clustermastercontrol':

Possible typo in stanza [splunktcp-ssl://9997] in /opt/splunk/etc/master-apps/asu_all_indexer_base/local/inputs.conf, line 7: queueSize = 5MB

Possible typo in stanza [splunktcp-ssl://9997] in /opt/splunk/etc/master-apps/asu_all_indexer_base/local/inputs.conf, line 8: persistentQueueSize = 100MB

Thoughts?

Re: Persistent Queues and index clustering

rturk — Wed, 28 Aug 2013 23:01:13 GMT

Hi Ckurtz,

I think I see your problem. It's in your stanza declaration. Instead of:

[splunktcp-ssl://9997]

try:

[splunktcp-ssl:9997]

Let me know how you go 🙂

References:

http://docs.splunk.com/Documentation/Splunk/5.0.4/Admin/Inputsconf

Re: Persistent Queues and index clustering

ckurtz — Wed, 28 Aug 2013 23:12:35 GMT

Apparently it's accepted with or without the // -- the inputs.conf doc shows both versions.

Removing the "//" doesn't solve the issue, and I certainly have it configured and working with the "//" on forwarders and indexers...

But a good catch anyway!

Re: Persistent Queues and index clustering

ckurtz — Mon, 07 Oct 2013 20:55:27 GMT

Official answer from my support ticket:

Indexers do not prevent incorrect configurations, they only show the "possible typo" warning, but the Cluster Master does validate, so correctly throws an error when trying to apply the bundle.

The actual answer is that persistentQueue is not designed for splunktcp stanzas, as per the docs (http://docs.splunk.com/Documentation/Splunk/5.0.4/Data/Usepersistentqueues), which I'd apparently missed the section on what persistentQueue was usable on:

"Persistent queues are not available for these input types: Monitor Batch File system change,
monitor Windows event log data, splunktcp (input from Splunk forwarders)"