topic Re: UF persistentQueue tcp,UF persistentQueueSize tcp://8088 in Getting Data In

UF persistentQueue tcp,UF persistentQueueSize tcp://8088

arjangoos — Wed, 11 Feb 2015 07:34:05 GMT

We have an application that sends it's data to the UF on tcp port 8088. When the indexers are down we want the UF to store the log data in a persistentQueue untill the indexers are online again. Is this possible?

in the inputs.conf we have this stanza:
[tcp://8088]
persistentQueueSize=2GB

But this is not working. Can anyone help me?

,We have an application that offers it's logging to the UF API on tcp://8088 and we want to use a persistentQueue to store data when the indexers would be down. Is this possible? What we did is create een app with inputs.conf:

[tcp://8088]
persistentQueueSize=2GB

But this is not working we expected.

Can someone help us?

Re: UF persistentQueue tcp,UF persistentQueueSize tcp://8088

MuS — Wed, 11 Feb 2015 07:52:48 GMT

Hi arjangoos,

on the forwarder run this command to check if the the config is applied correct:

$SPLUNK_HOME/bin/splunk cmd btool --debug inputs list tcp

Output looks something like this and yours should have a persistentQueueSize

/opt/splunk/etc/system/default/inputs.conf [tcp]
/opt/splunk/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunk/etc/system/default/inputs.conf acceptFrom = *
/opt/splunk/etc/system/default/inputs.conf connection_host = dns
/opt/splunk/etc/system/local/inputs.conf   host = VirtualBox
/opt/splunk/etc/system/default/inputs.conf index = default

Also check if the queue directory has been created, it should be at

$SPLUNK_HOME/var/run/splunk/tcpin/pq__8088

in your case.
Check as well splunkd.log of your forwarder for any error messages.

Hope this helps ...

cheers, MuS

Re: UF persistentQueue tcp,UF persistentQueueSize tcp://8088

arjangoos — Mon, 28 Sep 2020 18:53:27 GMT

Hi MuS:

/opt/splunkforwarder/etc/system/default/inputs.conf [tcp]
/opt/splunkforwarder/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunkforwarder/etc/system/default/inputs.conf acceptFrom = *
/opt/splunkforwarder/etc/system/default/inputs.conf connection_host = dns
/opt/splunkforwarder/etc/system/local/inputs.conf host = vaaasu4054.vld04.local
/opt/splunkforwarder/etc/system/default/inputs.conf index = default
/opt/splunkforwarder/etc/apps/meos_forwarders/local/inputs.conf [tcp://8088]
/opt/splunkforwarder/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunkforwarder/etc/system/local/inputs.conf host = vaaasu4054.vld04.local
/opt/splunkforwarder/etc/system/default/inputs.conf index = default
/opt/splunkforwarder/etc/apps/meos_forwarders/local/inputs.conf persistentQueueSize = 2GB
/opt/splunkforwarder/etc/apps/meos_forwarders/local/inputs.conf queueSize = 500KB

When I send data to the UF on port 8088 with an curl command it stops processing. But the persistentQueue is not growing. Why not?

Re: UF persistentQueue tcp,UF persistentQueueSize tcp://8088

MuS — Wed, 11 Feb 2015 09:37:10 GMT

Hard to say. I would start troubleshooting by increasing the log channels of TcpInputConfig TcpInputConn and TcpInputProc in log.cfg http://docs.splunk.com/Documentation/Splunk/6.2.1/Troubleshooting/Enabledebuglogging