Getting Data In

UF persistentQueue tcp,UF persistentQueueSize tcp://8088

arjangoos
Path Finder

We have an application that sends it's data to the UF on tcp port 8088. When the indexers are down we want the UF to store the log data in a persistentQueue untill the indexers are online again. Is this possible?

in the inputs.conf we have this stanza:
[tcp://8088]
persistentQueueSize=2GB

But this is not working. Can anyone help me?

,We have an application that offers it's logging to the UF API on tcp://8088 and we want to use a persistentQueue to store data when the indexers would be down. Is this possible? What we did is create een app with inputs.conf:

[tcp://8088]
persistentQueueSize=2GB

But this is not working we expected.

Can someone help us?

Tags (2)
0 Karma

MuS
SplunkTrust
SplunkTrust

Hi arjangoos,

on the forwarder run this command to check if the the config is applied correct:

$SPLUNK_HOME/bin/splunk cmd btool --debug inputs list tcp

Output looks something like this and yours should have a persistentQueueSize

/opt/splunk/etc/system/default/inputs.conf [tcp]
/opt/splunk/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunk/etc/system/default/inputs.conf acceptFrom = *
/opt/splunk/etc/system/default/inputs.conf connection_host = dns
/opt/splunk/etc/system/local/inputs.conf   host = VirtualBox
/opt/splunk/etc/system/default/inputs.conf index = default

Also check if the queue directory has been created, it should be at

$SPLUNK_HOME/var/run/splunk/tcpin/pq__8088

in your case.
Check as well splunkd.log of your forwarder for any error messages.

Hope this helps ...

cheers, MuS

0 Karma

arjangoos
Path Finder

Hi MuS:

/opt/splunkforwarder/etc/system/default/inputs.conf [tcp]
/opt/splunkforwarder/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunkforwarder/etc/system/default/inputs.conf acceptFrom = *
/opt/splunkforwarder/etc/system/default/inputs.conf connection_host = dns
/opt/splunkforwarder/etc/system/local/inputs.conf host = vaaasu4054.vld04.local
/opt/splunkforwarder/etc/system/default/inputs.conf index = default
/opt/splunkforwarder/etc/apps/meos_forwarders/local/inputs.conf [tcp://8088]
/opt/splunkforwarder/etc/system/default/inputs.conf _rcvbuf = 1572864
/opt/splunkforwarder/etc/system/local/inputs.conf host = vaaasu4054.vld04.local
/opt/splunkforwarder/etc/system/default/inputs.conf index = default
/opt/splunkforwarder/etc/apps/meos_forwarders/local/inputs.conf persistentQueueSize = 2GB
/opt/splunkforwarder/etc/apps/meos_forwarders/local/inputs.conf queueSize = 500KB

When I send data to the UF on port 8088 with an curl command it stops processing. But the persistentQueue is not growing. Why not?

0 Karma

MuS
SplunkTrust
SplunkTrust

Hard to say. I would start troubleshooting by increasing the log channels of TcpInputConfig TcpInputConn and TcpInputProc in log.cfg http://docs.splunk.com/Documentation/Splunk/6.2.1/Troubleshooting/Enabledebuglogging

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...