Getting Data In

Documented recipe for replicating subsets of data to 3rd party system throws errors in 4.3.1 and 4.3.2

Splunk Employee
Splunk Employee

In 4.3.1 and 4.3.2, when using the documented procedure to replicate subsets of data to 3rd party systems found here:

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Routeandfilterdatad#Replicate_a_subset_of_...

and using the following code recipe in $splunk_home/etc/system/local/outputs.conf:

[tcpout]
defaultGroup = nothing

[tcpout:firstlocation]
disabled = false
server = foo.bar1.com:9998
sendCookedData = false
indexAndForward = true

[tcpout:secondlocation]
disabled = false
server = foo.bar2.com:9998
sendCookedData=false
indexAndForward = true

splunk creates a large amount of errors in splunkd.log in this format:

04-24-2012 09:14:44.078 -0500 ERROR splunklogger - Uncaught exception in pipeline execution (tcp-output-generic-processor) - getting next event
04-24-2012 09:14:44.078 -0500 ERROR pipeline - Runtime exception in pipeline: indexerPipe, processor: tcp-output-generic-processor, error: vector::_M_range_check
04-24-2012 09:14:44.078 -0500 ERROR splunklogger - Uncaught exception in pipeline execution (tcp-output-generic-processor) - getting next event
04-24-2012 09:14:44.078 -0500 ERROR pipeline - Runtime exception in pipeline: indexerPipe, processor: tcp-output-generic-processor, error: vector::_M_range_check

and results in the data not being written to any index.

1 Solution

Splunk Employee
Splunk Employee

This is a known bug in 4.3.1 and 4.3.2. The bug is SPL-50576 and has been fixed and will be released with the 4.3.3 version of splunk. In the mean-time, the work around is as follows:

You must define a [tcpout:nothing], give it a false address, and tell it to drop events on queue full as shown in below example:

[tcpout]
defaultGroup = nothing

[tcpout:nothing]
disabled = false
server = falsefoo.bar.com:9998
indexAndForward = true 
dropEventsOnQueueFull = 1

[tcpout:firstlocation]
disabled = false
server = foo.bar1.com:9998
sendCookedData = false
indexAndForward = true

[tcpout:secondlocation]
disabled = false
server = foo.bar2.com:9998
sendCookedData=false
indexAndForward = true

This will circumvent the problem with routing to a non-existant group and index all data as normal, also routing any other data you have set up to go to the other output stanzas as expected.

View solution in original post

Splunk Employee
Splunk Employee

This is a known bug in 4.3.1 and 4.3.2. The bug is SPL-50576 and has been fixed and will be released with the 4.3.3 version of splunk. In the mean-time, the work around is as follows:

You must define a [tcpout:nothing], give it a false address, and tell it to drop events on queue full as shown in below example:

[tcpout]
defaultGroup = nothing

[tcpout:nothing]
disabled = false
server = falsefoo.bar.com:9998
indexAndForward = true 
dropEventsOnQueueFull = 1

[tcpout:firstlocation]
disabled = false
server = foo.bar1.com:9998
sendCookedData = false
indexAndForward = true

[tcpout:secondlocation]
disabled = false
server = foo.bar2.com:9998
sendCookedData=false
indexAndForward = true

This will circumvent the problem with routing to a non-existant group and index all data as normal, also routing any other data you have set up to go to the other output stanzas as expected.

View solution in original post