Splunk Enterprise

After Upgrade from Splunk 7.2.3 to Splunk 8.0.1 we get error TcpInputProc - Encountered Streaming S2S error=Received reference to unknown channel_code=132

QuintonS
Path Finder

Hi,

I am dealing with an issue where after upgrading our Splunk environment from 7.2.3 to 8.0.1 we are having endless errrors as stated in the title on the indexers within the cluster.
Error - 01-23-2020 15:58:09.056 +0200 ERROR TcpInputProc - Encountered Streaming S2S error=Received reference to unknown channel_code=132 for data received from src=1

Data flow is - UF --> Heavy Forwarder --> Indexer

Anyone that can shed some light on this?

Tags (1)
1 Solution

yaasirvatham_sp
Splunk Employee
Splunk Employee

In the Heavy Forwarders, You have to go to $SPLUNK_HOME/etc/system/local/Outputs.conf and add the value "negotiateProtocolLevel = 0" under the stanza [tcpout] then restart Splunk service.

After you add that value in the configuration file, Splunk will start to use the old protocol to connect with indexers and the connection should be established again.

[tcpout]
negotiateProtocolLevel = 0

View solution in original post

0 Karma

QuintonS
Path Finder

Thank you for the response, this solved my issue. Just another question is this only for the Heavy Forwarder to indexer or would it also be applicable from UF to Heavy Forwarder?

yaasirvatham_sp
Splunk Employee
Splunk Employee

In the Heavy Forwarders, You have to go to $SPLUNK_HOME/etc/system/local/Outputs.conf and add the value "negotiateProtocolLevel = 0" under the stanza [tcpout] then restart Splunk service.

After you add that value in the configuration file, Splunk will start to use the old protocol to connect with indexers and the connection should be established again.

[tcpout]
negotiateProtocolLevel = 0

0 Karma

QuintonS
Path Finder

Thank you for the response, this solved my issue. Just another question is this only for the Heavy Forwarder to indexer or would it also be applicable from UF to Heavy Forwarder?

0 Karma

QuintonS
Path Finder

I am asking since the Heavy Forwarders have also been upgraded to 8.0.1 but the UF's are still running 7.2.3 and are in the process of being upgraded.

0 Karma

andreasz
Path Finder

My Heavy Forwarders and Indexers are at version 8.0.2 and I still get the error. Why should we set the negotiateProtocolLevel to 0, if both servers (HF & Indexer) are already at the newest version?

0 Karma

arcsight_guru
Engager

Support confirmed that this is a bug (SPL-182112) for S2S communication between 8.x nodes. In my case I had issues between SH and INX. The recommendation was to set negotiateProtocolLevel=5 to downgrade the protocol version to 7.3. This can be done in the [tcpout] stanza on the sending node (SH), or in the [splunktcp] stanza on the receiving end (INX).

0 Karma

jhomerlopez
Explorer

Hi, this was be solved on my environment by applying the below config on outputs.conf on your HeavyForwarder.

[tcpout]
negotiateProtocolLevel = 0

Once applied, you need to restart splunk service.

0 Karma
Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...