Splunk Enterprise

After Upgrade from Splunk 7.2.3 to Splunk 8.0.1 we get error TcpInputProc - Encountered Streaming S2S error=Received reference to unknown channel_code=132

QuintonS
Path Finder

Hi,

I am dealing with an issue where after upgrading our Splunk environment from 7.2.3 to 8.0.1 we are having endless errrors as stated in the title on the indexers within the cluster.
Error - 01-23-2020 15:58:09.056 +0200 ERROR TcpInputProc - Encountered Streaming S2S error=Received reference to unknown channel_code=132 for data received from src=1

Data flow is - UF --> Heavy Forwarder --> Indexer

Anyone that can shed some light on this?

Tags (1)
1 Solution

yaasirvatham_sp
Splunk Employee
Splunk Employee

In the Heavy Forwarders, You have to go to $SPLUNK_HOME/etc/system/local/Outputs.conf and add the value "negotiateProtocolLevel = 0" under the stanza [tcpout] then restart Splunk service.

After you add that value in the configuration file, Splunk will start to use the old protocol to connect with indexers and the connection should be established again.

[tcpout]
negotiateProtocolLevel = 0

View solution in original post

0 Karma

QuintonS
Path Finder

Thank you for the response, this solved my issue. Just another question is this only for the Heavy Forwarder to indexer or would it also be applicable from UF to Heavy Forwarder?

yaasirvatham_sp
Splunk Employee
Splunk Employee

In the Heavy Forwarders, You have to go to $SPLUNK_HOME/etc/system/local/Outputs.conf and add the value "negotiateProtocolLevel = 0" under the stanza [tcpout] then restart Splunk service.

After you add that value in the configuration file, Splunk will start to use the old protocol to connect with indexers and the connection should be established again.

[tcpout]
negotiateProtocolLevel = 0

0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

The correct workaround should have been

 

 

[tcpout]
negotiateProtocolLevel = 5

 

 

 negotiateProtocolLevel = 0 is no longer valid (see enableOldS2SProtocol in 9.1.x outputs.conf)

with 9.1.x and is likely to cause issues.

0 Karma

QuintonS
Path Finder

Thank you for the response, this solved my issue. Just another question is this only for the Heavy Forwarder to indexer or would it also be applicable from UF to Heavy Forwarder?

0 Karma

QuintonS
Path Finder

I am asking since the Heavy Forwarders have also been upgraded to 8.0.1 but the UF's are still running 7.2.3 and are in the process of being upgraded.

0 Karma

andreasz
Path Finder

My Heavy Forwarders and Indexers are at version 8.0.2 and I still get the error. Why should we set the negotiateProtocolLevel to 0, if both servers (HF & Indexer) are already at the newest version?

0 Karma

arcsight_guru
Explorer

Support confirmed that this is a bug (SPL-182112) for S2S communication between 8.x nodes. In my case I had issues between SH and INX. The recommendation was to set negotiateProtocolLevel=5 to downgrade the protocol version to 7.3. This can be done in the [tcpout] stanza on the sending node (SH), or in the [splunktcp] stanza on the receiving end (INX).

0 Karma

jhomerlopez
Explorer

Hi, this was be solved on my environment by applying the below config on outputs.conf on your HeavyForwarder.

[tcpout]
negotiateProtocolLevel = 0

Once applied, you need to restart splunk service.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...