Splunk Enterprise

Encountered Streaming S2S error=Too many fields.

hrawat
Splunk Employee
Splunk Employee
ERROR TcpInputProc [nnnnnn FwdDataReceiverThread] - Encountered Streaming S2S error=Too many fields. fieldCount=nnnnn with source::<channel> for data received from src=x.x.x.x:nnnn.

Where <channel> has all the required info about troubled sourcetype/source/host.
Labels (1)
Tags (1)
0 Karma

hrawat
Splunk Employee
Splunk Employee

There is a hardcoded 256K (262144) fields limit on S2S receiver. So the connection will be terminated. It's likely that HF/SH will retry sending same data again, thus blocking HF/SH permanently.   
Check if there is any issue with the field extraction on FWD side. After all 256K fields are too many for an event.
Assuming you still need that event with 256K+ fields.

Here is what you do.
1. Move all props.conf/transforms.conf settings for the input source/sourcetype in question. ( Note ERROR log on indexing side provides source/sourcetype/host).
2. Add following config in the inputs.conf of the source stanza in question so that parsing is moved from HF/SH to IDX tier.
queue = indexQueue

richgalloway
SplunkTrust
SplunkTrust

Can you tell us more about this setting?  The inputs.conf.spec file says setting the value to "indexQueue" sends data 'directly into the index', implying no parsing is done (is that even possible?).  Under what conditions would we use indexQueue?

---
If this reply helps you, Karma would be appreciated.
0 Karma

PickleRick
SplunkTrust
SplunkTrust

That might be a bit wrongly worded but it's used here for example

https://docs.splunk.com/Documentation/Splunk/9.1.2/Forwarding/Routeandfilterdatad#Keep_specific_even...

If you look at Masa diagrams you'll see which one the indexQueue is

By default splunktcp input routes events into different queues depending on which keys are present in the data so if the data is not parsed, it's getting into the parsingQueue and so on. Check the system/default/inputs.conf

0 Karma
Get Updates on the Splunk Community!

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Deprecation of Splunk Observability Kubernetes “Classic Navigator” UI starting ...

Access to Splunk Observability Kubernetes “Classic Navigator” UI will no longer be available starting January ...

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...