Splunk Enterprise

Encountered Streaming S2S error=Too many fields.

hrawat_splunk
Splunk Employee
Splunk Employee
ERROR TcpInputProc [nnnnnn FwdDataReceiverThread] - Encountered Streaming S2S error=Too many fields. fieldCount=nnnnn with source::<channel> for data received from src=x.x.x.x:nnnn.

Where <channel> has all the required info about troubled sourcetype/source/host.
Labels (1)
Tags (1)
0 Karma

hrawat_splunk
Splunk Employee
Splunk Employee

There is a hardcoded 256K (262144) fields limit on S2S receiver. So the connection will be terminated. It's likely that HF/SH will retry sending same data again, thus blocking HF/SH permanently.   
Check if there is any issue with the field extraction on FWD side. After all 256K fields are too many for an event.
Assuming you still need that event with 256K+ fields.

Here is what you do.
1. Move all props.conf/transforms.conf settings for the input source/sourcetype in question. ( Note ERROR log on indexing side provides source/sourcetype/host).
2. Add following config in the inputs.conf of the source stanza in question so that parsing is moved from HF/SH to IDX tier.
queue = indexQueue

richgalloway
SplunkTrust
SplunkTrust

Can you tell us more about this setting?  The inputs.conf.spec file says setting the value to "indexQueue" sends data 'directly into the index', implying no parsing is done (is that even possible?).  Under what conditions would we use indexQueue?

---
If this reply helps you, Karma would be appreciated.
0 Karma

PickleRick
SplunkTrust
SplunkTrust

That might be a bit wrongly worded but it's used here for example

https://docs.splunk.com/Documentation/Splunk/9.1.2/Forwarding/Routeandfilterdatad#Keep_specific_even...

If you look at Masa diagrams you'll see which one the indexQueue is

By default splunktcp input routes events into different queues depending on which keys are present in the data so if the data is not parsed, it's getting into the parsingQueue and so on. Check the system/default/inputs.conf

0 Karma
Get Updates on the Splunk Community!

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...

New Articles from Academic Learning Partners, Help Expand Lantern’s Use Case Library, ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Your Guide to SPL2 at .conf24!

So, you’re headed to .conf24? You’re in for a good time. Las Vegas weather is just *chef’s kiss* beautiful in ...