Splunk Enterprise

Encountered Streaming S2S error=Too many fields.

hrawat
Splunk Employee
Splunk Employee
ERROR TcpInputProc [nnnnnn FwdDataReceiverThread] - Encountered Streaming S2S error=Too many fields. fieldCount=nnnnn with source::<channel> for data received from src=x.x.x.x:nnnn.

Where <channel> has all the required info about troubled sourcetype/source/host.
Labels (1)
Tags (1)
0 Karma

hrawat
Splunk Employee
Splunk Employee

There is a hardcoded 256K (262144) fields limit on S2S receiver. So the connection will be terminated. It's likely that HF/SH will retry sending same data again, thus blocking HF/SH permanently.   
Check if there is any issue with the field extraction on FWD side. After all 256K fields are too many for an event.
Assuming you still need that event with 256K+ fields.

Here is what you do.
1. Move all props.conf/transforms.conf settings for the input source/sourcetype in question. ( Note ERROR log on indexing side provides source/sourcetype/host).
2. Add following config in the inputs.conf of the source stanza in question so that parsing is moved from HF/SH to IDX tier.
queue = indexQueue

richgalloway
SplunkTrust
SplunkTrust

Can you tell us more about this setting?  The inputs.conf.spec file says setting the value to "indexQueue" sends data 'directly into the index', implying no parsing is done (is that even possible?).  Under what conditions would we use indexQueue?

---
If this reply helps you, Karma would be appreciated.
0 Karma

PickleRick
SplunkTrust
SplunkTrust

That might be a bit wrongly worded but it's used here for example

https://docs.splunk.com/Documentation/Splunk/9.1.2/Forwarding/Routeandfilterdatad#Keep_specific_even...

If you look at Masa diagrams you'll see which one the indexQueue is

By default splunktcp input routes events into different queues depending on which keys are present in the data so if the data is not parsed, it's getting into the parsingQueue and so on. Check the system/default/inputs.conf

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...