Multisite Distributed Search: Why am I getting search head error "Encountered an error deserializing SearchResultsInfo from Results Stream header"?

[andrearodrigues]

Hi,

In a multisite distributed search environment with 1 search head and 4 indexers, it seems that the Search Head has difficulties to retrieve answers from the different indexers. Found this error in the search result of the search head :

ERROR SearchResultParserExecutor - Encountered an error deserializing SearchResultsInfo from ResultsStream header.

Anybody knows if it linked and how to fix it?
Splunk Entreprise 6.3.1

[andrearodrigues]

Hi,

Update :

We noticed that issue also happens when performing search on the _audit index !! As you can see below only 3 of the 4 indexers retreive informations

[s2_splunk]

I found an open bug ticket with similar symptoms. Please open a case with Splunk support and provide a diag from your search head and at least one of your indexers.

[andrearodrigues]

Hi,

Case opened with Splunk on this issue. Indexers randomly missing when performing search form the SH on various index. It seems to be linked with this error, moreover we don't have it in the search.log when all indexers are displayed in the splunk-server field (i.e for a search with no problem).

[pj]

Is this a bug? What is the SPL code? I am seeing these errors in a large environment I am currently working with. It is also running 6.3.1. Thanks. It appears to jam the search, then after a number of seconds it then continues and results are displayed.

[BP9906]

Ditto, Saw it on an adhoc search and took too long to respond on the SHC.

04-07-2016 07:37:44.753 INFO DistributedSearchResultCollectionManager - Successfully created search result collector for peer=deindexer in 0.112000 seconds
04-07-2016 07:37:44.755 INFO DistributedSearchResultCollectionManager - Successfully created search result collector for peer=irindexer1 in 0.002000 seconds
04-07-2016 07:37:44.757 INFO DistributedSearchResultCollectionManager - Successfully created search result collector for peer=irindexer2 in 0.002000 seconds
04-07-2016 07:37:44.757 INFO UserManager - Setting user context: bp9906
04-07-2016 07:37:44.757 INFO UserManager - Done setting user context: NULL -> bp9906
04-07-2016 07:37:44.761 INFO DispatchThread - Disk quota = 10485760000
04-07-2016 07:37:45.941 ERROR SearchResultParserExecutor - Encountered an error deserializing SearchResultsInfo from ResultsStream header.
04-07-2016 07:37:45.943 INFO DispatchThread - Generating results preview took 1 ms
04-07-2016 07:37:47.542 INFO DispatchThread - Generating results preview took 1 ms
04-07-2016 07:37:49.510 INFO DispatchThread - Generating results preview took 1 ms
04-07-2016 07:37:51.700 INFO DispatchThread - Generating results preview took 1 ms
04-07-2016 07:37:52.770 INFO DispatchThread - Generating results preview took 1 ms
04-07-2016 07:37:53.844 INFO DispatchThread - Generating results preview took 1 ms
04-07-2016 07:37:55.042 INFO DispatchThread - Generating results preview took 1 ms
04-07-2016 07:38:01.782 INFO DispatchThread - Generating results preview took 1 ms

[ben_leung]

plus 1; where is the SPL code to follow up on?