Deployment Architecture

Search results may be incomplete, peer 's search ended prematurely

kaizidorfa
Engager

Hello,

I'm occasionally getting the above error on splunk web but I'm not sure where to start troubleshooting it. Any tips on what could be causing it?

Thanks

svasan_splunk
Splunk Employee
Splunk Employee

kaizidorfa,

Are you using clustering on ec2? We have noticed some weird clock behaviour on ec2 which was causing some problems. (The peers were thought to have timed out when the clock skips backwards which it seems to do every now and then. The peers then have to re-add themselves and this forces them to reject searches with the old generation).

This is fixed in an upcoming 5.0.x version (5.0.5 i think)

0 Karma

lspringer
Path Finder

We had this issue just recently and it turned out to be a problem where time was drifting too far apart on the Cluster Peers. Check the status of ntp on the servers.

Specfic error:

Search results may be incomplete, peer splunksearch01's search ended prematurely. Error = master/searhhead needs to fixup/re-synchronize generation state before this peer=A64795F9-1196-4D42-FF8F-B98A2E71A719 can participate in this search [ gen=245 baseGen=246 ]

sloshburch
Splunk Employee
Splunk Employee

I am also seeing this but NTP looks right. Any other ideas?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...