Knowledge Management

Restoring KV Store Collection: Socket Error or Timeout

djfangBAH
New Member

Hi everyone!

I'm trying to restore a KV store collection with about 100K records. I am consistently encountering the following error around the 5 minute mark when the restoration process is running:

05-14-2019 10:54:16.517 -0400 ERROR KVStorageProvider - An error occurred during the last operation ('saveBatchData', domain: '2', code: '4'): Failed to send "update" command with database "s_cpz-coU94dWXKCN+jKROv8LHlXVG8O_testingcollectionBfrJQ4CnAkHyN4DnrvYrGQiU": Failed to read 4 bytes: socket error or timeout
05-14-2019 10:54:16.741 -0400 WARN  KVStoreAdminHandler - No data found to restore matching specified parameters archiveName="backedup_collections_100k.tar.gz", appName="cpz-great-app", collectionName="testingcollection"
05-14-2019 10:54:16.741 -0400 ERROR KVStoreAdminHandler - [ "{ \"ErrorMessage\" : \"Failed to send \\\"update\\\" command with database \\\"s_cpz-coU94dWXKCN+jKROv8LHlXVG8O_testingcollectionBfrJQ4CnAkHyN4DnrvYrGQiU\\\": Failed to read 4 bytes: socket error or timeout\" }" ]\n

Based discussions within thread: https://answers.splunk.com/answers/682228/attempting-to-restore-a-kvstore-collection-has-any-1.html

I have also set the following within limits.conf:

[kvstore]
max_documents_per_batch_save = 1000000
max_rows_per_query = 1000000

With it consistently erroring out around the 5 minute mark is leading me to wonder if there's a timeout value within one of the conf settings.

Any thoughts on this would be greatly appreciated!

0 Karma

DavidHourani
Super Champion

Hi @sirajnp,

Seems like a network error: "socket error or timeout".

Which Splunk version are you running and is it in a clustered environment ? In case it's an SHC, please make sure that port 8191 is reachable between SHC members.

Cheers,
David

0 Karma

nagendra1111
New Member

are you able to solve this issue?

0 Karma

nagendra1111
New Member

@djfangBAH

0 Karma

sirajnp
Path Finder

05-14-2019 10:54:16.517 -0400 ERROR KVStorageProvider - An error occurred during the last operation ('saveBatchData', domain: '2', code: '4'): Failed to send "update" command with database "s_cpz-coU94dWXKCN+jKROv8LHlXVG8O_testingcollectionBfrJQ4CnAkHyN4DnrvYrGQiU": Failed to read 4 bytes: socket error or timeout

@DavidHourani - We are restoring KVStore from one search head to another, getting above error. What could be the issue

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...