topic Re: Restoring KV Store Collection: Socket Error or Timeout in Knowledge Management

Restoring KV Store Collection: Socket Error or Timeout

djfangBAH — Wed, 15 May 2019 21:47:51 GMT

Hi everyone!

I'm trying to restore a KV store collection with about 100K records. I am consistently encountering the following error around the 5 minute mark when the restoration process is running:

05-14-2019 10:54:16.517 -0400 ERROR KVStorageProvider - An error occurred during the last operation ('saveBatchData', domain: '2', code: '4'): Failed to send "update" command with database "s_cpz-coU94dWXKCN+jKROv8LHlXVG8O_testingcollectionBfrJQ4CnAkHyN4DnrvYrGQiU": Failed to read 4 bytes: socket error or timeout
05-14-2019 10:54:16.741 -0400 WARN  KVStoreAdminHandler - No data found to restore matching specified parameters archiveName="backedup_collections_100k.tar.gz", appName="cpz-great-app", collectionName="testingcollection"
05-14-2019 10:54:16.741 -0400 ERROR KVStoreAdminHandler - [ "{ \"ErrorMessage\" : \"Failed to send \\\"update\\\" command with database \\\"s_cpz-coU94dWXKCN+jKROv8LHlXVG8O_testingcollectionBfrJQ4CnAkHyN4DnrvYrGQiU\\\": Failed to read 4 bytes: socket error or timeout\" }" ]\n

Based discussions within thread: https://answers.splunk.com/answers/682228/attempting-to-restore-a-kvstore-collection-has-any-1.html

I have also set the following within limits.conf:

[kvstore]
max_documents_per_batch_save = 1000000
max_rows_per_query = 1000000

With it consistently erroring out around the 5 minute mark is leading me to wonder if there's a timeout value within one of the conf settings.

Any thoughts on this would be greatly appreciated!

Re: Restoring KV Store Collection: Socket Error or Timeout

nagendra1111 — Mon, 09 Dec 2019 12:21:06 GMT

are you able to solve this issue?

Re: Restoring KV Store Collection: Socket Error or Timeout

nagendra1111 — Mon, 09 Dec 2019 12:21:32 GMT

@djfangBAH

Re: Restoring KV Store Collection: Socket Error or Timeout

sirajnp — Wed, 30 Sep 2020 04:40:36 GMT

05-14-2019 10:54:16.517 -0400 ERROR KVStorageProvider - An error occurred during the last operation ('saveBatchData', domain: '2', code: '4'): Failed to send "update" command with database "s_cpz-coU94dWXKCN+jKROv8LHlXVG8O_testingcollectionBfrJQ4CnAkHyN4DnrvYrGQiU": Failed to read 4 bytes: socket error or timeout

@DavidHourani - We are restoring KVStore from one search head to another, getting above error. What could be the issue

Re: Restoring KV Store Collection: Socket Error or Timeout

DavidHourani — Sun, 22 Mar 2020 06:22:58 GMT

Hi @sirajnp,

Seems like a network error: "socket error or timeout".

Which Splunk version are you running and is it in a clustered environment ? In case it's an SHC, please make sure that port 8191 is reachable between SHC members.

Cheers,
David