Why is kvstore update failing with code 115?

[wpreston]

I've got a kvstore lookup who's data is updated every day from a scheduled search. I built it using the ideas that @dwaddle and @starcher presented at .conf2015 (presentation here if anyone is interested). This worked great for a few months, but now I'm getting an error message whenever the updating search runs. Neither the updating search nor structure of the data have changed so I don't think it has anything to do with the search itself. When the updating search runs, it returns a table of data but gets the following message when it tries to write to the kvstore:

"Could not append to collection 'CollectionOfIncidents': an error occurred while saving to the collection.  See search.log for more details."

When I look in search.log there is one more message:

ERROR KVStoreLookup - KV Store Lookup output failed with code -115 and message ''

Any ideas as to what this error code means or what could be causing the update to fail?

[claudio_manig]

Same here, did nromito's polst helped you out to fix it?

[briancronrath]

I know this post is old, but in case the answer here doesn't fix it for folks, I recently resolved this issue myself by removing an fields with a "." character in the name. Looks like it was breaking the ability to append.

[nromito_splunk]

The reason this message is displayed is because you're trying to write a multi-valued _key field to your KV Store.
For example:
I create a KV Store with the following values:
'{"name":"indexer1","id":123,"address":{"street":"250 Brannan","city":"San Francisco"}}'
'{"name":"indexer1","id":124,"address":{"street":"250 Brannan","city":"San Francisco"}}'
I then write a search like this:

index = _internal | head 1 | eval name = "indexer1"| lookup test_lookup name OUTPUT _key | outputlookup test_lookup append=true

This means my one event from the search on _internal will match both of the KV Store entries, and we create a new field=_key for that event due to the OUTPUT of the lookup. Since we matched two entries in the KV Store, the _key field on the event will evaluate to something like "_key" : [ "56e30ef4af0001b2aa352761", "56e30f0baf0001b2aa352762" ]. Since Splunk's KV Store only allows a single, unique value for _key, the search fails with the cryptic message ERROR KVStoreLookup - KV Store Lookup output failed with code -115 and message ''

tl;dr revise your search query, KV Store collection, or transforms.conf (max_matches=1) to ensure that you will not match an event to multiple KV Store entries when trying to write to the _key field.

[richgalloway]

What platform?

[wpreston]

Sorry, should have included that!

Splunk version: 6.3
OS: Windows 2008 R2

[robert_miller]

We are also starting to see this same error. Hopefully someone has a solution.