How to upsert to the KV Store via REST API?

[sjodle]

http://dev.splunk.com/view/SP-CAAAEY7 states that the KV store allows "upserts", but doesn't elaborate on how to perform them. Does this mean that I can perform an operation that inserts a document into the collection, or updates it if it already exists? How can I do this?

In my tests, POSTing a document with a duplicate _key to storage/collections/data/{collection} causes an error, and POSTing an update to storage/collections/data/{collection}/{id} where id doesn't exist also returns an error. Is there a single operation that's equivalent to SQL's INSERT ... ON DUPLICATE KEY UPDATE?

[dgladkikh_splun · Accepted solution]

@sjodle, this can be done using batch_save endpoint http://docs.splunk.com/Documentation/Splunk/6.3.2/RESTREF/RESTkvstore#storage.2Fcollections.2Fdata.2...

You can find few examples on this page as well http://dev.splunk.com/view/webframework-developapps/SP-CAAAEZG

As suggested by @Murali2888 - batch_save is very similar to SPL command outputlookup append=true, it inserts all new documents and replaces all documents which already exist in collection.

[sjodle]

Thanks! I'll check that out!

[Murali2888]

Hi sjodle,

This may not be what you are looking for, but an alternative option would be to use lookup/outputlookup command.
Below query would do the upserts into kvstore

Base Search | lookup <lookup_name> <lookup_field> OUTPUTNEW _key AS _key | outputlookup <lookup_name> append=true

[frobinson_splun]

Hi @sjodle,

You can create a new item by POSTing to the collection. This POST allocates the /{id} resource that you can then use for updating that item, using a POST to the /{id} endpoint.

In case it helps, having the "upsert" functionality might allow for mistakes where a user means to indicate an existing id but makes a POST using an incorrect id that happens to be non-allocated. If a new resource is created for the incorrect and non-allocated id, then the update can't be made to the correct id and the user might not find out.

Hope this helps!

[sjodle]

In case it helps, having the "upsert" functionality might allow for mistakes where a user means to indicate an existing id but makes a POST using an incorrect id that happens to be non-allocated. If a new resource is created for the incorrect and non-allocated id, then the update can't be made to the correct id and the user might not find out.

MongoDB (which the KV Store is built upon) mitigates this problem by requiring an "upsert: true" parameter to be explicitly passed, otherwise an error is returned. This ensures that the user understands the potential risks associated with upserts, and ensures that they don't happen by default without the user's knowledge.

[sjodle]

Thanks for the answer. I'll mark it as answered and proceed with my plan B: try an insert, except if it fails due to duplicate key, retry it as an update. Upserts would be an excellent feature addition in future versions, though!

[frobinson_splun]

Thanks for your suggestions, @sjodle! I've passed them along to the engineering team.