Can I update KV store through a search?

[pjvarjani]

Hi,

I am doing a POC to check if KV Store is a better option as compared to a traditional file lookup for my app. So I did the following things.

1. Initialized KV Store through collection.conf
2. removed all existing file-based lookups.
3. changed searches in savedsearch.conf to populate KV Store instead of a traditional lookup.

I am able to configure & use KV Store without any issue. But how I am getting a benefit here from using KV Store?

Let's say one of my saved search is scheduled to execute at every hour and dump hundreds of result to lookup/KV Store. Now I know each time my search executes 90% of data is going to be the similar what I have dumped previously in my lookup/KV Store. In traditional lookup I don't have option to update my lookup with selective(modified) fields. I know KV Store is gives flexibility of updating only certain records stored in it. But is it possible to update only few records in KV Store using search query?

Thanks,
Pankaj

[nekbote]

If Key field of the KVstore is unique, even though 90% of data is same, hourly search which feds data to KVstore will only add the10% that isn't already in KVStore and would update 90% which are existing if there has been change in any of their field values. Below is sample command to update KVStore using the hourly search

index=xyz |eval key=employeeid."-".employeedept |table employeeid,employeedept,key|outputlookup EmployeKVStore key_field=key append=true

[mrgibbon]

In theory you should be able to take your new data, search the KVStore for anything that doesn't match and then use the outputlookup command to store the new data inside the KVStore.
I cant help right now with the specific commands, but that approach should work.

http://dev.splunk.com/view/webframework-developapps/SP-CAAAEZH