Splunk Search

Can I update KV store through a search?

pjvarjani
Path Finder

Hi,

I am doing a POC to check if KV Store is a better option as compared to a traditional file lookup for my app. So I did the following things.

  1. Initialized KV Store through collection.conf
  2. removed all existing file-based lookups.
  3. changed searches in savedsearch.conf to populate KV Store instead of a traditional lookup.

I am able to configure & use KV Store without any issue. But how I am getting a benefit here from using KV Store?

Let's say one of my saved search is scheduled to execute at every hour and dump hundreds of result to lookup/KV Store. Now I know each time my search executes 90% of data is going to be the similar what I have dumped previously in my lookup/KV Store. In traditional lookup I don't have option to update my lookup with selective(modified) fields. I know KV Store is gives flexibility of updating only certain records stored in it. But is it possible to update only few records in KV Store using search query?

Thanks,
Pankaj

0 Karma

nekbote
Path Finder

If Key field of the KVstore is unique, even though 90% of data is same, hourly search which feds data to KVstore will only add the10% that isn't already in KVStore and would update 90% which are existing if there has been change in any of their field values. Below is sample command to update KVStore using the hourly search

index=xyz |eval key=employeeid."-".employeedept |table employeeid,employeedept,key|outputlookup EmployeKVStore key_field=key append=true

mrgibbon
Contributor

In theory you should be able to take your new data, search the KVStore for anything that doesn't match and then use the outputlookup command to store the new data inside the KVStore.
I cant help right now with the specific commands, but that approach should work.

http://dev.splunk.com/view/webframework-developapps/SP-CAAAEZH

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...