Splunk Search

Can I update KV store through a search?

pjvarjani
Path Finder

Hi,

I am doing a POC to check if KV Store is a better option as compared to a traditional file lookup for my app. So I did the following things.

  1. Initialized KV Store through collection.conf
  2. removed all existing file-based lookups.
  3. changed searches in savedsearch.conf to populate KV Store instead of a traditional lookup.

I am able to configure & use KV Store without any issue. But how I am getting a benefit here from using KV Store?

Let's say one of my saved search is scheduled to execute at every hour and dump hundreds of result to lookup/KV Store. Now I know each time my search executes 90% of data is going to be the similar what I have dumped previously in my lookup/KV Store. In traditional lookup I don't have option to update my lookup with selective(modified) fields. I know KV Store is gives flexibility of updating only certain records stored in it. But is it possible to update only few records in KV Store using search query?

Thanks,
Pankaj

0 Karma

nekbote
Path Finder

If Key field of the KVstore is unique, even though 90% of data is same, hourly search which feds data to KVstore will only add the10% that isn't already in KVStore and would update 90% which are existing if there has been change in any of their field values. Below is sample command to update KVStore using the hourly search

index=xyz |eval key=employeeid."-".employeedept |table employeeid,employeedept,key|outputlookup EmployeKVStore key_field=key append=true

mrgibbon
Contributor

In theory you should be able to take your new data, search the KVStore for anything that doesn't match and then use the outputlookup command to store the new data inside the KVStore.
I cant help right now with the specific commands, but that approach should work.

http://dev.splunk.com/view/webframework-developapps/SP-CAAAEZH

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...