Clean up ip_intel kvstore

[weetabixsplunk]

Hi guys,

I can see how this question comes across as dumb but I would like to remove duplicated entries from my ip_intel kv store.

I understand the whole purpose of a kv_store is used for data that constantly gets updated. I am finding lots of duplicated IPs in my ip_intel kvstore and I'd like to know if there's a better way to do it than just 

| inputlookup ip_intel | dedup ip

I'd like to cleanup the file from duplicates and not just create a dedup search.

Thanks! 🙂

[Azeemering]

Normally what I would do to keep the intel kvstores clean and fresh is to limit the age of the entries by running a daily cleanup job:

A simple vanilla flavored example:

|inputlookup ip_intel |eval treshold = now()-86400 |where time>treshold ||outputlookup ip_intel

[inventsekar]

Hi @weetabixsplunk i am not sure of this  ip_intel kvstore, but, in first place how the duplicate entries entered this kvstore and can not you simply search the duplicate entries and delete them, i am not sure, just asking your ideas, suggestions.