Hi guys,
I can see how this question comes across as dumb but I would like to remove duplicated entries from my ip_intel kv store.
I understand the whole purpose of a kv_store is used for data that constantly gets updated. I am finding lots of duplicated IPs in my ip_intel kvstore and I'd like to know if there's a better way to do it than just
| inputlookup ip_intel | dedup ip
I'd like to cleanup the file from duplicates and not just create a dedup search.
Thanks! 🙂
Normally what I would do to keep the intel kvstores clean and fresh is to limit the age of the entries by running a daily cleanup job:
A simple vanilla flavored example:
|inputlookup ip_intel |eval treshold = now()-86400 |where time>treshold ||outputlookup ip_intel
Hi @weetabixsplunk i am not sure of this ip_intel kvstore, but, in first place how the duplicate entries entered this kvstore and can not you simply search the duplicate entries and delete them, i am not sure, just asking your ideas, suggestions.