Splunk Search

Storing queried data so that it can be accessed across search head clusters attached to the same idexer


I have begun to accumulate some reference information about my company's AWS environment based on a bunch of queries.  Things like what accounts and VPCs, we have and when they were first seen (among other info).  Been happily accumulating this data into lookup tables, but now I realize that users on another Search Head Cluster would benefit from what I am doing on my SHC (which is reserved for Splunk ES)

Lookup tables don't cut it anymore since they are maintained on the SHC so their data is not available to the other SHC.

Is there a best practice on how to maintain such data so that it can be accessed from 2+ SHCs? 

Some solutions that I can think of:

  • Use a Summary Index.   Seems less than ideal because I am shooting for current state including some past info.  So using  a summary index would probably involve rewriting the current state of objects tracked -would not be the worst thing in the world to rewrite a few thousand entries daily, but I feel like an updatable source is more sensible.     
  • Just build all of the KOs in each environment.  This incurs the cost of maintaining all KOs in each environment.

are there other ways to approach what I want?

(I'm really hoping that there is an answer like "you can make a KV store on the indexer")

0 Karma

How about using rsync or something similar to copy the lookups from ES to the other SHC?
If this reply helps you, Karma would be appreciated.


Thanks, that is certainly among possible solutions, but unfortunately not available to me.

I do not have access to the box, so I cannot set up rsync on a file.  Also my PS consultant has bemoaned that she is not allowed to use rsync in our AWS environment.


0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...