Splunk Search

Storing queried data so that it can be accessed across search head clusters attached to the same idexer

MonkeyK
Builder

I have begun to accumulate some reference information about my company's AWS environment based on a bunch of queries.  Things like what accounts and VPCs, we have and when they were first seen (among other info).  Been happily accumulating this data into lookup tables, but now I realize that users on another Search Head Cluster would benefit from what I am doing on my SHC (which is reserved for Splunk ES)

Lookup tables don't cut it anymore since they are maintained on the SHC so their data is not available to the other SHC.

Is there a best practice on how to maintain such data so that it can be accessed from 2+ SHCs? 

Some solutions that I can think of:

  • Use a Summary Index.   Seems less than ideal because I am shooting for current state including some past info.  So using  a summary index would probably involve rewriting the current state of objects tracked -would not be the worst thing in the world to rewrite a few thousand entries daily, but I feel like an updatable source is more sensible.     
  • Just build all of the KOs in each environment.  This incurs the cost of maintaining all KOs in each environment.

are there other ways to approach what I want?

(I'm really hoping that there is an answer like "you can make a KV store on the indexer")

0 Karma

richgalloway
SplunkTrust
SplunkTrust
How about using rsync or something similar to copy the lookups from ES to the other SHC?
---
If this reply helps you, an upvote would be appreciated.

MonkeyK
Builder

Thanks, that is certainly among possible solutions, but unfortunately not available to me.

I do not have access to the box, so I cannot set up rsync on a file.  Also my PS consultant has bemoaned that she is not allowed to use rsync in our AWS environment.

 

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!