Splunk Search

Storing queried data so that it can be accessed across search head clusters attached to the same idexer

MonkeyK
Builder

I have begun to accumulate some reference information about my company's AWS environment based on a bunch of queries.  Things like what accounts and VPCs, we have and when they were first seen (among other info).  Been happily accumulating this data into lookup tables, but now I realize that users on another Search Head Cluster would benefit from what I am doing on my SHC (which is reserved for Splunk ES)

Lookup tables don't cut it anymore since they are maintained on the SHC so their data is not available to the other SHC.

Is there a best practice on how to maintain such data so that it can be accessed from 2+ SHCs? 

Some solutions that I can think of:

  • Use a Summary Index.   Seems less than ideal because I am shooting for current state including some past info.  So using  a summary index would probably involve rewriting the current state of objects tracked -would not be the worst thing in the world to rewrite a few thousand entries daily, but I feel like an updatable source is more sensible.     
  • Just build all of the KOs in each environment.  This incurs the cost of maintaining all KOs in each environment.

are there other ways to approach what I want?

(I'm really hoping that there is an answer like "you can make a KV store on the indexer")

0 Karma

richgalloway
SplunkTrust
SplunkTrust
How about using rsync or something similar to copy the lookups from ES to the other SHC?
---
If this reply helps you, Karma would be appreciated.

MonkeyK
Builder

Thanks, that is certainly among possible solutions, but unfortunately not available to me.

I do not have access to the box, so I cannot set up rsync on a file.  Also my PS consultant has bemoaned that she is not allowed to use rsync in our AWS environment.

 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...