Archive
Highlighted

How to index a csv/lookup file from SH to the indexer in a SH cluster

Super Champion

hi
We have SH clustered environment and we have a lookup which is collected daily and this comes to SH and is present in all the SH cluster members correctly. But we have a requirement to index this (for legal and audit purposes), so the customer can see how the data looked like previously (say 20 days ago). What's the best way to index this csv/lookup file which is present in the SH cluster? when i thinking more and more it becomes complex and complex
(Options I thought so far: if I write an inputs.conf and pull this from a SH, this will have copy in every SH member and the inputs will be replicated too thus causing multiple copies to be indexed. Another alternative I thought is to run a Universal forwarder separately ..hmm..)

Any other easy methods to pull a lookup file from SH to Indexer?

Cheers

0 Karma
Highlighted

Re: How to index a csv/lookup file from SH to the indexer in a SH cluster

SplunkTrust
SplunkTrust

You can just use the summary indexing (saved search fetching data from lookup and saving onto index) to achieve the same. Since, it's a saved search, the SHC will take care of it's execution (once). See this on how to setup a saved search to do summary indexing.

https://docs.splunk.com/Documentation/Splunk/6.5.2/Knowledge/Usesummaryindexing#Set_up_summary_index...

View solution in original post

Highlighted

Re: How to index a csv/lookup file from SH to the indexer in a SH cluster

Super Champion

great method. Thanks mate

0 Karma
Highlighted

Re: How to index a csv/lookup file from SH to the indexer in a SH cluster

SplunkTrust
SplunkTrust

I would run the same search at the same time but remove the outputlookup command and append the collect command so that it gets collected into an index. Be sure you're forwarding events from the SHC to the indexers as per best practices first.

  ... | collect index=myIndex
Highlighted

Re: How to index a csv/lookup file from SH to the indexer in a SH cluster

Super Champion

great option too. thank you

0 Karma
Highlighted

Re: How to index a csv/lookup file from SH to the indexer in a SH cluster

Esteemed Legend

We do this using summary indexing by scheduling a search like this to run daily (or whatever the requirement is):

|inputlookup MyFile.csv | collect index=MySummaryIndex