How to index a csv/lookup file from SH to the indexer in a SH cluster

[koshyk]

hi
We have SH clustered environment and we have a lookup which is collected daily and this comes to SH and is present in all the SH cluster members correctly. But we have a requirement to index this (for legal and audit purposes), so the customer can see how the data looked like previously (say 20 days ago). What's the best way to index this csv/lookup file which is present in the SH cluster? when i thinking more and more it becomes complex and complex
(Options I thought so far: if I write an inputs.conf and pull this from a SH, this will have copy in every SH member and the inputs will be replicated too thus causing multiple copies to be indexed. Another alternative I thought is to run a Universal forwarder separately ..hmm..)

Any other easy methods to pull a lookup file from SH to Indexer?

Cheers

[woodcock]

We do this using summary indexing by scheduling a search like this to run daily (or whatever the requirement is):

|inputlookup MyFile.csv | collect index=MySummaryIndex

[jkat54]

I would run the same search at the same time but remove the outputlookup command and append the collect command so that it gets collected into an index. Be sure you're forwarding events from the SHC to the indexers as per best practices first.

  ... | collect index=myIndex

[koshyk]

great option too. thank you

[somesoni2 · Accepted solution]

You can just use the summary indexing (saved search fetching data from lookup and saving onto index) to achieve the same. Since, it's a saved search, the SHC will take care of it's execution (once). See this on how to setup a saved search to do summary indexing.

https://docs.splunk.com/Documentation/Splunk/6.5.2/Knowledge/Usesummaryindexing#Set_up_summary_index...

[koshyk]

great method. Thanks mate