do we need to automate KV store look ups?/

[ramarcsight]

I am currently using CSV but due to the frequent activity of CSV which is there in my Search head, there is a bundle replication problem to all indexers under it, so I am thinking to move to KV store lookup.

I got the basic idea is to create a collection in the collection.conf

so my question: I am creating a collection on SH.

if I create a collection in collections.conf do I need to restart the Splunk ES? I have one SH and few indexers under it in a cluster.

and create a collection is enough or do I need to automate it?

what is the difference between normal lookup and automate lookup?

Will kv store will help me solve the problem as I am writing and deleting from the csv more frequently and consurrently ??

[kashz]

If I understand your problem statement, its basically being able to decide whether to use CSV vs KV Stores. For more clarity, take a look at this Splunk Documentation: http://dev.splunk.com/view/SP-CAAAEY7

Index based approach: Questions that I faced during a similar dilemma was being able to retrieve latest scores from the index, one solution is to use |stats latest(field1) by field2,

But KV stores are the best approach in my personal opinion,
Normal Lookup: So you can create a lookup file (kv-store) and create a lookup definition in the settings>lookup>lookup definitions view. But that lookup is not being applied. You need to apply that lookup say, to a data model for example.

Automated lookup: Once you define a lookup, create a lookup definition and then define an automatic lookup in the settings>lookup>automatic lookups, it applied INSTANTLY to the data that splunk ingests. PS. While creating an automatic lookup, you actually specify to what it applies sourcetype or host or source allowing splunk to use that information and apply your lookup to the data when it is being ingested.