Why is Automated lookup using kvstore collection not working?

[wmuselle]

I have created a collection in app/local/collections.conf

a matching lookup in app/local/transforms.conf

I have 5 key fields which together for the unique key, the combination of these is also stored in the _key field.

The data is populated from an index which is filled from a dbconnect source, and automatically updated up into to collection. All this works just fine.

when I use the lookup in SPL using the five fields as input, I nicely get referenced data back. when I create this lookup as part of a data model, it also provides the extra fields in the datamodel.

However if I try to use this in an automated lookup, I cannot get it to work.

I have verified the correct use of the sourcetype (and also tried defining against source)

I have verified the rights and tried using all on app and global level

I have duplicated the full config on a csv file and this works just fine

but against the kvstore the automatic lookup just wont work.

illustration of the files and configs

collections.conf  in app/local
[my_collection]
field.inputfield1 = string
field.inputfield2 = string
field.inputfield3 = string
field.inputfield4 = string
field.inputfield5 = string
field.outputfield1 = string
...

transforms.conf  in app/local
[my_collection_lookup]
external_type = kvstore
collection = my_collection
fields_list = _key, inputfield1, inputfield2,inputfield3,inputfield4,inputfield5, outputfield1 ...

props.conf in app/local
[sourcetype_stanza]
LOOKUP-enrich_kv = my_collection_lookup inputfield1 AS datafield1 inputfield2 AS datafield2 inputfield3 AS datafield3 inputfield4 AS datafield4 inputfield5 as datafield5 OUTPUTNEW _key as key outputfield1 ....

any experiences/thoughts/ideas ?

[wmuselle · Accepted solution]

found it for reference :

https://docs.splunk.com/Documentation/Splunk/8.1.3/Knowledge/Makeyourlookupautomatic 

Enable replication for a KV store collection
In Splunk Enterprise, KV Store collections are not bundle-replicated to indexers by default, and lookups run locally on the search head rather than on remote peers. When you enable replication for a KV Store collection, you can run the lookups on your indexers which let you use automatic lookups with your KV Store collections.

To enable replication for a KV Store collection and allow lookups against that collection to be automatic:

• Open collections.conf.
• Set replicate to true in the stanza for the collection.

This parameter is set to false by default.
Restart Splunk Enterprise to apply your changes.

[DmitriyGolovnya]

Hi! may be you know why after adding replicate = true to KV lookup in collections.conf, my datamodel’s lookup isn’t working, even in datamodel editor (preview). it just can’t join, all output fields is empty (without replicate all works fine)?