Splunk Search

Whitelist a lookup for bundle replication

Influencer

I blacklist lookups from bundle replication by size in distsearch.conf as below

[replicationSettings]
excludeReplicatedLookupSize = 2

I now have a requirement to bypass the above condition for a specific lookup that is greater than 2 MB.
Is there a way I can craft the white list to take precedence just for the lookup that I need?
The reason I need this as part of the bundle is because I use this lookup as an auto lookup and is growing in size.

Labels (1)
0 Karma

SplunkTrust
SplunkTrust

Can you put the lookup in an app and deploy it to your search heads and indexers?

0 Karma

Influencer

Hi @jkat54 the lookup is auto generated on a daily basis from a search and new records are added every day. Having to push the app to search heads and indexer will be a manual process every day.

0 Karma

Motivator

Hello @gpradeepkumarreddy,

not a response that you asking, but a suggestion anyway:

  • is switching to KVstore instead of static lookup an option?

Please consider KV-Store vs CSV lookup:
https://dev.splunk.com/enterprise/docs/developapps/kvstore/#The-KV-Store-vs-CSV-files

https://dev.splunk.com/enterprise/docs/developapps/kvstore/migrateyourappfromusingcsv/

  • another options is to use gziped CSV files.
0 Karma

Influencer

Hi @PavelP can you provide any pointers for using gziped csv files?

0 Karma

Communicator

if you append .gz to the csv file name, it will automatically compress/decompress the resulting lookup file.

e.g. | outputlookup lookup.csv
becomes
| outputlookup lookup.csv.gz

Can save a lot of space.

Obviously there are caveats.
You cannot append to a compressed lookup

0 Karma

Influencer

Good to know. In my case the lookup gets appended every day with new records. So I guess not an option for me.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!