Splunk Enterprise Security

How to resolve replication errors on knowledge bundle size over 200MB due to Splunk Enterprise Security identities and assets?

Path Finder


I'm looking for some answer and suggestion how I could decrease/workaround the knowledge bundle replication errors we're seeing in our environment which is a multi site indexer clustering with simple search head (no clustering, pooling, etc) that is installed with Enterprise Security.

Identities_expanded.csv and assets_by_asn.csv are HUGE, I mean just these 2 files are 150MB+. This makes the knowledge bundle huge as well which we need to replicate between Europe and US across the WAN where our 2 Splunk sites reside. 4-4 indexers / site.

If I blacklist the csv files, then I'm full of errors during search - complaining about missing lookups.

Do you have any workaround/suggestion how we could tackle this problem?


0 Karma


Did you ever resolve this issue?

0 Karma


Are you getting timeout errors? You could try bumping up that timeout clock for these.

A lot of apps with large lookups also recommend that you put a copy of them onto the indexers themselves, however since this is probably not recommended for ES and generally overkill, you could try making your own custom app containing the CSV files and the transforms.conf lookup stanzas then deploy it to your index clusters and see if that helps, since it should be able to detect and use the lookup files locally at that point.

0 Karma

Path Finder

Thanks, however timeout is already increased to 60000 (seconds) - didn't really help. Timeouts and especially delays/lags happen upon searches very frequently. Not to mention the network folks are pretty upset because we're putting unnecessary load on the WAN connection between US and EU (4x200MB transmitted just for the knowledge bundle replication).

0 Karma

Tune In & Win!

Don't miss out on your
chance to take home free
prizes by helping our players
save the Splunk Cloudom!

Dungeons & Data
Monsters: Splunk O11y
Day Editions Games
stream live:
5/4 at 6:30pm PST
5/5 at 7:00pm PST