Splunk Enterprise Security

How to resolve replication errors on knowledge bundle size over 200MB due to Splunk Enterprise Security identities and assets?

tkiss
Path Finder

Hi,

I'm looking for some answer and suggestion how I could decrease/workaround the knowledge bundle replication errors we're seeing in our environment which is a multi site indexer clustering with simple search head (no clustering, pooling, etc) that is installed with Enterprise Security.

Identities_expanded.csv and assets_by_asn.csv are HUGE, I mean just these 2 files are 150MB+. This makes the knowledge bundle huge as well which we need to replicate between Europe and US across the WAN where our 2 Splunk sites reside. 4-4 indexers / site.

If I blacklist the csv files, then I'm full of errors during search - complaining about missing lookups.

Do you have any workaround/suggestion how we could tackle this problem?

Thanks
tkiss

0 Karma

AndySplunks
Communicator

Did you ever resolve this issue?

0 Karma

goodsellt
Contributor

Are you getting timeout errors? You could try bumping up that timeout clock for these.

A lot of apps with large lookups also recommend that you put a copy of them onto the indexers themselves, however since this is probably not recommended for ES and generally overkill, you could try making your own custom app containing the CSV files and the transforms.conf lookup stanzas then deploy it to your index clusters and see if that helps, since it should be able to detect and use the lookup files locally at that point.

0 Karma

tkiss
Path Finder

Thanks, however timeout is already increased to 60000 (seconds) - didn't really help. Timeouts and especially delays/lags happen upon searches very frequently. Not to mention the network folks are pretty upset because we're putting unnecessary load on the WAN connection between US and EU (4x200MB transmitted just for the knowledge bundle replication).

0 Karma
Get Updates on the Splunk Community!

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Get Inspired! We’ve Got Validation that Your Hard Work is Paying Off

We love our Splunk Community and want you to feel inspired by all your hard work! Eric Fusilero, our VP of ...

What's New in Splunk Enterprise 9.4: Features to Power Your Digital Resilience

Hey Splunky People! We are excited to share the latest updates in Splunk Enterprise 9.4. In this release we ...