Splunk Enterprise Security

How to resolve replication errors on knowledge bundle size over 200MB due to Splunk Enterprise Security identities and assets?

tkiss
Path Finder

Hi,

I'm looking for some answer and suggestion how I could decrease/workaround the knowledge bundle replication errors we're seeing in our environment which is a multi site indexer clustering with simple search head (no clustering, pooling, etc) that is installed with Enterprise Security.

Identities_expanded.csv and assets_by_asn.csv are HUGE, I mean just these 2 files are 150MB+. This makes the knowledge bundle huge as well which we need to replicate between Europe and US across the WAN where our 2 Splunk sites reside. 4-4 indexers / site.

If I blacklist the csv files, then I'm full of errors during search - complaining about missing lookups.

Do you have any workaround/suggestion how we could tackle this problem?

Thanks
tkiss

0 Karma

AndySplunks
Communicator

Did you ever resolve this issue?

0 Karma

goodsellt
Contributor

Are you getting timeout errors? You could try bumping up that timeout clock for these.

A lot of apps with large lookups also recommend that you put a copy of them onto the indexers themselves, however since this is probably not recommended for ES and generally overkill, you could try making your own custom app containing the CSV files and the transforms.conf lookup stanzas then deploy it to your index clusters and see if that helps, since it should be able to detect and use the lookup files locally at that point.

0 Karma

tkiss
Path Finder

Thanks, however timeout is already increased to 60000 (seconds) - didn't really help. Timeouts and especially delays/lags happen upon searches very frequently. Not to mention the network folks are pretty upset because we're putting unnecessary load on the WAN connection between US and EU (4x200MB transmitted just for the knowledge bundle replication).

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...