Knowledge Management

Knowledge bundle size issues

muradgh
Path Finder

Hi All,

I have this error message on the SH in Splunk:
{

Knowledge bundle size=3525MB exceeds max limit=2048MB. Distributed searches are running against an outdated knowledge bundle. Please remove/disable files from knowledge bundle or increase maxBundleSize in distsearch.conf.

}

What I did is increase the maxBundleSize in distsearch.conf :
I did this command on the server:
/opt/splunk/bin/splunk btool distsearch list --debug | grep maxBundleSize
and the result is:
/opt/splunk/etc/system/default/distsearch.conf                  maxBundleSize = 2048
So inside the /opt/splunk/etc/system/local/distsearch.conf I added the:

[replicationSettings]
maxBundleSize = 4000

Restarted Splunk, and noticed that the first error message is gone, but a new Yellow warning appeared:
{

The current bundle directory contains a large lookup file that might cause bundle replication fail. The path to the directory is /opt/splunk/var/run/InvestBank-SH-1-1681121119-1681121612.delta.

}

muradgh_0-1681123932468.png


So I went to this path to check what is going on there:
cd /opt/splunk/var/run

muradgh_1-1681124580843.png

I have found 2 large files and one medium.
Can someone please advise me on what to do past this point? 

I have found someone posted to check the below search:
index =_internal sourcetype=splunkd component=Archiver Archiving large_file=*
| stats count latest(size_in_bytes) by large_file

muradgh_2-1681125073509.png

I don't know if this has any relation to the subject.



Please note that my Splunk environment is not a cluster.

 

Labels (1)
Tags (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Look inside the large bundle files (they're just tarballs) to see what's making them so large.  It's probably one or more huge lookup files.  Make sure the lookups are expected to be that big as it's possible a bad search is appending rather than replacing data in the lookup.

If the lookup needs to be that large then remove it from the replication bundle ([repicationDenyList] in distsearch.conf) and distribute it to the indexers via another method.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...