Knowledge Bundle

[CarsonZa]

I was investigating bundle sizes coming from one of my SHC and came across several apps in the bundle that had the following in the lookup directory. Qualys is just one example there are several other apps where index.default and index.alive are present. Can someone tell me what these are and what they're doing in a knowledge bundle.

qualys_kb.csv_1534282613.index.default

qualys_kb.csv_1643803241.755269.cs.index.alive

[burwell]

Hi. Have you looked at the distsearch settings wrt bundles?

https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Limittheknowledgebundlesize

So in the distsearch.conf there is both replicationWhitelist and replicationBlacklist.

These are regex that specify what gets put into the knowledge bundles.

To find out exactly what is in place, use btool on your Splunk Search head and examine the setting. I like to add --debug in order that I can see exactly which app is contributing to the setting. By that I mean an app can have a distsearch.conf, you might have settings in etc/system/local/distsearch.conf etc

/opt/splunk/bin/splunk btool distsearch list replicationWhitelist --debug

/opt/splunk/bin/splunk btool distsearch list replicationBlacklist --debug

For example for me

/opt/splunk/bin/splunk btool distsearch list replicationWhitelist --debug
[replicationWhitelist]
/opt/splunk/etc/apps/splunk_archiver/default/distsearch.conf javabin = apps/splunk_archiver/java-bin/...
/opt/splunk/etc/system/default/distsearch.conf               kvstore = kvstore_*/...
/opt/splunk/etc/system/default/distsearch.conf               other = (system|(apps/(?!pdfserver)*)|users(/_reserved)?/*/*)/(bin|lookups)/...

(etc)

[CarsonZa]

Thank you for the response. I am familiar with replicationblacklist, however my questions is what are index.default and index.alive doing in a lookup directory in a knowledge bundle.

[burwell · Accepted solution]

Hi. I see people talking about the issue on Splunk's slack usersgroups instance in the admin channel.

I pinged you there.

There is mention that the .alive indicates that activity is happening.

If you don't want that in your knowledge bundle I would blacklist it, but it is a good question.

[CarsonZa]

Thank you,  im gonna add the details from Slack for anyone else who might come across this. 

"...Once a lookup exceeds the max memtable limit, Splunk will bucketify it, creating a kind of mini index."

So if you're seeing index.alive or index.default just backlist the respective lookup in distsearch.conf and in rare circumstance you could increase max_mem_usage_mb in limits.conf

[isoutamo]

I have seen even some tsidx files there… I just found those, so I haven’t have time to figure out wha5 and why those are there. I hope that someone knows that already.

Splunk 7.3.3 SHC with multisite IDX cluster.