Splunk Enterprise

Knowledge Bundle

CarsonZa
Contributor

I was investigating bundle sizes coming from one of my SHC and came across several apps in the bundle that had the following in the lookup directory. Qualys is just one example there are several other apps where index.default and index.alive are present. Can someone tell me what these are and what they're doing in a knowledge bundle.

qualys_kb.csv_1534282613.index.default

qualys_kb.csv_1643803241.755269.cs.index.alive

Labels (1)
1 Solution

burwell
SplunkTrust
SplunkTrust

Hi. I see people talking about the issue on Splunk's slack usersgroups instance in the admin channel.

I pinged you there.

There is mention that the .alive indicates that activity is happening.

If you don't want that in your knowledge bundle I would blacklist it, but it is a good question.

View solution in original post

burwell
SplunkTrust
SplunkTrust

Hi. Have you looked at the distsearch settings wrt bundles?

https://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Limittheknowledgebundlesize

So in the distsearch.conf there is both replicationWhitelist and replicationBlacklist.

These are regex that specify what gets put into the knowledge bundles.

To find out exactly what is in place, use btool on your Splunk Search head and examine the setting. I like to add --debug in order that I can see exactly which app is contributing to the setting. By that I mean an app can have a distsearch.conf, you might have settings in etc/system/local/distsearch.conf etc


/opt/splunk/bin/splunk btool distsearch list replicationWhitelist --debug

/opt/splunk/bin/splunk btool distsearch list replicationBlacklist --debug

 

For example for me

/opt/splunk/bin/splunk btool distsearch list replicationWhitelist --debug
[replicationWhitelist]
/opt/splunk/etc/apps/splunk_archiver/default/distsearch.conf javabin = apps/splunk_archiver/java-bin/...
/opt/splunk/etc/system/default/distsearch.conf               kvstore = kvstore_*/...
/opt/splunk/etc/system/default/distsearch.conf               other = (system|(apps/(?!pdfserver)*)|users(/_reserved)?/*/*)/(bin|lookups)/...

(etc)
0 Karma

CarsonZa
Contributor

Thank you for the response. I am familiar with replicationblacklist, however my questions is what are index.default and index.alive doing in a lookup directory in a knowledge bundle.

0 Karma

burwell
SplunkTrust
SplunkTrust

Hi. I see people talking about the issue on Splunk's slack usersgroups instance in the admin channel.

I pinged you there.

There is mention that the .alive indicates that activity is happening.

If you don't want that in your knowledge bundle I would blacklist it, but it is a good question.

CarsonZa
Contributor

Thank you,  im gonna add the details from Slack for anyone else who might come across this. 

"...Once a lookup exceeds the max memtable limit, Splunk will bucketify it, creating a kind of mini index."

So if you're seeing index.alive or index.default just backlist the respective lookup in distsearch.conf and in rare circumstance you could increase max_mem_usage_mb in limits.conf

isoutamo
SplunkTrust
SplunkTrust

I have seen even some tsidx files there… I just found those, so I haven’t have time to figure out wha5 and why those are there. I hope that someone knows that already.

Splunk 7.3.3 SHC with multisite IDX cluster.

0 Karma
Get Updates on the Splunk Community!

Splunk Certification Support Alert | Pearson VUE Outage

Splunk Certification holders and candidates!  Please be advised of an upcoming system maintenance period for ...

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...