Deployment Architecture

The percentage of small buckets exceeded

michel_wolf
Path Finder

Hello guys,

I checked the splunk answers but I can´t find a solution for my problem. I have an indexer cluster with 2 idx and 2 sites and for my _internal index I get many small buckets. In the answers I found some notes about connection issues, but in this environment I don´t have connection problems.

All splunk instances are installed in 7.3.3

I get the following error:
alt text

I checked with the |dbinspect my _internal index and didn´t find any issues here:
|dbinspect index=_internal
|fields - splunk_server
|table startEpoch endEpoch *

alt text

Do you have any hints for me why new buckets are generated instead of using the existing one?
I don´t change many things on the default configuration for the _internal index:

/opt/splunk/bin/splunk cmd btool indexes --debug list _internal

/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf [_internal]
/opt/splunk/etc/system/default/indexes.conf archiver.enableDataArchive = false
/opt/splunk/etc/system/default/indexes.conf archiver.maxDataArchiveRetentionPeriod = 0
/opt/splunk/etc/system/default/indexes.conf assureUTF8 = false
/opt/splunk/etc/system/default/indexes.conf bucketRebuildMemoryHint = auto
/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf coldPath = volume:main/_internaldb/colddb
/opt/splunk/etc/system/default/indexes.conf coldPath.maxDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf coldToFrozenDir =
/opt/splunk/etc/system/default/indexes.conf coldToFrozenScript =
/opt/splunk/etc/system/default/indexes.conf compressRawdata = true
/opt/splunk/etc/system/default/indexes.conf datatype = event
/opt/splunk/etc/system/default/indexes.conf defaultDatabase = main
/opt/splunk/etc/system/default/indexes.conf enableDataIntegrityControl = false
/opt/splunk/etc/system/default/indexes.conf enableOnlineBucketRepair = true
/opt/splunk/etc/system/default/indexes.conf enableRealtimeSearch = true
/opt/splunk/etc/system/default/indexes.conf enableTsidxReduction = false
/opt/splunk/etc/system/default/indexes.conf fileSystemExecutorWorkers = 5
/opt/splunk/etc/system/default/indexes.conf frozenTimePeriodInSecs = 2592000
/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf homePath = volume:main/_internaldb/db
/opt/splunk/etc/system/default/indexes.conf homePath.maxDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf hotBucketTimeRefreshInterval = 10
/opt/splunk/etc/system/default/indexes.conf indexThreads = auto
/opt/splunk/etc/system/default/indexes.conf journalCompression = gzip
/opt/splunk/etc/system/default/indexes.conf maxBloomBackfillBucketAge = 30d
/opt/splunk/etc/system/default/indexes.conf maxBucketSizeCacheEntries = 0
/opt/splunk/etc/system/default/indexes.conf maxConcurrentOptimizes = 6
/opt/splunk/etc/system/default/indexes.conf maxDataSize = 1000
/opt/splunk/etc/system/default/indexes.conf maxGlobalDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf maxGlobalRawDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf maxHotBuckets = 3
/opt/splunk/etc/system/default/indexes.conf maxHotIdleSecs = 0
/opt/splunk/etc/system/default/indexes.conf maxHotSpanSecs = 432000
/opt/splunk/etc/system/default/indexes.conf maxMemMB = 5
/opt/splunk/etc/system/default/indexes.conf maxMetaEntries = 1000000
/opt/splunk/etc/system/default/indexes.conf maxRunningProcessGroups = 8
/opt/splunk/etc/system/default/indexes.conf maxRunningProcessGroupsLowPriority = 1
/opt/splunk/etc/system/default/indexes.conf maxTimeUnreplicatedNoAcks = 300
/opt/splunk/etc/system/default/indexes.conf maxTimeUnreplicatedWithAcks = 60
/opt/splunk/etc/system/default/indexes.conf maxTotalDataSizeMB = 500000
/opt/splunk/etc/system/default/indexes.conf maxWarmDBCount = 300
/opt/splunk/etc/system/default/indexes.conf memPoolMB = auto
/opt/splunk/etc/system/default/indexes.conf minHotIdleSecsBeforeForceRoll = auto
/opt/splunk/etc/system/default/indexes.conf minRawFileSyncSecs = disable
/opt/splunk/etc/system/default/indexes.conf minStreamGroupQueueSize = 2000
/opt/splunk/etc/system/default/indexes.conf partialServiceMetaPeriod = 0
/opt/splunk/etc/system/default/indexes.conf processTrackerServiceInterval = 1
/opt/splunk/etc/system/default/indexes.conf quarantineFutureSecs = 2592000
/opt/splunk/etc/system/default/indexes.conf quarantinePastSecs = 77760000
/opt/splunk/etc/system/default/indexes.conf rawChunkSizeBytes = 131072
/opt/splunk/etc/slave-apps/_cluster/default/indexes.conf repFactor = auto
/opt/splunk/etc/system/default/indexes.conf rotatePeriodInSecs = 60
/opt/splunk/etc/system/default/indexes.conf rtRouterQueueSize = 10000
/opt/splunk/etc/system/default/indexes.conf rtRouterThreads = 0
/opt/splunk/etc/system/default/indexes.conf selfStorageThreads = 2
/opt/splunk/etc/system/default/indexes.conf serviceInactiveIndexesPeriod = 60
/opt/splunk/etc/system/default/indexes.conf serviceMetaPeriod = 25
/opt/splunk/etc/system/default/indexes.conf serviceOnlyAsNeeded = true
/opt/splunk/etc/system/default/indexes.conf serviceSubtaskTimingPeriod = 30
/opt/splunk/etc/system/default/indexes.conf splitByIndexKeys =
/opt/splunk/etc/system/default/indexes.conf streamingTargetTsidxSyncPeriodMsec = 5000
/opt/splunk/etc/system/default/indexes.conf suppressBannerList =
/opt/splunk/etc/system/default/indexes.conf suspendHotRollByDeleteQuery = false
/opt/splunk/etc/system/default/indexes.conf sync = 0
/opt/splunk/etc/system/default/indexes.conf syncMeta = true
/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf thawedPath = $SPLUNK_DB/_internaldb/thaweddb
/opt/splunk/etc/system/default/indexes.conf throttleCheckPeriod = 15
/opt/splunk/etc/system/default/indexes.conf timePeriodInSecBeforeTsidxReduction = 604800
/opt/splunk/etc/system/default/indexes.conf tsidxReductionCheckPeriodInSec = 600
/opt/splunk/etc/system/default/indexes.conf tsidxWritingLevel = 1
/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf tstatsHomePath = volume:main/_internaldb/datamodel_summary
/opt/splunk/etc/system/default/indexes.conf warmToColdScript =

Thanks

Michel

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...