topic Re: BucketMover Errors in Getting Data In

BucketMover Errors

rmorlen — Tue, 12 Nov 2013 17:34:43 GMT

We are seeing about 100,000 events per hour with:

0600 ERROR BucketMover - aborting move because recursive copy from src='/splunkidx/defaultdb/db/db_1384235999_1384149600_72640' to dst='/splunkidx/defaultdb/colddb/inflight-db_1384235999_1384149600_72640' failed (reason='Too many links').

I realize the "Too many links" message is a Linux issue. Not sure why it is happening but more importantly how to I stop the error messages?

Re: BucketMover Errors

rmorlen — Mon, 28 Sep 2020 15:15:42 GMT

What I have found is that there is a Linux limit of 32,000 files in a directory and my colddb directory has hit that limit although we only keep 30 days of data.

Is there a way to manually cause a roll from cold to frozen?

Or increase the size of the warm bucket so that it is rolling to cold so often?

Settings from indexes.conf:

[main]
homePath = /splunkidx/defaultdb/db
coldPath = /splunkidx/defaultdb/colddb
thawedPath = /splunkidx/defaultdb/thaweddb
maxDataSize = auto_high_volume
maxTotalDataSizeMB = 400000
maxHotSpanSecs = 86400
frozenTimePeriodInSecs = 2592000
maxWarmDBCount = 30

Re: BucketMover Errors

rmorlen — Thu, 12 Dec 2013 13:29:06 GMT

Buckets were rolling too often from warm to cold. We were trying to get too fancy on out settings. Only needed FrozenTimePeriodInSecs = 259200, not maxWarmDBCount, not MaxHotSpacSecs.