Deployment Architecture

ProcessTracker / fsck / bloomfilter - Failed: Failed to move bloomfilter to final dir:

lmyrefelt
Builder

HI,

After trying to move and separate the bloomfilter creation to a separate disk(s) and using a volume-stanza for bloomfilters as follows .. i also configured a longer range (365 days) to generate the bloomfilters for :
[default]
maxBloomBackfillBucketAge = 365d
[volume:bloom]
path = $SPLUNK_BLOOM/

$SPLUNK_BLOOM is configured under splunk-launch.conf ;
SPLUNK_BLOOM=/opt/splunk_db/cold_frozen2/bloom

And i can see it creates all the directories that it needs
/opt/splunk_db/cold_frozen2/bloom//bloomfilter/db_1423610749_1423588528_711/

02-11-2015 10:22:13.632 +0100 INFO ProcessTracker - (child_273__Fsck) Fsck - (bloomfilter only) Rebuild for bucket='/opt/splunk_db/hot_warm//db/db_1413936005_1413636034_85' took 31.29 seconds

02-11-2015 10:22:13.632 +0100 WARN ProcessTracker - (child_273__Fsck) Fsck - Write of bloomfilter to='/opt/splunk_db/cold_frozen2/bloom/index-name/bloomfilter/db_1413936005_1413636034_85' failed: Failed to move bloomfilter to final dir: Cannot rename src="/opt/splunk_db/hot_warm/index-name/db/db_1413936005_1413636034_85-tmp/bloomfilter.tmp" to dst="/opt/splunk_db/cold_frozen2/bloom/index-name/bloomfilter/db_1413936005_1413636034_85/bloomfilter": Invalid cross-device link

After / On a restart i get the message that fsck started to rebuild bloomfilters ... however the path they these messages are pointing to is the old path / standard path to "hot / Warm" .

I belive

Invalid cross-device link

might be a key here ?

ANyone else that have been moving around on the bloomfilter structure ?

Edit: i found this one that is also related to the issue;
02-11-2015 10:53:26.250 +0100 WARN BucketMover - BucketManifestUpdateExitHandler: process handling bucket="db_1413936005_1413636034_85" exited with code=102; search for any previous messages that might have been produced by the external process

0 Karma
1 Solution

lmyrefelt
Builder

The problem i have is that it does not seem to generate / being able to move the temporary bloom filters its seems to create to its final a new directory on separate disks specified / configured as volumes inside splunk as per described in indexes.conf.spec .

Support case / Enchantment request:
224317

View solution in original post

0 Karma

lmyrefelt
Builder

The problem i have is that it does not seem to generate / being able to move the temporary bloom filters its seems to create to its final a new directory on separate disks specified / configured as volumes inside splunk as per described in indexes.conf.spec .

Support case / Enchantment request:
224317

0 Karma

lmyrefelt
Builder

This has been "accepted" as an bug and should be scheduled to be fixed, according to support it should be pretty easy.
:)

Closing it.

0 Karma

mikaelbje
Motivator

Recommendation from case 224317: This issue was fixed by SPL-99961 after review by Dev Team on your issue. Fix Version/s: 6.2.4

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...