Deployment Architecture

ProcessTracker / fsck / bloomfilter - Failed: Failed to move bloomfilter to final dir:

lmyrefelt
Builder

HI,

After trying to move and separate the bloomfilter creation to a separate disk(s) and using a volume-stanza for bloomfilters as follows .. i also configured a longer range (365 days) to generate the bloomfilters for :
[default]
maxBloomBackfillBucketAge = 365d
[volume:bloom]
path = $SPLUNK_BLOOM/

$SPLUNK_BLOOM is configured under splunk-launch.conf ;
SPLUNK_BLOOM=/opt/splunk_db/cold_frozen2/bloom

And i can see it creates all the directories that it needs
/opt/splunk_db/cold_frozen2/bloom//bloomfilter/db_1423610749_1423588528_711/

02-11-2015 10:22:13.632 +0100 INFO ProcessTracker - (child_273__Fsck) Fsck - (bloomfilter only) Rebuild for bucket='/opt/splunk_db/hot_warm//db/db_1413936005_1413636034_85' took 31.29 seconds

02-11-2015 10:22:13.632 +0100 WARN ProcessTracker - (child_273__Fsck) Fsck - Write of bloomfilter to='/opt/splunk_db/cold_frozen2/bloom/index-name/bloomfilter/db_1413936005_1413636034_85' failed: Failed to move bloomfilter to final dir: Cannot rename src="/opt/splunk_db/hot_warm/index-name/db/db_1413936005_1413636034_85-tmp/bloomfilter.tmp" to dst="/opt/splunk_db/cold_frozen2/bloom/index-name/bloomfilter/db_1413936005_1413636034_85/bloomfilter": Invalid cross-device link

After / On a restart i get the message that fsck started to rebuild bloomfilters ... however the path they these messages are pointing to is the old path / standard path to "hot / Warm" .

I belive

Invalid cross-device link

might be a key here ?

ANyone else that have been moving around on the bloomfilter structure ?

Edit: i found this one that is also related to the issue;
02-11-2015 10:53:26.250 +0100 WARN BucketMover - BucketManifestUpdateExitHandler: process handling bucket="db_1413936005_1413636034_85" exited with code=102; search for any previous messages that might have been produced by the external process

0 Karma
1 Solution

lmyrefelt
Builder

The problem i have is that it does not seem to generate / being able to move the temporary bloom filters its seems to create to its final a new directory on separate disks specified / configured as volumes inside splunk as per described in indexes.conf.spec .

Support case / Enchantment request:
224317

View solution in original post

0 Karma

lmyrefelt
Builder

The problem i have is that it does not seem to generate / being able to move the temporary bloom filters its seems to create to its final a new directory on separate disks specified / configured as volumes inside splunk as per described in indexes.conf.spec .

Support case / Enchantment request:
224317

0 Karma

lmyrefelt
Builder

This has been "accepted" as an bug and should be scheduled to be fixed, according to support it should be pretty easy.
:)

Closing it.

0 Karma

mikaelbje
Motivator

Recommendation from case 224317: This issue was fixed by SPL-99961 after review by Dev Team on your issue. Fix Version/s: 6.2.4

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...