Deployment Architecture

ProcessTracker / fsck / bloomfilter - Failed: Failed to move bloomfilter to final dir:

lmyrefelt
Builder

HI,

After trying to move and separate the bloomfilter creation to a separate disk(s) and using a volume-stanza for bloomfilters as follows .. i also configured a longer range (365 days) to generate the bloomfilters for :
[default]
maxBloomBackfillBucketAge = 365d
[volume:bloom]
path = $SPLUNK_BLOOM/

$SPLUNK_BLOOM is configured under splunk-launch.conf ;
SPLUNK_BLOOM=/opt/splunk_db/cold_frozen2/bloom

And i can see it creates all the directories that it needs
/opt/splunk_db/cold_frozen2/bloom//bloomfilter/db_1423610749_1423588528_711/

02-11-2015 10:22:13.632 +0100 INFO ProcessTracker - (child_273__Fsck) Fsck - (bloomfilter only) Rebuild for bucket='/opt/splunk_db/hot_warm//db/db_1413936005_1413636034_85' took 31.29 seconds

02-11-2015 10:22:13.632 +0100 WARN ProcessTracker - (child_273__Fsck) Fsck - Write of bloomfilter to='/opt/splunk_db/cold_frozen2/bloom/index-name/bloomfilter/db_1413936005_1413636034_85' failed: Failed to move bloomfilter to final dir: Cannot rename src="/opt/splunk_db/hot_warm/index-name/db/db_1413936005_1413636034_85-tmp/bloomfilter.tmp" to dst="/opt/splunk_db/cold_frozen2/bloom/index-name/bloomfilter/db_1413936005_1413636034_85/bloomfilter": Invalid cross-device link

After / On a restart i get the message that fsck started to rebuild bloomfilters ... however the path they these messages are pointing to is the old path / standard path to "hot / Warm" .

I belive

Invalid cross-device link

might be a key here ?

ANyone else that have been moving around on the bloomfilter structure ?

Edit: i found this one that is also related to the issue;
02-11-2015 10:53:26.250 +0100 WARN BucketMover - BucketManifestUpdateExitHandler: process handling bucket="db_1413936005_1413636034_85" exited with code=102; search for any previous messages that might have been produced by the external process

0 Karma
1 Solution

lmyrefelt
Builder

The problem i have is that it does not seem to generate / being able to move the temporary bloom filters its seems to create to its final a new directory on separate disks specified / configured as volumes inside splunk as per described in indexes.conf.spec .

Support case / Enchantment request:
224317

View solution in original post

0 Karma

lmyrefelt
Builder

The problem i have is that it does not seem to generate / being able to move the temporary bloom filters its seems to create to its final a new directory on separate disks specified / configured as volumes inside splunk as per described in indexes.conf.spec .

Support case / Enchantment request:
224317

0 Karma

lmyrefelt
Builder

This has been "accepted" as an bug and should be scheduled to be fixed, according to support it should be pretty easy.
:)

Closing it.

0 Karma

mikaelbje
Motivator

Recommendation from case 224317: This issue was fixed by SPL-99961 after review by Dev Team on your issue. Fix Version/s: 6.2.4

0 Karma
Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

REGISTER NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more ...

Security Highlights | November 2022 Newsletter

 November 2022 2022 Gartner Magic Quadrant for SIEM: Splunk Named a Leader for the 9th Year in a RowSplunk is ...

Platform Highlights | November 2022 Newsletter

 November 2022 Skill Up on Splunk with our New Builder Tech Talk SeriesCan you build it? Yes you can! *play ...