Archive

Can I run splunk on btrfs?

notneeded
Explorer

Hello,

I just downloaded splunk today to try it out on a few of our servers, but found out very quickly that it doesn't support btrfs:

Filesystem type is not supported: buf.f_type = 0x9123683e
  1. Why does splunk care about the file system anyway?
  2. Is there a way to "force" btrfs support, maybe with reduced functionality?
  3. Is official support for btrfs planned?

The output of locktest looks like this:

~/splunk]% bin/locktest                                 
Could not create a lock in the SPLUNK_DB directory.
Filesystem type is not supported: buf.f_type = 0x9123683e
If supporting this filesystem type is important to you, please file an Enhancement Request with Splunk Support with the fs info number listed.

~/splunk]% ls $SPLUNK_DB 
audit/  authDb/  blockSignature/  defaultdb/  fishbucket/  hashDb/  historydb/  _internaldb/  sample/  summarydb/  test.ijKHJ9  test.R0jT0h  test.T65SU0

Output of strace:

~/splunk]% strace bin/locktest
execve("bin/locktest", ["bin/locktest"], [/* 32 vars */]) = 0
brk(0)                                  = 0x245b000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8ad5120000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=27312, ...}) = 0
mmap(NULL, 27312, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f8ad5119000
close(3)                                = 0
open("/lib64/libc.so.6", O_RDONLY)      = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\355\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1832712, ...}) = 0
mmap(NULL, 3664040, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f8ad4b84000
mprotect(0x7f8ad4cf9000, 2097152, PROT_NONE) = 0
mmap(0x7f8ad4ef9000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x175000) = 0x7f8ad4ef9000
mmap(0x7f8ad4efe000, 18600, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f8ad4efe000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8ad5118000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8ad5117000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f8ad5116000
arch_prctl(ARCH_SET_FS, 0x7f8ad5117700) = 0
mprotect(0x7f8ad4ef9000, 16384, PROT_READ) = 0
mprotect(0x7f8ad5121000, 4096, PROT_READ) = 0
munmap(0x7f8ad5119000, 27312)           = 0
stat("/home/xx/splunk/var/lib/splunk", {st_mode=S_IFDIR|0711, st_size=258, ...}) = 0
umask(0777)                             = 022
umask(066)                              = 0777
gettimeofday({1290061806, 661323}, NULL) = 0
getpid()                                = 15124
open("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
statfs("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", {f_type=0x9123683e,     f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
statfs("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
statfs("/home/xx/splunk/var/lib/splunk", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
write(2, "Could not create a lock in the S"..., 52Could not create a lock in the SPLUNK_DB directory.) = 52
statfs("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
statfs("/home/xx/splunk/var/lib/splunk/test.dHc3Nt", {f_type=0x9123683e, f_bsize=4096, f_blocks=2228224, f_bfree=1430113, f_bavail=1128033, f_files=0, f_ffree=0, f_fsid={196237592, 245698777}, f_namelen=255, f_frsize=4096}) = 0
write(2, "Filesystem type is not supported"..., 201Filesystem type is not supported: buf.f_type = 0x9123683e
If supporting this filesystem type is important to you, please file an Enhancement Request with Splunk Support with the fs info number listed.) = 201
exit_group(9)    
Tags (1)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

Given the change of state, I unaccepted my prior answer (that was years old) and marked ogdin's as correct.

0 Karma

samtheuser
New Member

Splunk v6.2 released. Still aborts with unhelpful message when running on top of btrfs.

0 Karma

phoenixdigital
Builder

Not sure why a workaround solution to this was removed from this thread but I am re-adding it again.

As of Splunk 6.1.3 (build 220630) there is still no support for BTRFS.

A workaround which I recommend for non production environments is to create a blockfile formatted to ext4 and mount this at /opt/splunk/var.

A 10G file can be created with the following on Fedora 20 (likely the same for most Linux flavours)

dd if=/dev/zero of=/opt/splunk-var-blockfile bs=1M count=10240
mkfs.ext4 /opt/splunk-var-blockfile 
mount /opt/splunk-var-blockfile /opt/splunk/var

In order for the file to be mounted on reboot edit /etc/fstab and add the following line

/opt/splunk-var-blockfile /opt/splunk/var/        auto    loop    0 0
0 Karma

elvisior
Explorer

xfs is the default filesystem in RHEL 7, not btrfs so splunk will work fine on it

0 Karma

matthieu_araman
Communicator

thanks for the answer, that's a good news.
sorry my mistake, I was effectively looking at performance and support for xfs

0 Karma

matthieu_araman
Communicator

any news about btrfs support with splunk 6.1 or why it's not supported at this time ?
Btrfs is the default file system for the just released "RedHat Enterprise 7" and io oriented benchmark tests seems to indicate it could be a good choice for filesystem with lots of data such as splunk.

0 Karma

phoenixdigital
Builder

I just encountered this issue too as I was about to go on a work trip. I installed Splunk on my Fedora 18 laptop and lo and behold it doesnt work.

Still no solution for this after 3 years?

0 Karma

dwaddle
SplunkTrust
SplunkTrust

I don't know precisely why Splunk threw an unsupported message about your btrfs - but it is usually very picky about the underlying filesystem's locking semantics. What does the Splunk 'locktest' command say about your btrfs?

phoenixdigital
Builder

Thanks for this it appears to work and is acceptable for a development environment but I would have hoped that Splunk could support btrfs at least in version 6.

It is now the default filesystem for many Linux distributions.

0 Karma

phoenixdigital
Builder

And no btrfs support in 6.0?

Can someone please give us an update of when it might be available?

0 Karma

pygo
New Member

July 2013 and splunk 5.0.3 - still no btrfs support.

0 Karma

elvisior
Explorer

On fedora 18 I've hit this problem as well, my workaround (that doesn't work see additional comment below) might be helpful to some people.

I just replace the locktest (/opt/splunk/bin/locktest) with a bash script which does an 'exit 0'.

Edit: Ok, this worked for a little while and then resulted in the indexer blocking and lots of weird things happening.

0 Karma

glitchcowboy
Path Finder

Perfect, thanks! I'm sure running ext3 on top of btrfs isn't terribly quick, but on my laptop it'll fly. Where'd I put that imaginary SSD now???

0 Karma

tprzelom
Path Finder

Jan 2013, need brtfs support

0 Karma

Reboli_NL
New Member

December 2012 and Splunk 5.0.1 - still no btrfs support.

0 Karma

srickaue
New Member

it's now April 2012 and Splunk 4.3.1 - still no official btrfs support ;(

0 Karma

sn
Engager

Hi Henry/Archie(?),

first create a file as big as you think you will need for splunk data

dd if=/dev/zero of=/mydata/splunkfs.img bs=1M count=5000

Then format it with ext4 (sans journal, which is kind of moot in this case):

mkfs.ext4 -O ^has_journal /mydata/splunkfs.img

Add a line to /etc/fstab like this:

/mydata/splunkfs.img  /opt/splunk/var/        auto    loop    0 0

Stop splunk:

/opt/splunk/bin/splunk stop

Then just something like

cd /opt/splunk

mkdir vartmp

mv var/* vartmp

mount var

mv vartmp/* var

/opt/splunk/bin/splunk start

henryvalk
Engager

"As a workaround I've loopback-mounted a file formatted with ext4 to /opt/splunk/var and copied the contents of the original directory there. Seems to work just fine so far."

Please provide a "how to" for this workaround as I am new to linux and dont completely follow how this is achieved. Any help appreciated. I created a loopback file system but I am confused about recreating the var directory as an ext4 filesystem since this did not work when following this procedure..http://www.walkernews.net/2007/07/01/create-linux-loopback-file-system-on-disk-file/#comment-15959

Regards, Henry

sn
Engager

Since btrfs is planned to become the default in Fedora 17 and others will surely follow, I think splunk should start to support it as soon as possible.
As a workaround I've loopback-mounted a file formatted with ext4 to /opt/splunk/var and copied the contents of the original directory there. Seems to work just fine so far.

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!