Recommended filesystem for hot/warm buckets

[stwong]

Hi,

We're setting up Splunk 6.4 on RedHat Enterprise Linux 7.
We intend to use ext4 but there is no native snapshot support (we learnt that for hot buckets, backup on snapshot is required). Then we're testing btrfs. However, we got system crash when doing some bonnie++ test on btrfs filesystem on SSD.

Would like to know:
- any recommended filesystem for hot/warm bucket
- can we take the random seek result of bonnie++ as IOPS value (read some post about this topic and would like to know if there is any update on this topic).

Thanks a lot.
Rgds

[jrodman]

For what it's worth, a lot of people I trust are now quite bullish on ext4 with lvm2 snapshots as a completely legit solution. I don't have direct experience.

[esix_splunk]

Here is a list of supported file systems for the latest Splunk Version:

http://docs.splunk.com/Documentation/Splunk/6.4.2/Installation/Systemrequirements#Supported_file_sys...

If you're looking at snapshot capability, ZFS might be the better choice. It's pretty robust and I've encountered deployments on ZFS doing snapshots to fulfill back requirements. (Although backing up of buckets in a clustered environment is questionable.....)

[jkat54]

Yes, random seek in bonnie++ is the iops. You might find this varies greatly between bonnie++ versions however, and thus I recomend fio for these tests instead. I know Splunk prefers bonnie++ however, so if you're getting reliable results, please continue to use bonnie++ at your leisure.

As for filesystems, you might get next to compile on RHEL:
http://next3.sourceforge.net/

[stwong]

Will give next a try. Thanks.