We run a fairly good chunk of our installation on RAIDZ-2's - 288 TB raw capacity across 12 indexers so far, which will be doubled soon. Not one issue so far except for discovering that option, which was needlessly difficult.
As far as performance goes, since I had 12 identical machines, I did a bunch of benchmarks, comparing 3x8 hardware RAID arrays (mirror striped across all three as well as just a linear array), various md(4) and LVM-setups, as well as ZFS mirror and various RAIDZ-setups. All drives are Crucial 1 TB SSDs (M500 I think - don't think we got the M600 here). In general, md(4) sucked massively. LVM fared a bit better, but was still blown out of the water by ZFS. Hardware RAID only surpassed on reads, but fell behind massively on writes, pretty much across the board. The only place ZFS really "failed" was a for-the-hell-of-it 24-drive RAIDZ3, which had the worst performance of all 🙂
In the end, we went with 3 x 8-wide RAIDZ2, because that hit a sweet-spot with performance and redundancy.
... View more