Recommended filesystem for Centos/Redhat

[zscgeek]

What filesystem is recommended for maximum performance on centos/redhat 5.x? (64 bit)

We were thinking either EXT3 or XFS as they are what we have used the most but wanted to get the official splunk recommendation. This will be for our indexing servers that will be doing high volume indexing and searching and storing data for long periods of time.

Overall hardware config will be:

• 2x Quad Core Intel CPU (L5410)
• 16GB Ram
• 16x1TB SATA drives (7200 rpm) in an eSAS chassis.
• LSI RAID 5 or 10 depending on performance

[Jeremiah · Accepted solution]

Raid 10 is supposed to have the best performance, and I've heard that recommended from several sources. I'm not sure the filesystem makes as significant a difference as long as its supported (someone please correct me if I'm wrong.).

http://www.splunk.com/wiki/Community:HardwareTuningFactors

[Lowell]

I use XFS for my primary splunk partition ($SPLUNK_HOME/), and then I use ext3 for the $SPLUNK_HOME/var/run partition (so that all the search jobs and temporary files that splunk creates doesn't cause fragmentation issues with the actual indexing process). (There is one minor bug with outputlookup because of my separate partitions, which has been reported and should be fixed soon.) I've also got all my partitions on top of LVM for easy partition reallocation.

[mcluver]

Is it absolutely necessary to do it this way if we're thinking about going with XFS on an Indexer?

[zscgeek]

My concern with filesystems is that some handle large number of files better then others. Splunk can end up creating insane numbers of files when you get to holding several TB worth of logs in the raw dirs. EXT3 is the OS default of course but I would not mind knowing if people have hit limitations with it.