Archive

Splunk on a clustered file system

Splunk Employee
Splunk Employee

What types of clustered file systems does Splunk support?

Or, more importantly, are there any types that Splunk does NOT support?

1 Solution

SplunkTrust
SplunkTrust

I assume you mean supported for the purpose of storing buckets.

According to the official docs, with the possible exception of Veritas VXFS no cluster filesystem is documented as supported. (The docs are not clear if VXFS is supported in cluster mode or not).

For any cluster filesystem to work, it would obviously have to pass locktest. Most claim to support "POSIX" locking semantics, so they could have a good chance of success here.

Even if supported, however, I would be concerned about performance due to the need for cluster filesystems to frequently contact a metadata controller when certain filesystem operations occur.

View solution in original post

Engager

The only 2 POSIX file-locking clustered file systems worth considering are GFS2 and OCFS2. But be aware you will probably need to dedicate one core for locking overhead, since all it does is eat and serve data, as it is still a very CPU intensive task. Probably a better idea to use a script with keepalived on the secondary node pointing at the primary node, to fail over to an identically configured Splunk instance on the secondary node. Wont get load balancing between two active pair, but not eat a core just to do it in a cluster either.

SplunkTrust
SplunkTrust

I assume you mean supported for the purpose of storing buckets.

According to the official docs, with the possible exception of Veritas VXFS no cluster filesystem is documented as supported. (The docs are not clear if VXFS is supported in cluster mode or not).

For any cluster filesystem to work, it would obviously have to pass locktest. Most claim to support "POSIX" locking semantics, so they could have a good chance of success here.

Even if supported, however, I would be concerned about performance due to the need for cluster filesystems to frequently contact a metadata controller when certain filesystem operations occur.

View solution in original post

Splunk Employee
Splunk Employee

VxFS is has been in production use by some customers (I wasn't involved, they may still be.)

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!