I'm moving my Splunk server to a new VM based box and I can either build it as a RHEL5/6 box or a Windows Server 2008 R2 box. My team and I are generally proficient in both, and we have a mixed environment of both. Has anyone looked at or noticed any performance differences across Splunk on those two OSs?
You should be advised that running Splunk in a virtual environment is not recommended. Splunk needs fast access to the hardware and the VM will only add more layers between splunk and the hardware. Having said that, we run Splunk off RHEL and the only issues I've had that have been dependent on the OS have been with the PDF server. Everything else is pretty much a breeze. Other than that RHEL has given us no issues what so ever.
Maybe some one can comment on Windows.
Yeah, we have a small enough deployment that we haven't noticed any issues with running it off VMs so far, but we have some long term plans to spec up to physical boxes when we need it.
I would run it under RH. I have personally found the performance of a nix system on a VM running Splunk to exceed that of a Windows server. Also while it is true that running Splunk on a VM affects performance it isn't necessarily not recommended, as long as you can give it enough cores and the required 800 IOPS then it should operate satisfactorily.
While I would also run on RHEL, there are limitations for both OS's.
On a *nix indexer, you cannot do remote polling of windows machines through WMI. Any logs from windows machines must come through a forwarder, off a network share, or through syslog.
PDF server is not supported on windows, afaik - you'd need a *nix machine to run that.
Wow, the pdf server is not supported on windows? Man its was a pain to get it working on our RHLE search heads... I thought on windows it just might be a matter of pressing next a few times and "I agree" before having it working.