topic Re: Can I put splunk installation on NAS? in Installation

Can I put splunk installation on NAS?

melonman — Fri, 20 Apr 2012 02:14:47 GMT

Hi,

I was wondering if I can put splunk installation (SPLUNK_HOME) on NAS?
Is it supported?

Thank you,

Re: Can I put splunk installation on NAS?

Damien_Dallimor — Fri, 20 Apr 2012 02:25:33 GMT

You can safely use NAS storage ie:NFS, for Search Head Pooling and Mounted Bundles, no problem.

But you would not want to run your Splunk Indexers on NAS storage because the IOPS latency would be detrimental to performance.

Considerations regarding NFS

Alternatively to local disk , I've used SAN block based storage over a fibre channel and achieved acceptable IOPS on my Splunk Indexers.

Re: Can I put splunk installation on NAS?

melonman — Fri, 20 Apr 2012 02:54:49 GMT

I can see the part of config files and indexes can be in NAS or NFS, but splunk installation itself is OK to put on NAS/NFS and poot from mounted NAS/NFS?
e.g. /mount/nfs1/splunk/bin/splunk start

Re: Can I put splunk installation on NAS?

Damien_Dallimor — Fri, 20 Apr 2012 04:44:57 GMT

In a distributed architecture it is fine to use an NFS volume as your shared storage for Search Head Pooling and Search Peer Bundle Mounting (etc/users , etc/apps , etc/system directorys )

I would not however install your Splunk application instance(either Standalone Splunk, Search Head, Search Peer etc..) on an NFS volume for the performance reasons alluded to in my first reply.

Re: Can I put splunk installation on NAS?

MasterOogway — Thu, 09 Aug 2012 14:52:54 GMT

Taking a view from the other side, I would say yes. You absolutely can install Splunk on NAS and have your Indexers Search heads reside there. The only caution I have is to make sure your storage team has a very "tuned" environment and your servers mounting NFS with the correct options.

Case in point. I have just completed an upgrade of (4) separate Splunk environments running SuSE LINUX and all the storage requirements were met by using NetApps Filers, to a consolidated shared environment for 1) Platform; and a second Shared environment for 2) Applications. Again, all based on NetApp 6210 Filers.

I confirmed any concern of not handling throughput to the filer by running Bonnie++ testing against the NFS mounts. My results came back like this:

Sequential Output: 86.6MB/sec

Sequential Input (random): 104.2MB/sec

When compared to the measured indexing rate I see on a single Indexing server I believe the NAS NFS can stand on its own. This server by itself sees in excess of 15-20GB/day of data with the following rates:

Events/Sec: 1350

Indexing rate(KBps): 370

The server in question is setup with the following:

(1) 10GB interface; 48GB Memory; (24) 1600MHz CPU's

NFS mount options: nfs rw,bg,hard,intr,rsize=32768,wsize=32768,timeo=60,nfsvers=3 0 0

With this knowledge and the understanding of our future growth rates (which are large), I won't hesitate to continue using Tier I & Tier II NAS as the backend for both hotdb/warmdb (Tier I NAS), and colddb (Tier II NAS). But let me reiterate, don't walk into this blind. Make sure your NAS can handle the load Splunk will put on it when indexing larger amounts of data. Do some extensive testing with Bonnie++ and compare it against what you can get against local or SAN disk.

Good luck.

Re: Can I put splunk installation on NAS?

melonman — Thu, 09 Aug 2012 14:55:13 GMT

Thanks, this is also great information!

Re: Can I put splunk installation on NAS?

tfussell — Wed, 14 Jun 2017 08:38:44 GMT

Thanks MasterOogway for the detailed explanation. I was hoping you could share even more details on the NetApp configurations. for example (although maybe I'm missing something else)
-What are the disk/SSDs that you are using (any flexcache or flashpool)?
-With 2 x 10GB network cards using LACP?
-Just a 2-node cDOT cluster or more (4, 6, ...)?

Do you have an statistics on the IOPs that the Netapps are providing?

Any info that you can provide on the configs would be helpful. thanks in advance!