topic Re: Is running Splunk indexes over iSCSI a good idea? in Knowledge Management

Is running Splunk indexes over iSCSI a good idea?

Glenn — Mon, 24 Oct 2011 16:34:29 GMT

Does anyone have any experience or opinions about running Splunk with its indexes running over iSCSI? Is iSCSI compatible with Splunk at all?

We are talking about 400gb/day across four indexers (which replicate to four standby indexers). We have a 10GBit Cisco Nexus network available, which should allow for quite low latency access. Our iSCSI volumes would be provided by a NetApp filer.

Even if performance would not be good enough for the hot or warm buckets, would it be reasonable to store historical data (cold buckets) there?

Re: Is running Splunk indexes over iSCSI a good idea?

kddenton — Mon, 24 Oct 2011 16:57:46 GMT

iSCSI is more operating system compatibility then Splunk. Its really just another drive.

As Splunk goes its all about IOPS.

Here is another post that I found on this.

link:(http://www.splunk.com/support/forum:SplunkAdministration/3206"iSCSI")

Re: Is running Splunk indexes over iSCSI a good idea?

thartmann — Mon, 24 Oct 2011 17:07:57 GMT

We've been running with out colddb's on iSCSI for a while now and it's been just fine. I think kddenton is right on, it's all about the IOPS.

Re: Is running Splunk indexes over iSCSI a good idea?

dwaddle — Mon, 24 Oct 2011 17:12:27 GMT

An adequately robust iSCSI setup should be fine. Generally speaking, Splunk should be unaware. With the physical spindles being provided off of a NetApp, iSCSI would be preferred to NFS. (I won't say that iSCSI is better or worse than FCP, however.)

I would recommend that you make sure that your iSCSI environment has good multipathing and failover capability. Hardware iSCSI initiators / NICs would obviously be preferable. Let multipathd provide path load-balancing and failover.

NetApp typically provides fairly good IOPS, which is Splunk's primary concern. You'll want to make sure that your NetApp storage is provisioned in such a way that all of the indexers have access to sufficient (and preferably dedicated) spindles to keep up your IOPS requirements. (Putting volumes for all 4 indexes into the same aggregate would probably not perform very well under a load)

Re: Is running Splunk indexes over iSCSI a good idea?

auspexian — Wed, 05 Sep 2012 03:22:32 GMT

We have been using Netapp SAS drives. But we just bought a couple of Netapp with SATA drives. We were told we can get only about 400IOPS from netapp with sata. Anyone successfully using splunk with ISCSI on Netapp with SATA drives?

Tks

John

Re: Is running Splunk indexes over iSCSI a good idea?

dwaddle — Wed, 05 Sep 2012 18:09:46 GMT

I'd recommend pulling this into a totally new question. You will probably want to get details on the SATA drives you're using - Model, RPM, etc - as well as how many of them you're using in your aggregate. 400 IOPS may be "good" for the configuration you have...