- Find Answers
- :
- Splunk Administration
- :
- Admin Other
- :
- Knowledge Management
- :
- Is running Splunk indexes over iSCSI a good idea?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Does anyone have any experience or opinions about running Splunk with its indexes running over iSCSI? Is iSCSI compatible with Splunk at all?
We are talking about 400gb/day across four indexers (which replicate to four standby indexers). We have a 10GBit Cisco Nexus network available, which should allow for quite low latency access. Our iSCSI volumes would be provided by a NetApp filer.
Even if performance would not be good enough for the hot or warm buckets, would it be reasonable to store historical data (cold buckets) there?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
An adequately robust iSCSI setup should be fine. Generally speaking, Splunk should be unaware. With the physical spindles being provided off of a NetApp, iSCSI would be preferred to NFS. (I won't say that iSCSI is better or worse than FCP, however.)
I would recommend that you make sure that your iSCSI environment has good multipathing and failover capability. Hardware iSCSI initiators / NICs would obviously be preferable. Let multipathd provide path load-balancing and failover.
NetApp typically provides fairly good IOPS, which is Splunk's primary concern. You'll want to make sure that your NetApp storage is provisioned in such a way that all of the indexers have access to sufficient (and preferably dedicated) spindles to keep up your IOPS requirements. (Putting volumes for all 4 indexes into the same aggregate would probably not perform very well under a load)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We have been using Netapp SAS drives. But we just bought a couple of Netapp with SATA drives. We were told we can get only about 400IOPS from netapp with sata. Anyone successfully using splunk with ISCSI on Netapp with SATA drives?
Tks
John
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'd recommend pulling this into a totally new question. You will probably want to get details on the SATA drives you're using - Model, RPM, etc - as well as how many of them you're using in your aggregate. 400 IOPS may be "good" for the configuration you have...
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
An adequately robust iSCSI setup should be fine. Generally speaking, Splunk should be unaware. With the physical spindles being provided off of a NetApp, iSCSI would be preferred to NFS. (I won't say that iSCSI is better or worse than FCP, however.)
I would recommend that you make sure that your iSCSI environment has good multipathing and failover capability. Hardware iSCSI initiators / NICs would obviously be preferable. Let multipathd provide path load-balancing and failover.
NetApp typically provides fairly good IOPS, which is Splunk's primary concern. You'll want to make sure that your NetApp storage is provisioned in such a way that all of the indexers have access to sufficient (and preferably dedicated) spindles to keep up your IOPS requirements. (Putting volumes for all 4 indexes into the same aggregate would probably not perform very well under a load)
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We've been running with out colddb's on iSCSI for a while now and it's been just fine. I think kddenton is right on, it's all about the IOPS.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
iSCSI is more operating system compatibility then Splunk. Its really just another drive.
As Splunk goes its all about IOPS.
Here is another post that I found on this.
link:(http://www.splunk.com/support/forum:SplunkAdministration/3206"iSCSI")
Stay Connected: Your Guide to February Tech Talks, Office Hours, and Webinars!
Preparing your Splunk Environment for OpenSSL3
Incident Response: Reduce Incident Recurrence with Automated Ticket Creation
