Getting Data In

Has anyone tried using fusemount for frozen storage?

AGLbwa
Path Finder

So I'm about to try using Azure Blob Storage fuse-mounted (using blobfuse) as frozen storage, I'm wondering if anyone else has tried this (even with S3) and what the results were? I mean unless the semantics are horribly broken it should work, but the devil (as always) is going to be in the details. I'm doing this (initially) with one indexer in the cluster and will report back if no-one else has preceded me down this path into madness!

Fingers crossed and see you (hopefully) on the other side!

B-)

1 Solution

AGLbwa
Path Finder

Update: had that one node running for a fortnight with /opt/frozen fusemounted to Azure Blob Storage (using blobfuse) with no errors. Not all filesystem semantics are supported (timestamps can be hinky and du returns BS), but it's good enough for frozen (and to prove it, yes I did thaw some randomly selected data (on a different platform) and yes, I could search it). I've cut across other nodes in the cluster and am almost finished.

Hope this helps someone else deciding whether or not to tread the path to madness and eventual despair! (Would recommend!)

B-)

View solution in original post

AGLbwa
Path Finder

Final update and one massive caveat that I haven't had a chance to fully investigate. This setup works brilliantly except if you have a DNS failure. We had a failure of the primary DNS server and this meant that name resolution on the system was a crapshoot (possibly due to shitstemd name resolution) - this made the fusemounts unusable (need to raise with MS - filesystem operations DO NOT timeout), and exposed a bug in Splunk, (Splunk relies on the underlying filesystem to timeout, and if it doesn't neither will Splunk), which meant Splunk would hang coming up as it attempted to access frozen storage (but there were no logs to indicate this).

AGLbwa
Path Finder

Update: had that one node running for a fortnight with /opt/frozen fusemounted to Azure Blob Storage (using blobfuse) with no errors. Not all filesystem semantics are supported (timestamps can be hinky and du returns BS), but it's good enough for frozen (and to prove it, yes I did thaw some randomly selected data (on a different platform) and yes, I could search it). I've cut across other nodes in the cluster and am almost finished.

Hope this helps someone else deciding whether or not to tread the path to madness and eventual despair! (Would recommend!)

B-)

Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...