Getting Data In

Has anyone tried using fusemount for frozen storage?

AGLbwa
Path Finder

So I'm about to try using Azure Blob Storage fuse-mounted (using blobfuse) as frozen storage, I'm wondering if anyone else has tried this (even with S3) and what the results were? I mean unless the semantics are horribly broken it should work, but the devil (as always) is going to be in the details. I'm doing this (initially) with one indexer in the cluster and will report back if no-one else has preceded me down this path into madness!

Fingers crossed and see you (hopefully) on the other side!

B-)

1 Solution

AGLbwa
Path Finder

Update: had that one node running for a fortnight with /opt/frozen fusemounted to Azure Blob Storage (using blobfuse) with no errors. Not all filesystem semantics are supported (timestamps can be hinky and du returns BS), but it's good enough for frozen (and to prove it, yes I did thaw some randomly selected data (on a different platform) and yes, I could search it). I've cut across other nodes in the cluster and am almost finished.

Hope this helps someone else deciding whether or not to tread the path to madness and eventual despair! (Would recommend!)

B-)

View solution in original post

AGLbwa
Path Finder

Final update and one massive caveat that I haven't had a chance to fully investigate. This setup works brilliantly except if you have a DNS failure. We had a failure of the primary DNS server and this meant that name resolution on the system was a crapshoot (possibly due to shitstemd name resolution) - this made the fusemounts unusable (need to raise with MS - filesystem operations DO NOT timeout), and exposed a bug in Splunk, (Splunk relies on the underlying filesystem to timeout, and if it doesn't neither will Splunk), which meant Splunk would hang coming up as it attempted to access frozen storage (but there were no logs to indicate this).

AGLbwa
Path Finder

Update: had that one node running for a fortnight with /opt/frozen fusemounted to Azure Blob Storage (using blobfuse) with no errors. Not all filesystem semantics are supported (timestamps can be hinky and du returns BS), but it's good enough for frozen (and to prove it, yes I did thaw some randomly selected data (on a different platform) and yes, I could search it). I've cut across other nodes in the cluster and am almost finished.

Hope this helps someone else deciding whether or not to tread the path to madness and eventual despair! (Would recommend!)

B-)

Get Updates on the Splunk Community!

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...