Splunk Search

Is it possible to mount HOT to a ram disk for performance?

daniel333
Builder

All,

Just day dreaming here a little as I read the indexes.conf file documentation a bit. I was thinking, assuming you're willing to risk the data loss wouldn't it make sense to house your Hot buckets to a RAM disk, then roll from HOT to warm to actual disk?

Crazy?

https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

HOT and WARM are on the same filesystem; you cannot separate that. The only real difference between HOT and WARM buckets is that HOT buckets are open for R/W whereas WARM buckets are read-only.
Splunk really doesn't care how you implement the HOT/WARM storage as long as it is exposed as a supported filesystem.
You will very definitely want to very carefully assess your failure scenarios with RAM disk.

jkat54
SplunkTrust
SplunkTrust

I concur with the above.

Also consider the limitation of maybe at most 512gb available...

Would be great for a very specific use case though!

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

An unscheduled reboot will loose all the data from a RAMdisk. and since some hot buckets stay around a long time, you would loose all that data. You'd also loose the warm buckets, too, since they have to reside on the same FS.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

True.

You will very definitely want to very carefully assess your failure scenarios with RAM disk.

In other words: Don't do this without Splunk index replication. Spread your indexers across multiple racks/switches/PDUs/etc.
Even after doing anything you can think of, accept the remaining risk of losing data due to catastrophic failures.

gjanders
SplunkTrust
SplunkTrust

Also consider what will happen when a server requires a reboot for maintenance purposes.

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...