Splunk Search

Is it possible to mount HOT to a ram disk for performance?

daniel333
Builder

All,

Just day dreaming here a little as I read the indexes.conf file documentation a bit. I was thinking, assuming you're willing to risk the data loss wouldn't it make sense to house your Hot buckets to a RAM disk, then roll from HOT to warm to actual disk?

Crazy?

https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

HOT and WARM are on the same filesystem; you cannot separate that. The only real difference between HOT and WARM buckets is that HOT buckets are open for R/W whereas WARM buckets are read-only.
Splunk really doesn't care how you implement the HOT/WARM storage as long as it is exposed as a supported filesystem.
You will very definitely want to very carefully assess your failure scenarios with RAM disk.

jkat54
SplunkTrust
SplunkTrust

I concur with the above.

Also consider the limitation of maybe at most 512gb available...

Would be great for a very specific use case though!

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

An unscheduled reboot will loose all the data from a RAMdisk. and since some hot buckets stay around a long time, you would loose all that data. You'd also loose the warm buckets, too, since they have to reside on the same FS.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

True.

You will very definitely want to very carefully assess your failure scenarios with RAM disk.

In other words: Don't do this without Splunk index replication. Spread your indexers across multiple racks/switches/PDUs/etc.
Even after doing anything you can think of, accept the remaining risk of losing data due to catastrophic failures.

gjanders
SplunkTrust
SplunkTrust

Also consider what will happen when a server requires a reboot for maintenance purposes.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...