Splunk Search

Is it possible to mount HOT to a ram disk for performance?

daniel333
Builder

All,

Just day dreaming here a little as I read the indexes.conf file documentation a bit. I was thinking, assuming you're willing to risk the data loss wouldn't it make sense to house your Hot buckets to a RAM disk, then roll from HOT to warm to actual disk?

Crazy?

https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

HOT and WARM are on the same filesystem; you cannot separate that. The only real difference between HOT and WARM buckets is that HOT buckets are open for R/W whereas WARM buckets are read-only.
Splunk really doesn't care how you implement the HOT/WARM storage as long as it is exposed as a supported filesystem.
You will very definitely want to very carefully assess your failure scenarios with RAM disk.

jkat54
SplunkTrust
SplunkTrust

I concur with the above.

Also consider the limitation of maybe at most 512gb available...

Would be great for a very specific use case though!

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

An unscheduled reboot will loose all the data from a RAMdisk. and since some hot buckets stay around a long time, you would loose all that data. You'd also loose the warm buckets, too, since they have to reside on the same FS.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

True.

You will very definitely want to very carefully assess your failure scenarios with RAM disk.

In other words: Don't do this without Splunk index replication. Spread your indexers across multiple racks/switches/PDUs/etc.
Even after doing anything you can think of, accept the remaining risk of losing data due to catastrophic failures.

gjanders
SplunkTrust
SplunkTrust

Also consider what will happen when a server requires a reboot for maintenance purposes.

0 Karma
Get Updates on the Splunk Community!

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...