Experiences rolling frozen storage into HSM (Hierarchal Storage Management)

[dwaddle]

Does anyone have experience integrating splunk with a hierarchal storage management system (like AMASS, Legato, or Tivoli Storage Manager), specifically with respect to frozen data?

The general idea behind all of these products is the data is 'transparently' migrated to tape/optical and leave behind an inode. There's a piece of kernel extension that picks up on open() / read() against the dummy inode and pulls that data back onto local storage.

I can't see any concrete reason why Splunk would not support such an arrangement for frozen index data, but wanted to see if anyone had experience in how it might work in practice.

[gkanapathy · Accepted solution]

No reason it wouldn't work, assuming it can work on an entire directory at a time. Just understand that while you could script Splunk to roll data out of cold into frozen and then into HSM, Splunk will never on its own read back the inode. Once it's frozen, Splunk forgets everything about it and it will be up to you (or whoever) to bring the data back to someplace where Splunk will look at it (e.g., the thawed folder)