topic What are the best long-term storage options to store data from Splunk Indexers? in Deployment Architecture

What are the best long-term storage options to store data from Splunk Indexers?

Taifa — Mon, 13 Jun 2022 20:26:06 GMT

Hello,

I am interested in knowing the best storage option among these three (DAS, NAS or SAN) when you want to store the data from indexers for the long term.

Thanks,

Re: Splunk Indexers and long term data storage

gcusello — Mon, 13 Jun 2022 13:46:51 GMT

Hi @Taifa,

when you say "Long term" are you meaning data that are queried very rarely I suppose.

If this is your meaning, the only rule is to choose the less expensive because using one kind of storage or another is relevant only for performances, but for long term data, usually performances aren't relevant.

If you have to archive them only for compliance, you could also archive them off line (as Thawed data) and connect them to Splunk only if needed.

Ciao.

Giuseppe

Re: Splunk Indexers and long term data storage

Stefanie — Mon, 13 Jun 2022 13:48:09 GMT

Which technology are you more familiar with?
SAN and NAS might be more on the expensive side. But you'd get better performance and scalability.

I'd say DAS might be the better solution, but that's my opinion. It's cheaper than the other two. However it does lack some scalability that the other two have.

Re: Splunk Indexers and long term data storage

Taifa — Mon, 13 Jun 2022 14:38:11 GMT

Thanks gcusello,

by long term, the intention is to use the them as the indexers for daily searches while maintaining/keeping the logs for 3years minimum

Re: Splunk Indexers and long term data storage

Taifa — Mon, 13 Jun 2022 14:39:43 GMT

Thanks Stefanie.

Re: Splunk Indexers and long term data storage

gcusello — Mon, 13 Jun 2022 14:41:47 GMT

Hi @Taifa,

as I said, if you use data older that some months but you want to maintain them on line searchable, you have to set the retention of your data and use as Cold Folder one store less expensive because you don't use it every day.

I hint to analyze, if the choice of having off line data (restorable in few minutes) could be acceptable for you.

Ciao.

Giuseppe