What is the best way to preservate data apart from indexing it?

[3DGjos]

Hello, I have to preservate all the raw events that are stored in a 3 hour window of time and I need to be sure that data can be searcheable independently from the index my data is originally stored.

what would be the best way to achieve this? I mean, I can just store the data in a saved report. But my doubt is how to search that data as if I was looking in the original index. I mean I will need to investigate and make queries over the data in the future.

[richgalloway]

Can you explain your requirements a little more?  I'm trying to reconcile "searcheable independently from the index" and "search that data as if I was looking in the original index".

See if this helps at all: https://community.splunk.com/t5/Archive/How-to-export-real-raw-events-from-Splunk/td-p/450147

[3DGjos]

Hello, thanks for your answer.

I have an index with lots of activity, that index does not have that much retention period. And it wont have.

So, I need a way to keep all events that were ingested in a certain period of time. I can export the events in raw or csv format, but that way they wont be "searchable" as I normally would do in splunk, I mean the file will be outside splunk. I can make another index and | collect the events to that new index, but im trying to avoid doing that.

what I need is to store the events, inside splunk and the events must be SPL-quereable. If I store them in a report, I don't know how to query those events from the report.

[richgalloway]

The only way to keep data SPL-searchable is to have them in an index.  Keeping them in an index means having the storage and retention, which you say you won't have.  You appear to be stuck between a rock and a hard place.

One possibility is to have this index preserve frozen data (via coldToFrozenScript or coldToFrozenDir) on a storage location apart from the indexes.  Perhaps S3.  Then you could stand up a standalone Splunk instance and copy those S3 buckets to the thawed directory.  All of your archived data is now searchable on the separate instance.