Smartstore data migration : do we need to rebuild reduced buckets before migrating to object storage?

[manduki491]

hello Splunkers ,

Need some clarification on Smartstore data migration. as per the docs , You can still search any existing buckets that were tsidx-reduced before migration to SmartStore.  e.g. we have 18 months of data retention. We need to keep 6 months of data in local/cache storage due to frequent audit/forensic searches that need raw data fields.

questions:

1> is it possible to  migrate tsidx reduced buckets to obj store without need for rebuild & Indexer cluster  will still search them as normal ( slower) process for tsidx reduced buckets ? OR do we need to rebuild all buckets before initiating data migration to obj store. in our case then we need to rebuild all the buckets from 7 to 18 months! In some cases we may run out of local space if we have to do this 😐

2> What is the performance impact to search reduced bucket with addition of smartstore. Since cache manager has to fetch bucket from remote store & then rebuild it locally in cache(?) before it being searchable , the two levels of performance hit is too much? Anyone have had such a situation.

thanks for your attention 🙂

Manduki

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/AboutSmartStore

Tsidx reduction. Do not set enableTsidxReduction to "true". Tsidx reduction modifies bucket contents and is not supported by SmartStore. Note: You can still search any existing buckets that were tsidx-reduced before migration to SmartStore. As with non-SmartStore deployments such searches will likely run slowly.

[PickleRick]

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/AboutSmartStore#Features_not_supported_b...

Tsidx reduction. Do not set enableTsidxReduction to "true". Tsidx reduction modifies bucket contents and is not supported by SmartStore. Note: You can still search any existing buckets that were tsidx-reduced before migration to SmartStore. As with non-SmartStore deployments such searches will likely run slowly. See Reduce tsidx disk usage.

[manduki491]

ah okay , thanks for the confirmation. So what they meant is  for Indexes that are still on the local non S3 storage & not those indexes converted to smartstore i.e. moved to obj store. 

"You can still search any existing buckets that were tsidx-reduced before migration to SmartStore."

Which means all the reduced buckets will need rebuilt to full before updating indexes.conf config to move buckets to Smartstore.

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Reducetsidxdiskusage#Restore_reduced_buc...

[PickleRick · Accepted solution]

As I understand it (haven't tested though) - if you enable smartstore while having reduced buckets, cachemgr will happily upload them to smartstore and will keep them in reduced state fetching them to the cache whenever they are needed but not rebuilding indexes so searching will be slow.

But if you have non-reduced buckets in smartstore you can't reduce them anymore because you can't modify contents of the remote storage. You only fetch a copy to cache when needed.

[manduki491]

thanks a lot! definitely something to check in non production environment then. will post back with updates 🙂

[manduki491]

Verified in production , Smartstore will support upload and searching of reduced buckets. You need to turn off tsidx reduction for that index before enabling Smartstore config on it of course.