https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702498#M28747 <P>Appreciate your response&nbsp;<a href="https://community.splunk.com/t5/user/viewprofilepage/user-id/263556">@marnall</a>&nbsp;.&nbsp; My questions comes from our recent scenario, where we did a splunk upgrade using Infrastructure as code and we are using smartstore for indexing.&nbsp; We were in the opinion that the data get moved to external storage once it hits the warm bucket but unfortunately, we lost some of the data during the migration.&nbsp; The only reason we could think of is the hot buckets which are stored locally did not get rolled over to warm bucket which could have been available in the external storage and are available for later searches.&nbsp; We have another migration scheduled for this weekend, so I want to be cent percent sure we don't have any data loss.</P> Tue, 22 Oct 2024 13:47:30 GMT Richy_s 2024-10-22T13:47:30Z https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702408#M28745 <P>I am planning on upgrading our Splunk infrastructure which requires our Splunk indexers to go offline for few minutes.&nbsp; &nbsp;I am using smartstore for splunk indexing.&nbsp; Before I start the upgrade and take down our indexers, I want to roll over all the data that is in hot bucket to the smartstore and then start the upgrade.&nbsp; What is the best way to do this ?&nbsp;&nbsp;</P><P>&nbsp;</P><P>&nbsp;</P> Mon, 21 Oct 2024 17:36:27 GMT https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702408#M28745 Richy_s 2024-10-21T17:36:27Z https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702420#M28746 <P>You can gently tell the indexers to go offline using "/opt/splunk/bin/splunk offline" . They will stop indexing, roll hot buckets to warm and upload them to remote storage, then you can bring them up again and they will rejoin the cluster.<BR /><BR />Ref:&nbsp;<A href="https://docs.splunk.com/Documentation/Splunk/9.3.1/Indexer/Takeapeeroffline" target="_blank">https://docs.splunk.com/Documentation/Splunk/9.3.1/Indexer/Takeapeeroffline</A></P> Mon, 21 Oct 2024 19:17:21 GMT https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702420#M28746 marnall 2024-10-21T19:17:21Z https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702498#M28747 <P>Appreciate your response&nbsp;<a href="https://community.splunk.com/t5/user/viewprofilepage/user-id/263556">@marnall</a>&nbsp;.&nbsp; My questions comes from our recent scenario, where we did a splunk upgrade using Infrastructure as code and we are using smartstore for indexing.&nbsp; We were in the opinion that the data get moved to external storage once it hits the warm bucket but unfortunately, we lost some of the data during the migration.&nbsp; The only reason we could think of is the hot buckets which are stored locally did not get rolled over to warm bucket which could have been available in the external storage and are available for later searches.&nbsp; We have another migration scheduled for this weekend, so I want to be cent percent sure we don't have any data loss.</P> Tue, 22 Oct 2024 13:47:30 GMT https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702498#M28747 Richy_s 2024-10-22T13:47:30Z https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702512#M28748 <P>Infrastructure as code? Does that mean you are terminating the indexers rather than shutting them off, upgrading, then turning them on again?</P> Tue, 22 Oct 2024 17:48:16 GMT https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702512#M28748 marnall 2024-10-22T17:48:16Z https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702536#M28749 <P>No. It doesn't work like that. A bucket doesn't "roll to smartstore". A bucket rolls to warm and cache manager uploads it to smartstore when it can. So if you:</P><P>1) Didn't give Splunk a chance to upload the bucket to smartstore and</P><P>2) Didn't have more copies of a bucket (or just destroyed all instances at once)</P><P>yes, you might have experienced data loss.</P><P>&nbsp;</P> Tue, 22 Oct 2024 20:50:06 GMT https://community.splunk.com/t5/Deployment-Architecture/Splunk-indexer-hot-bucket-roll-to-smartstore/m-p/702536#M28749 PickleRick 2024-10-22T20:50:06Z