Cluster Master restarts kills s3-compliant on-prem SmartStore since upgrading to Splunk 9

[javiergn]

Hi all. I’ve got an interesting case:

• $Customer using on-prem fully s3-compliant storage: DELL ECS
• When restarting the Cluster Master (and only the CM, not the Indexers) that triggers thousands of timeout events from S3.
• Is like the S3 cluster is having a denial-of-service attack
• This is ONLY happening since we upgraded from 8.2.6 to Splunk 9.0.4 in April 29th. No issues before.
• See screenshot below.

What exactly are the Indexers requesting from s3 when the CM is restarted?

How is this process different in Splunk 8.2.6 and Splunk 9.0.4?

Regards,

J