In our env, we've had a high value for remote.s3.multipart_upload.part_size to fix a bug present in versions prior to 8.1. When I recently reverted the setting to its default, these logs started popping up from time to time:
WARN S3Client [34628 cachemanagerUploadExecutorWorker-0] - command=multipart-upload command=put transactionId=.... rTxnId=...status=completed success=N uri=... statusCode=500 statusDescription="Internal Server Error" payload="<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>...</RequestId><HostId>...</HostId></Error>"
ERROR S3Client [103868 cachemanagerUploadExecutorWorker-0] - multipart-upload command=end transactionId=... rTxnId=... parts=5 uploadId="..." status=completed success=N payload="<Error><Code>InternalError</Code><Message>We encountered an internal error. Please try again.</Message><RequestId>...</RequestId><HostId>...</HostId></Error>"
I checked, the uploaded file is the same size as the local one. I was also able to confirm the remote and local files have the same CRC, so it appears the upload worked. The file modtime in the S3 console is three seconds before the failed upload error message is recorded. My guess is there was a temporary issue, and the retry succeeded. I can't find any follow-up logs about the bucket id after the error message. Does anyone else observe this, should I just accept these error messages? Is there something I can do about it?
