quincy: rgw: make incomplete multipart upload part of bucket check efficient #57405
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously the incomplete multipart portion of bucket check would list all entries in the _multipart_ namespace across all shards and then analyze them in memory before taking further action. Since all index entries for a given multipart upload are all on the same shard by design, we can work on this asynchronously shard by shard. Furthermore since all entries for a given multipart upload are sequential in the bucket index, we can use a small window to analyze each of the uploads. This should make the operation quicker and use much less memory in the worst cases. Signed-off-by: J. Eric Ivancich <ivancich@redhat.com> (cherry picked from commit ebe9893) Conflicts: src/rgw/driver/rados/rgw_bucket.{cc,h} - located in src/rgw - small adjustments due to zipper progress
…m bi As one of the steps in `radosgw-admin bucket check --fix ...` it looks for bucket index entries for incomplete multipart uploads that do not have a corresponding ".meta" entry in the same bucket index. It then intends to delete those entries, however the function that it calls to perform the bucket index deletions was flawed and did not direct the removals to the appropriate shard(s), but instead a non-existant oid. This commit determines the appropriate shard for each of the entries to be removed and asynchronously issues a librados call to omap_rm_keys. Signed-off-by: J. Eric Ivancich <ivancich@redhat.com> (cherry picked from commit 0521c2a) Conflicts: src/rgw/driver/rados/rgw_rados.cc - remove bitx debugging
ivancich
added
bug-fix
needs-qa
performance
wip-eric-testing-1
|
After multiple tries I was unable to get a clean build on shaman, so I'm going to drop my QA attempt and let this be done through the standard process. |
thanks @ivancich. it looks like the centos 9 builds are failing because they're running some new ceph-container logic that they shouldn't be |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
backport tracker: https://tracker.ceph.com/issues/65891
backport of #57083
parent tracker: https://tracker.ceph.com/issues/65769
this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/master/src/script/ceph-backport.sh