repair: Introduce repair_partition_estimation_ratio config option #18634
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
area/repair
P1
denesb
reviewed
May 13, 2024
db/config.cc
Outdated
| @@ -968,6 +968,8 @@ db::config::config(std::shared_ptr<db::extensions> exts) | |||
| , enable_repair_based_node_ops(this, "enable_repair_based_node_ops", liveness::LiveUpdate, value_status::Used, true, "Set true to use enable repair based node operations instead of streaming based.") | |||
| , allowed_repair_based_node_ops(this, "allowed_repair_based_node_ops", liveness::LiveUpdate, value_status::Used, "replace,removenode,rebuild,bootstrap,decommission", "A comma separated list of node operations which are allowed to enable repair based node operations. The operations can be bootstrap, replace, removenode, decommission and rebuild.") | |||
| , enable_compacting_data_for_streaming_and_repair(this, "enable_compacting_data_for_streaming_and_repair", liveness::LiveUpdate, value_status::Used, true, "Enable the compacting reader, which compacts the data for streaming and repair (load'n'stream included) before sending it to, or synchronizing it with peers. Can reduce the amount of data to be processed by removing dead data, but adds CPU overhead.") | |||
| , repair_partition_estimation_ratio(this, "repair_partition_estimation_ratio", liveness::LiveUpdate, value_status::Used, 0.1, | |||
There was a problem hiding this comment.
we should have the word count in the name, becuase currently it is unclear what quality of a partiiton we estimate here. I suggest: repair_partition_count_estimate_fraction.
There was a problem hiding this comment.
Originally, I had a word number in the name which I dropped. The count looks good. Changed in v2.
In commit 642f9a1 (repair: Improve estimated_partitions to reduce memory usage), a 10% hard coded estimation ratio is used. This patch introduces a new config option to specify the estimation ratio of partitions written by repair out of the total partitions. It is set to 0.1 by default. Fixes scylladb#18615
asias
force-pushed
the
repair_bf_estimation_config
branch
from
025c380
to
340eae0
Compare
denesb
approved these changes
May 13, 2024
🟢 CI State: SUCCESS✅ - Build Build Details:
|
This was referenced May 14, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In commit 642f9a1 (repair: Improve estimated_partitions to reduce memory usage), a 10% hard coded estimation ratio is used.
This patch introduces a new config option to specify the estimation ratio of partitions written by repair out of the total partitions.
It is set to 0.1 by default.
Fixes #18615