feat: replay all regions in a batched manner #3808
Draft
+417
−66
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
When using Kafka WAL, the read amplification of region replay can become exceedingly high as multiple regions are assigned to a topic, requiring numerous WAL entries to be pulled repeatedly. This PR aims to remove the read amplification by performing region replay in a batched manner.
Specifically, each RegionOpenRequest now includes an optional WalReader field. This field serves as a mutable reference to the WalReader instance. If the WalReader is Some, region replay is bypassed, and essential information for replaying the region is stored within the WalReader.
Once the region server completes handling all RegionOpenRequests, the WalReader contains the necessary information for replaying all regions. This information enables replaying all regions in a batched manner without introducing any read amplification.
On the other hand, to support replaying regions by topic rather than by region, we have added the group_by_namespace interface to the LogStore. This interface will perform a group by operation on the input namespaces according to certain rules. For Kafka WAL, this interface will group namespaces by topic.
Once we obtain the streamlined namespaces, we then fetch logs from multiple namespaces, aka. topics, in parallel. We parse data from each log, then insert it into a RegionPutRequest, which is then processed by the region server. Considering that replayed data should not be written to the WAL again, we have added an Option to the RegionPutRequest. If it is Some, it indicates that this is replayed data. If it is None, it indicates that this is newly written data.
This PR is still in its draft version. I will add necessary tests and refine the codes.
Checklist