KAFKA-10199: Shutdown with new remove operation in state updater #15894
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lucasbru
reviewed
May 8, 2024
| clearInputQueue(); | ||
| updatingTasks.clear(); | ||
| pausedTasks.clear(); | ||
| changelogReader.clear(); |
There was a problem hiding this comment.
why was this added? Where was it executed before?
There was a problem hiding this comment.
Before, it was executed in removeUpdatingAndPausedTasks().
There was a problem hiding this comment.
I refactored this code a bit.
| public void start() { | ||
| if (stateUpdaterThread == null) { | ||
| if (!restoredActiveTasks.isEmpty() || !exceptionsAndFailedTasks.isEmpty()) { |
There was a problem hiding this comment.
I thought that since we allow to restart the state updater, we should verify that the state updater starts with clean queue to avoid invalid states coming from a past run of the state updater. At the moment, we never restart the state updater but I thought having clear invariants might be good.
There was a problem hiding this comment.
I wonder why we'd even allow restarting the state updater. but we don't need to fix it now
| @@ -954,8 +954,7 @@ private void recycleTaskFromStateUpdater(final Task task, | |||
| } | |||
| } | |||
|
|
|||
| /** Returns true if the task closed clean */ | |||
| private boolean closeTaskClean(final Task task, | |||
| private void closeTaskClean(final Task task, | |||
| final Set<Task> tasksToCloseDirty, | |||
cadonna
force-pushed
the
stateupdater_shutdown_with_new_remove
branch
from
b306901
to
80d2983
Compare
Uses the new remove operation of the state updater that returns a future to shutdown the task manager.
cadonna
force-pushed
the
stateupdater_shutdown_with_new_remove
branch
from
80d2983
to
e732de3
Compare
The removed verification is based on a wrong assumption about when standby tasks are promoted. This is related to KIP-988.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uses the new remove operation of the state updater that returns
a future to shutdown the task manager.
Committer Checklist (excluded from commit message)