-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: The query node fails to exit normally during rolling upgrades #32901
Comments
/assign @weiliu1031 |
issue: milvus-io#32901 Cause release segment request need be send to delegator, but it need replica to info find segment's delegator. but the stopping query node will be marked as read only in replica, then `replica.Contains()` just return true for rwNode in replica. then it can't get replica info by stopping query node and release segment will be blocked. This PR make `replica.Contains()` return true for both roNode and rwNode. Signed-off-by: Wei Liu <wei.liu@zilliz.com>
…2929) issue: #32901 Cause release segment request need be send to delegator, but it need replica to info find segment's delegator. but the stopping query node will be marked as read only in replica, then `replica.Contains()` just return true for rwNode in replica. then it can't get replica info by stopping query node and release segment will be blocked. This PR make `replica.Contains()` return true for both roNode and rwNode. Signed-off-by: Wei Liu <wei.liu@zilliz.com>
/assgin @zhuwenxing |
please verify this with latest image |
…lvus-io#32929) issue: milvus-io#32901 Cause release segment request need be send to delegator, but it need replica to info find segment's delegator. but the stopping query node will be marked as read only in replica, then `replica.Contains()` just return true for rwNode in replica. then it can't get replica info by stopping query node and release segment will be blocked. This PR make `replica.Contains()` return true for both roNode and rwNode. Signed-off-by: Wei Liu <wei.liu@zilliz.com>
…lvus-io#32929) issue: milvus-io#32901 Cause release segment request need be send to delegator, but it need replica to info find segment's delegator. but the stopping query node will be marked as read only in replica, then `replica.Contains()` just return true for rwNode in replica. then it can't get replica info by stopping query node and release segment will be blocked. This PR make `replica.Contains()` return true for both roNode and rwNode. Signed-off-by: Wei Liu <wei.liu@zilliz.com>
/assign @zhuwenxing |
verified and fixed in master-20240511-8a9a4219-amd64 |
failed for v2.4.1 -- > master-20240524-36cbce4d-amd64 |
2.3-latest --> 2.4-latest : pass |
…3394) issue: #32901 pr #32814 introduce the compatible issue, when upgrade to milvus latest, the query coord may skip update dist due to the lastModifyTs doesn't changes. but for old version querynode, the lastModifyTs in GetDataDistritbuionResponse is always 0, which makes qc skip update dist. then qc will keep retry the task to watch channel again and again. this PR add compatible with old version querynode, when lastModifyTs is 0, qc will update it's data distribution. Signed-off-by: Wei Liu <wei.liu@zilliz.com>
v2.4.1 --> master-20240530-589d4dfd-amd64: pass |
Is there an existing issue for this?
Environment
Current Behavior
Expected Behavior
No response
Steps To Reproduce
No response
Milvus Log
failed job: https://qa-jenkins.milvus.io/blue/organizations/jenkins/rolling_update_for_operator_test_simple/detail/rolling_update_for_operator_test_simple/3359/pipeline
log:
artifacts-kafka-mixcoord-3359-server-logs.tar.gz
Anything else?
No response
The text was updated successfully, but these errors were encountered: