Circuit breaker not working #48204
Replies: 3 comments 3 replies
-
The circuit breaker is about evicting endpoints within a group (service+subset+port). So you have a group (helloworldv2), with 0 endpoints. You explicitly send to this (via the VS), and there is no endpoints so it fails. The circuit breaker is irrelevant here. You can kind of work around this in a way using "locality lb". This was originally intended for other purposes but works here:
And then do not use subsets at all. This will send requests first to things matching version=v1, then to version=v2 if those are failing. |
Beta Was this translation helpful? Give feedback.
-
Hello John. apiVersion: networking.istio.io/v1beta1 kind: DestinationRule metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: > {"apiVersion":"networking.istio.io/v1beta1","kind":"DestinationRule","metadata":{"annotations":{},"creationTimestamp":"2024-04-20T00:20:45Z","generation":1,"managedFields":[{"apiVersion":"networking.istio.io/v1alpha3","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{".":{},"f:host":{},"f:subsets":{}}},"manager":"kubectl-client-side-apply","operation":"Update","time":"2024-04-20T00:20:45Z"}],"name":"reviews","namespace":"bookinfo","resourceVersion":"55546","uid":"af9cee21-4bdd-462d-b89f-eedcf917f4ee"},"spec":{"host":"reviews","subsets":[{"labels":{"version":"v1"},"name":"v1"},{"labels":{"version":"v2"},"name":"v2"},{"labels":{"version":"v3"},"name":"v3"}],"trafficPolicy":{"connectionPool":{"http":{"http1MaxPendingRequests":1,"maxRequestsPerConnection":1},"tcp":{"maxConnections":1}},"outlierDetection":{"baseEjectionTime":"3m","consecutive5xxErrors":1,"interval":"1s","maxEjectionPercent":100}}}} creationTimestamp: '2024-04-20T00:20:45Z' generation: 2 managedFields: - apiVersion: networking.istio.io/v1alpha3 fieldsType: FieldsV1 fieldsV1: 'f:metadata': 'f:annotations': {} 'f:spec': .: {} 'f:host': {} 'f:subsets': {} manager: kubectl-client-side-apply operation: Update time: '2024-04-20T00:20:45Z' - apiVersion: networking.istio.io/v1beta1 fieldsType: FieldsV1 fieldsV1: 'f:metadata': 'f:annotations': 'f:kubectl.kubernetes.io/last-applied-configuration': {} 'f:spec': 'f:trafficPolicy': .: {} 'f:connectionPool': .: {} 'f:http': .: {} 'f:http1MaxPendingRequests': {} 'f:maxRequestsPerConnection': {} 'f:tcp': .: {} 'f:maxConnections': {} 'f:outlierDetection': .: {} 'f:baseEjectionTime': {} 'f:consecutive5xxErrors': {} 'f:interval': {} 'f:maxEjectionPercent': {} manager: kubectl-client-side-apply operation: Update time: '2024-04-20T22:18:40Z' name: reviews namespace: bookinfo resourceVersion: '95768' uid: af9cee21-4bdd-462d-b89f-eedcf917f4ee spec: host: reviews trafficPolicy: connectionPool: http: maxRequestsPerConnection: 1 loadBalancer: simple: ROUND_ROBIN localityLbSetting: enabled: true failoverPriority: - version=v1 - version=v2 - version=v3 outlierDetection: consecutive5xxErrors: 1 interval: 1s baseEjectionTime: 2m maxEjectionPercent: 100 however, did get expected results, all passed, using Fortio: Appreciate if you may advise. Thank you |
Beta Was this translation helpful? Give feedback.
-
Environment details: |
Beta Was this translation helpful? Give feedback.
-
I have deployed a sample app (helloword) with two versions (v1 and v2), I have defined circuit breaker to stop sending traffic to unhealthy deployment, after I delete manually the v2 deployment, and observing the kiali graph, it does not seem to stop the traffic to unhealthy deployment (v2).
Deployment.yaml:
Gateway-vs-dr.yaml:
Beta Was this translation helpful? Give feedback.
All reactions