HTTPChaos not injecting faults with an Istio sidecar

[varun-jupiter]

Bug Report

What version of Kubernetes are you using?
Client Version: v1.28.4
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.10-eks-508b6b3

What version of Chaos Mesh are you using?
Controller manager Version: version.Info{GitVersion:"chart-2.6.3", GitCommit:"b159d1c5a361ca350ca7789c9eb7fc917479db39", BuildDate:"2024-01-23T15:04:53Z", GoVersion:"go1.19.7", Compiler:"gc", Platform:"linux/amd64"}

What did you do? / Minimal Reproducible Example
We have an istio service mesh which runs a istio-proxy sidecar for every pod.
I tried injecting HTTPChaos into a few pods.
Daemon logs give no error and the chaos_tproxy is up and running however, the requests are not passing through the chaos_tproxy.

On disabling the sidecar using sidecar.istio.io/inject: false, the HTTPChaos gets applied just fine and the daemon logs show the requests being processed.

Further, on recovering chaos, the pods start restarting since the health check fails. This problem does not auto resolve and the only way around this is redeploying the application.

What did you expect to see?
Even with the istio sidecar, the HTTPChaos should be applied fine.
There should be no impact on the recovery of chaos on the application.

What did you see instead?
Specified above in What did you do.

Output of chaosctl

[varun-jupiter]

Chaos injected was:

apiVersion: chaos-mesh.org/v1alpha1
metadata:
  namespace: ****
  name: wefwefg-1
spec:
  selector:
    namespaces:
      - ****
    labelSelectors:
      app: ****
  mode: all
  target: Response
  delay: 5s
  port: 8080
  path: /some/path/*
  duration: 3m

[STRRL]

unfortunately, chaos mesh could not "just works" with istio, becaus some functionalities of both chaos mesh and istio require modify pod network.

We do not do enough works on the compatibility with istio. we would be very appreciated if you could help us improve this.