IO chaos injection delete incompletely, chaosFS still exists.

[niaoniconi]

Bug Report: about io chaos deletion

---

kubectl version

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.3", GitCommit:"816c97ab8cff8a1c72eccca1026f7820e93e0d25", GitTreeState:"clean", BuildDate:"2022-01-25T21:25:17Z", GoVersion:"go1.17.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.11", GitCommit:"5824e3251d294d324320db85bf63a53eb0767af2", GitTreeState:"clean", BuildDate:"2022-06-16T05:33:55Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}

chaos-mesh version

---

how to reproduce error

• use yaml to create io chaos, yaml file:

kind: IOChaos
apiVersion: chaos-mesh.org/v1alpha1
metadata:
  namespace: test
  name: test-latency
spec:
  selector:
    namespaces:
      - test
    labelSelectors:
      statefulset.kubernetes.io/pod-name: mxchaos-test-1-0
  mode: one
  action: latency
  delay: 20ms
  path: /mxdata/primary/mxseg0/base
  percent: 100
  volumePath: /mxdata/primary/mxseg0/base
  duration: 30s

command:

k apply -f latency-test.yaml

• use yaml to delete io chaos

command:

k delete -f latency-test.yaml

Expect

• create success
  the kubectl command run successfully

k apply -f latency-test.yaml
iochaos.chaos-mesh.org/test-latency created

i can see the process and the chaos-fs

root       1541  0.0  0.0 4440352 5580 ?        S    10:25   0:00 /usr/local/bin/toda --path /mxdata/primary/mxseg0/base --verbose inf

drwxr-xr-x 2 root    root       6 Mar 29 10:25 __chaosfs__base_

• delete
  i want the kubectl delete command run successfully
  i want the process to be canceled
  i want the chaosFS __chaosfs__base_ to be deleted

See

• create io chaos met expect
• delete io chaos
  i see the process to be canceled
  but the chaosFS __chaosfs__base_ still exist
  and the command never finished, sustain follow status for minutes(at list half hours)

k delete -f latency-test.yaml
iochaos.chaos-mesh.org "test-latency" deleted

---

Output of chaosctl

I only paste part of logs because of too much logs,ask me to paste more if need

2024-03-29T10:14:21.933Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:97    received request        {"webhook": "/mutate-chaos-mesh-org-v1alpha1-iochaos", "UID": "ed2c6aa7-ea5c-4967-afb9-c23c69cd33a5", "kind": "chaos-mesh.org/v1alpha1, Kind=IOChaos", "resource": {"group":"chaos-mesh.org","version":"v1alpha1","resource":"iochaos"}}
        2024-03-29T10:14:21.935Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:136   wrote response  {"webhook": "/mutate-chaos-mesh-org-v1alpha1-iochaos", "code": 200, "reason": "", "UID": "ed2c6aa7-ea5c-4967-afb9-c23c69cd33a5", "allowed": true}
        2024-03-29T10:14:21.940Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:97    received request        {"webhook": "/validate-chaos-mesh-org-v1alpha1-iochaos", "UID": "4e803107-34a2-4f3c-8e32-d1f68afef6eb", "kind": "chaos-mesh.org/v1alpha1, Kind=IOChaos", "resource": {"group":"chaos-mesh.org","version":"v1alpha1","resource":"iochaos"}}
        2024-03-29T10:14:21.941Z        INFO    IOChaos-resource        logr@v1.2.2/logr.go:252 validate update {"name": "test-latency"}
        2024-03-29T10:14:21.942Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:136   wrote response  {"webhook": "/validate-chaos-mesh-org-v1alpha1-iochaos", "code": 200, "reason": "", "UID": "4e803107-34a2-4f3c-8e32-d1f68afef6eb", "allowed": true}
        2024-03-29T10:14:21.979Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:97    received request        {"webhook": "/mutate-chaos-mesh-org-v1alpha1-iochaos", "UID": "247c7579-1664-4611-8431-a391c6965fe9", "kind": "chaos-mesh.org/v1alpha1, Kind=IOChaos", "resource": {"group":"chaos-mesh.org","version":"v1alpha1","resource":"iochaos"}}
        2024-03-29T10:14:21.982Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:136   wrote response  {"webhook": "/mutate-chaos-mesh-org-v1alpha1-iochaos", "code": 200, "reason": "", "UID": "247c7579-1664-4611-8431-a391c6965fe9", "allowed": true}
        2024-03-29T10:14:21.987Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:97    received request        {"webhook": "/validate-chaos-mesh-org-v1alpha1-iochaos", "UID": "c7f71239-0c56-4d7c-b206-6b11bdf7f3b8", "kind": "chaos-mesh.org/v1alpha1, Kind=IOChaos", "resource": {"group":"chaos-mesh.org","version":"v1alpha1","resource":"iochaos"}}
        2024-03-29T10:14:21.988Z        INFO    IOChaos-resource        logr@v1.2.2/logr.go:252 validate update {"name": "test-latency"}
        2024-03-29T10:14:21.988Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:136   wrote response  {"webhook": "/validate-chaos-mesh-org-v1alpha1-iochaos", "code": 200, "reason": "", "UID": "c7f71239-0c56-4d7c-b206-6b11bdf7f3b8", "allowed": true}
        2024-03-29T10:14:22.543Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:97    received request        {"webhook": "/mutate-chaos-mesh-org-v1alpha1-iochaos", "UID": "0d54a47e-3b82-4377-a313-5dc8b070178f", "kind": "chaos-mesh.org/v1alpha1, Kind=IOChaos", "resource": {"group":"chaos-mesh.org","version":"v1alpha1","resource":"iochaos"}}
        2024-03-29T10:14:22.545Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:136   wrote response  {"webhook": "/mutate-chaos-mesh-org-v1alpha1-iochaos", "code": 200, "reason": "", "UID": "0d54a47e-3b82-4377-a313-5dc8b070178f", "allowed": true}
        2024-03-29T10:14:22.551Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:97    received request        {"webhook": "/validate-chaos-mesh-org-v1alpha1-iochaos", "UID": "3af58668-be4e-4b87-a0ea-d6364f4b9f1f", "kind": "chaos-mesh.org/v1alpha1, Kind=IOChaos", "resource": {"group":"chaos-mesh.org","version":"v1alpha1","resource":"iochaos"}}
        2024-03-29T10:14:22.551Z        INFO    IOChaos-resource        logr@v1.2.2/logr.go:252 validate update {"name": "test-latency"}
        2024-03-29T10:14:22.552Z        DEBUG   controller-runtime.webhook.webhooks     admission/http.go:136   wrote response  {"webhook": "/validate-chaos-mesh-org-v1alpha1-iochaos", "code": 200, "reason": "", "UID": "3af58668-be4e-4b87-a0ea-d6364f4b9f1f", "allowed": true}
        2024-03-29T10:15:25.464Z        INFO    setup   logr@v1.2.2/logr.go:252 watcher channel has closed, restart watcher
        2024-03-29T10:15:27.465Z        INFO    setup   logr@v1.2.2/logr.go:252 Launching watcher for ConfigMaps
        2024-03-29T10:15:27.465Z        INFO    inject-webhook  logr@v1.2.2/logr.go:252 Watching for ConfigMaps for changes     {"template namespace": "chaos-mesh", "labels": {"app.kubernetes.io/component":"webhook"}}
        2024-03-29T10:15:27.476Z        INFO    setup   logr@v1.2.2/logr.go:252 Triggering ConfigMap reconciliation
        2024-03-29T10:15:27.476Z        INFO    inject-webhook  logr@v1.2.2/logr.go:252 Fetching Template Configs...