[Feature] We need to keep the state of the jobs on the cluster consistent with the state of the jobs on the platform

[Mrart]

Search before asking

• I had searched in the feature and found no similar feature requirement.

Description

1. Synchronize the job status when the k8s/yarn cluster is abnormal
2. If the operation is abnormal, the status can be synchronized to the platform
3. After the platform is restarted, the job status can be synchronized

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[caicancai]

@Mrart I am happy to participate in this part of the discussion. Do you have a corresponding solution? I also think this function is very important. I have also been trying to sort it out recently.