fix(core): Handle removing non-empty directories and non-existing paths in cache

[Exelord]

Current Behavior

When running NX concurrently sometimes NX is trying to remove non-existing path (which have been already deleted by other process) or non-empty directories (which contains files created by other process but not yet removed)

To prevent race conditions, remove from fs-extra works in following way:

Removes a file or directory. The directory can have contents. If the path does not exist, silently does nothing.

Expected Behavior

Remove operations should work with non-existing paths as well as non-empty directories

Related Issue(s)

Fixes #22140

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Updated (UTC)
nx-dev	⬜️ Ignored (Inspect)	Visit Preview	May 9, 2024 6:40pm

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 936f05b. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

🟥 Failed Commands
nx-cloud record -- nx format:check --base=0322b9804f3328b76acfa2848e52acf3d28cc55f --head=936f05b02c7e88489761e30fb0226cb0e449faf6

✅ Successfully ran 2 targets

• nx documentation --no-dte
• nx affected -t e2e-macos-ci --parallel=1 --base=0322b9804f3328b76acfa2848e52acf3d28cc55f --head=936f05b02c7e88489761e30fb0226cb0e449faf6

---

Sent with 💌 from NxCloud.

[FrozenPandaz]

Thank you so much for isolating the issue!

Hm, this removes the usage of our rust implementation of removing files where we also use a fs-extras type library.

https://github.com/nrwl/nx/blob/master/packages/nx/src/native/cache/file_ops.rs#L8

The rust one proved to be much faster so we should continue using that one and address the issue there.

@Exelord If you're familiar with Rust, we would appreciate if you continue looking into the issue. If not, no worries, you've been a great help already. @AgentEnder can take over from here.

[Exelord]

Great! Please do take over. I have tried to use rust fs-extra but it does not work the way described, so it would require some native implementation which I cannot help with. While JS remove might be slower it does fix the issue, so I went with that as a quick fix.

Thank you :)