Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(deleteAction): Queue delete requests #45237

Merged
merged 3 commits into from May 15, 2024
Merged

Conversation

Fenn-CS
Copy link
Contributor

@Fenn-CS Fenn-CS commented May 8, 2024

When multiple files are deleted at once, all the requests bombard the server simultaneously, causing performance issues.

This commit adds queuing that limits the concurrency of these requests to 5 at a time.


Without Queuing

not-queued-requests

With Queuing

queued-requests

Resolves : #45234

@Fenn-CS Fenn-CS requested review from susnux, artonge and Pytal May 8, 2024 20:10
@Fenn-CS Fenn-CS self-assigned this May 8, 2024
@Fenn-CS Fenn-CS requested a review from skjnldsv as a code owner May 8, 2024 20:10
@Fenn-CS Fenn-CS force-pushed the 45234-que-deletion-requests branch from f53850e to c650f7d Compare May 8, 2024 20:11
@Fenn-CS Fenn-CS force-pushed the 45234-que-deletion-requests branch from c650f7d to 35e9de0 Compare May 9, 2024 08:52
@Fenn-CS Fenn-CS requested a review from Pytal May 9, 2024 08:56
Copy link
Member

@joshtrichards joshtrichards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly an improvement, but since this is all browser side aren't we creating a new problem: i.e. if the browser is closed before the entire delete session finishes?

Maybe we go farther and add the full item list to a new background job handled server-side? We already expire the trash contents in a background job so...

Perhaps more appropriate in a future follow-up (if the browser being closed aborting the delete queue is considered acceptable for now).

@susnux
Copy link
Contributor

susnux commented May 10, 2024

Maybe we go farther and add the full item list to a new background job handled server-side? We already expire the trash contents in a background job so...

I do not know any WebDAV standard allowing bulk delete so this would need a custom API so definitely a thing we need to think through.

@susnux susnux force-pushed the 45234-que-deletion-requests branch from 35e9de0 to 36ec764 Compare May 10, 2024 16:18
@susnux
Copy link
Contributor

susnux commented May 10, 2024

/compile

@solracsf
Copy link
Member

solracsf commented May 10, 2024

Certainly an improvement, but since this is all browser side aren't we creating a new problem: i.e. if the browser is closed before the entire delete session finishes?

What about adding a word on the already exitsting message like:

Bulk delete in progress, please keep this window open in your browser

Could this be a thing?

@Fenn-CS
Copy link
Contributor Author

Fenn-CS commented May 13, 2024

@solracsf I tracked your concerns here : #45281 I think we can manage fix it from there.

@Fenn-CS Fenn-CS force-pushed the 45234-que-deletion-requests branch 2 times, most recently from 4181490 to 424989c Compare May 13, 2024 10:04
@Fenn-CS Fenn-CS force-pushed the 45234-que-deletion-requests branch from 424989c to 5e74c11 Compare May 14, 2024 08:24
@Fenn-CS
Copy link
Contributor Author

Fenn-CS commented May 14, 2024

/compile

@artonge
Copy link
Contributor

artonge commented May 14, 2024

/backport to stable29

@artonge
Copy link
Contributor

artonge commented May 14, 2024

/backport to stable28

@artonge
Copy link
Contributor

artonge commented May 14, 2024

/backport to stable27

Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
When multiple files are deleted at once, all the requests bombard the server
simultaneously, causing performance issues.

This commit adds queuing that limits the concurrency of these requests to 5
at a time.

Signed-off-by: fenn-cs <fenn25.fn@gmail.com>
@Fenn-CS Fenn-CS force-pushed the 45234-que-deletion-requests branch from d52b8a8 to 21a22a3 Compare May 15, 2024 10:51
@Fenn-CS
Copy link
Contributor Author

Fenn-CS commented May 15, 2024

/compile

Signed-off-by: nextcloud-command <nextcloud-command@users.noreply.github.com>
@Altahrim Altahrim merged commit 0ad77fa into master May 15, 2024
104 checks passed
@Altahrim Altahrim deleted the 45234-que-deletion-requests branch May 15, 2024 12:08
@Altahrim Altahrim removed the 3. to review Waiting for reviews label May 15, 2024
Copy link

backportbot bot commented May 15, 2024

The backport to stable27 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable27
git pull origin stable27

# Create the new backport branch
git checkout -b backport/45237/stable27

# Cherry pick the change from the commit sha1 of the change against the default branch
# This might cause conflicts, resolve them
git cherry-pick 23c71bd8 21a22a39 18f5507e

# Push the cherry pick commit to the remote repository and open a pull request
git push origin backport/45237/stable27

Error: No changes found in backport branch


Learn more about backports at https://docs.nextcloud.com/server/stable/go.php?to=developer-backports.

@artonge
Copy link
Contributor

artonge commented May 16, 2024

/backport to stable27

Copy link

backportbot bot commented May 16, 2024

The backport to stable27 failed. Please do this backport manually.

# Switch to the target branch and update it
git checkout stable27
git pull origin stable27

# Create the new backport branch
git checkout -b backport/45237/stable27

# Cherry pick the change from the commit sha1 of the change against the default branch
# This might cause conflicts, resolve them
git cherry-pick 23c71bd8 21a22a39 18f5507e

# Push the cherry pick commit to the remote repository and open a pull request
git push origin backport/45237/stable27

Error: No changes found in backport branch


Learn more about backports at https://docs.nextcloud.com/server/stable/go.php?to=developer-backports.

Copy link

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🏗️ In progress
Development

Successfully merging this pull request may close these issues.

[Bug]: Performance degradation due to multiple simultaneous deletion requests from trashbin
8 participants