Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix excessive memory usage #562

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

x-wesley
Copy link

When the website has too much text content, the response will take up too much memory.

Use sync.Pool to reduce memory applications.

Thanks.

@WGH-
Copy link
Collaborator

WGH- commented Nov 26, 2020

Does it really help that much, given that the []byte body, headers, and even net/http.Response objects are still allocated?

@x-wesley
Copy link
Author

Does it really help that much, given that the []byte body, headers, and even net/http.Response objects are still allocated?

Yeah, when there are a lot of URLs in a website and a lot of content on the webpage, a lot of memory space will be requested in a short time.

@asciimoo
Copy link
Member

@x-wesley do you have measurements about how much improvement this change causes?

@x-wesley
Copy link
Author

x-wesley commented Dec 1, 2020

@x-wesley do you have measurements about how much improvement this change causes?

yeah
In May, we have a scheduled task. We will receive a high memory usage alarm (>80%) every time the task is executed.
The problem is located through pprof, and the peak memory usage after modification is about 50%-60%.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants