Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: AMS get get hang after few days with kuberntes deployment #2273

Open
2 tasks done
GavinH1984 opened this issue Nov 9, 2023 · 2 comments · May be fixed by #2346
Open
2 tasks done

[Bug]: AMS get get hang after few days with kuberntes deployment #2273

GavinH1984 opened this issue Nov 9, 2023 · 2 comments · May be fixed by #2346
Labels
type:bug Something isn't working

Comments

@GavinH1984
Copy link
Contributor

What happened?

Ams get hang after few days runing in kubernetes environment.

Affects Versions

master

What engines are you seeing the problem on?

AMS

How to reproduce

  • Deploy the ams into kurbernetes environment.
  • Ams memory set Xms=8196m, XMX=8196m
  • Kurbernetes deploy set the resource limit to 4 core, 16GB memory.
  • After few days, the pod memory will continue increase, when it exceed the limit ot kurberntes deployment, the ams will and hang and get restart.

Relevant log output

N/A

Anything else

relate jmap:

jmap.log

jmap heap

image

native memory summary:

image

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

  • I agree to follow this project's Code of Conduct
@GavinH1984 GavinH1984 added the type:bug Something isn't working label Nov 9, 2023
@baiyangtx
Copy link
Contributor

The memory usage of the Class Metadata (1GB) and Internal (2GB) sections seems to exceed expectations by a significant amount.

@baiyangtx
Copy link
Contributor

@zhoujinsong
Should we introduce these two parameters in the JVM arguments of AMS to control off-heap memory size?

-XX:MaxDirectMemorySize

-XX:MaxMetaspaceSize

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:bug Something isn't working
Projects
None yet
2 participants