Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Having issue loading my HQQ quantized model #35

Open
BeichenHuang opened this issue Apr 24, 2024 · 0 comments
Open

Having issue loading my HQQ quantized model #35

BeichenHuang opened this issue Apr 24, 2024 · 0 comments

Comments

@BeichenHuang
Copy link

BeichenHuang commented Apr 24, 2024

Hi! I am trying to load my HQQ quantized model using the offloading strategy, but I have problem in the model safetensor files.
I notice that in your HQQ quantized model safetensor files, the weights, taking layer 0 expert 1 as an example, are saved like:
956eff2810816630e5aebacff688349
But I use the the code from official HQQ websit, the saved model is only one .pt file:
How to splite the weight into all thoses components?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant