可以将通过lora训练好的模型进行量化后导出吗
#1692
Replies: 1 comment
-
|
可参考 https://github.com/PanQiWei/AutoGPTQ 在服务器端量化 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
我当前在云服务器中对模型进行训练，但是本地进行推理时无法加载训练好的13B的模型，所以有办法将训练好的模型进行量化导出吗
Beta Was this translation helpful? Give feedback.
All reactions