Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

关于LLaMA-70B-PowerInfer-GGUF的chat版本 #143

Open
NerounCstate opened this issue Feb 6, 2024 · 1 comment
Open

关于LLaMA-70B-PowerInfer-GGUF的chat版本 #143

NerounCstate opened this issue Feb 6, 2024 · 1 comment
Labels
question Further information is requested

Comments

@NerounCstate
Copy link

我发现hunggingface只提供了LLaMA-70B的预测器，可以用于chat版本吗？或者会推出chat的预测器吗？
@NerounCstate NerounCstate added the question Further information is requested label Feb 6, 2024
@hodlen
Copy link
Collaborator

hodlen commented Feb 13, 2024

LLaMA-70B的预测器不能用于chat版本模型，这是因为预测器是在我们重新训练模型后，与稀疏版本的模型一一匹配的。我们暂时没有支持Llama chat模型的计划，我们目前对模型支持的计划请见 #93 。

The predictor for LLaMA-70B cannot be used for the chat version of the model because the predictor is matched one-to-one with the sparse version of the model after we retrain it. We currently do not have plans to support the Llama chat model. Please see #93 for our current plans regarding model support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hodlen @NerounCstate