We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
在MacOS,用python调用qwen_cpp载入模型进行推理,只能启动CPU,无法使用GPU。
执行代码: from qwen_cpp import Pipeline model = "/Users/xxx/AIproject/qwen.cpp/qwen14b-ggml.bin" tiktoken = "/Users/xxx/AIproject/models/Qwen/Qwen-14B-Chat/qwen.tiktoken" pipeline = Pipeline(model, tiktoken) pipeline.chat([sentence])
请问如何使用python推理时使用GPU推理?
我用命令行的方式推理是可以使用GPU推理的,速度很快。
我的电脑配置是:Macbook Pro M3 Max 64G macOS 版本:14.1.2 (23B2091)
The text was updated successfully, but these errors were encountered:
为什么我们正好相反,我想只在cpu上运行,结果我在命令行中是cpu,使用你上面的代码却是在所有gpu上一起运行。。
Sorry, something went wrong.
No branches or pull requests
在MacOS,用python调用qwen_cpp载入模型进行推理,只能启动CPU,无法使用GPU。
执行代码:
from qwen_cpp import Pipeline
model = "/Users/xxx/AIproject/qwen.cpp/qwen14b-ggml.bin"
tiktoken = "/Users/xxx/AIproject/models/Qwen/Qwen-14B-Chat/qwen.tiktoken"
pipeline = Pipeline(model, tiktoken)
pipeline.chat([sentence])
请问如何使用python推理时使用GPU推理?
我用命令行的方式推理是可以使用GPU推理的,速度很快。
我的电脑配置是:Macbook Pro M3 Max 64G
macOS 版本:14.1.2 (23B2091)
The text was updated successfully, but these errors were encountered: