Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[llm] modelscope has no _chat_stream #357

Open
zzhangpurdue opened this issue Mar 19, 2024 · 0 comments
Open

[llm] modelscope has no _chat_stream #357

zzhangpurdue opened this issue Mar 19, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request llm issues about llm usage

Comments

@zzhangpurdue
Copy link
Collaborator

          > 感谢,你的反馈,关于例子我们有版本更新,你看的是中文版本的readme中确实没有更新,请先参考英文文档,我立刻更新。 另外,调用小模型的例子目前采用的是vllm的方案,用户可以本地部署vllm模型拉起,具体参考:https://github.com/modelscope/modelscope-agent/blob/master/docs/local_deploy.md 可以参考本地部署的文档,将模型部署成服务的方式。

同时,我们也会持续增加使用case。

感谢你的回复。我这里的需求应该没有这么复杂,也不需要gradio的UI,只是想使用modelscope上已有的模型,比如Qwen-7B之类的,通过直接下载的方式将模型下载到本地(这个是可以完成的),然后跑一个简单的比如天气查询或者图像生成之类的任务。现在看起来modelscope上的模型因为没有实现_chat_stream方法而没有办法运行。

Originally posted by @douyh in #353 (comment)

@zzhangpurdue zzhangpurdue self-assigned this Mar 19, 2024
@zzhangpurdue zzhangpurdue added enhancement New feature or request llm issues about llm usage labels Mar 19, 2024
@zzhangpurdue zzhangpurdue changed the title [llm] modelscope has not _chat_stream [llm] modelscope has no _chat_stream Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request llm issues about llm usage
Projects
None yet
Development

No branches or pull requests

1 participant