Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

对Ollama模型加载keep_alive参数的支持,让模型可以在显存内保留更长时间 #1536

Closed
1 of 2 tasks
SuperMaxine opened this issue May 11, 2024 · 3 comments
Closed
1 of 2 tasks
Labels
enhancement New feature or request

Comments

@SuperMaxine
Copy link

SuperMaxine commented May 11, 2024

Search before asking

  • issues 中没有找到类似的内容。
    I searched in the issues and found nothing similar.

feature

加入调用Ollama模型时对keep_alive参数的设置选项

描述 Motivation

ollama默认在5分钟未使用后会将模型从显存中卸载,导致下次调用时会经历较长时间的加载过程,较为影响使用体验。

Solution

Ollama在api调用时提供了keep_alive选项可以设置保留时长或永不卸载选项,希望可以加入设置页面让用户自主设置。详见Ollama关于API调用的文档:
https://github.com/ollama/ollama/blob/c02db93243353855b983db2a1562a02b57e66db1/docs/faq.md#how-do-i-keep-a-model-loaded-in-memory-or-make-it-unload-immediately

还有其他内容吗?Anything else?

No response

你是否愿意提交一份 PR?Are you willing to submit a PR?

  • 我愿意提供 PR! I'm willing to submit a PR!
@SuperMaxine SuperMaxine added the enhancement New feature or request label May 11, 2024
@yetone
Copy link
Collaborator

yetone commented May 12, 2024

好的٩(•̤̀ᵕ•̤́๑)ᵒᵏᵎᵎᵎᵎ

@yetone
Copy link
Collaborator

yetone commented May 12, 2024

最新版已支持 嘻嘻

@yetone yetone closed this as completed May 12, 2024
@SuperMaxine
Copy link
Author

最新版已支持 嘻嘻

!!!!!!!

大佬好快!!!!

大佬nb!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants