Custom context size for local models #2586
Closed
gelim
started this conversation in
Feature Requests & Suggestions
Replies: 1 comment 3 replies
-
|
Yes, this week there will be a solution for this. |
Beta Was this translation helpful? Give feedback.
3 replies
-
|
hey @danny-avila how did you fix this? |
Beta Was this translation helpful? Give feedback.
-
|
Ping @danny-avila :) |
Beta Was this translation helpful? Give feedback.
-
|
Sorry forgot to close this discussion. It's implemented here:
You can use presets now for this |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
I'm stumbling on the 4096 context size limit on local models served via litellm.
Is there a way/plan to support custom context size limit?
Cheers
Beta Was this translation helpful? Give feedback.
All reactions