To run multiple OpenAI-compatible end points in the same front-end. #2200
thusinh1969
started this conversation in
General
Replies: 1 comment
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Is your feature request related to a problem? Please describe.
GGUF posed accuracy drop because of missing of bfloat16. We have to switch to vLLM and Triton.
Describe the solution you'd like
Wishes to be able to run multiple OpenAI-compatible end points in the same front-end of Open WebUI likes what it does with ollama's multiple GGUF.
Additional context
This will be great as Open WebUI is one of the best for simplification, pre-prompting and all.
Thanks,
Steve
Beta Was this translation helpful? Give feedback.
All reactions