Discussions

• 1 You must be logged in to vote

  🙏

  Loading checkpoint shards take forever: llama-13b

  Cherchercher asked Apr 18, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  Unexpected Response from OpenLLM on RTX 3090

  zak23 started Mar 11, 2024 in General

  

  

  0
• 2 You must be logged in to vote

  💬

  When Google Gemma model will integrate to OpenLLM

  charles-123456 started Feb 26, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  What font is used in the readme demo gif?

  varunaswal asked Feb 26, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  Is beam search supported with vllm backend?

  yctam started Feb 21, 2024 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  error downloading Llama2

  yalicohen12 asked Jan 31, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Is there a docker-compose.yaml sample file for OpenLLM container available somewhere?

  ghevge asked Jan 30, 2024 in Q&A · Unanswered

  

  

  0
• 3 You must be logged in to vote

  🙏

  openllm- embedding models

  RohithPRK asked Jan 24, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  What are the models that are suited to run on m1 macbook pro

  Bennoo asked Jun 27, 2023 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  💬

  Why my chat responses are very bad?

  gaolitao started Jan 5, 2024 in General

  

  

  0
• 1 You must be logged in to vote

  💬

  Inference GPTQ Model via bentoml

  OriAlpha started Dec 27, 2023 in General

  

  

  0
• 1 You must be logged in to vote

  💬

  OpenLLM on CPU

  QLutz started Dec 19, 2023 in General

  

  

  1
• 1 You must be logged in to vote

  💬

  Sweating a RTX 4090: Switching between multiple LLMs (from the API or using a "router")

  Speedway1 started Dec 19, 2023 in General

  

  

  0
• 2 You must be logged in to vote

  🙏

  Any way to use OpenLLM endpoints with OpenAI and LangChain?

  newparad1gm asked Dec 11, 2023 in Q&A · Unanswered

  

  

  4
• 1 You must be logged in to vote

  🙏

  Benchmark speed between OpenAI and OpenLLM

  louis030195 asked Dec 12, 2023 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Using adapters in a bentofile

  QLutz asked Dec 5, 2023 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💬

  vLLM and adapters

  QLutz started Dec 4, 2023 in General · Closed

  

  

  2
• 1 You must be logged in to vote

  🙏

  Differences between gpt4all and OpenLLM?

  code2graph asked Aug 28, 2023 in Q&A · Unanswered

  

  

  

  

  3
• 1 You must be logged in to vote

  🙏

  start intfloat/e5-large-v2

  tbugfinder asked Nov 17, 2023 in Q&A · Closed · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  load multiple (two) model-ids

  tbugfinder asked Nov 9, 2023 in Q&A · Closed · Unanswered

  

  

  3
• 1 You must be logged in to vote

  🙏

  embeddings endpoint understanding

  wei-ann-Github asked Oct 3, 2023 in Q&A · Unanswered

  

  

  4
• 1 You must be logged in to vote

  🙏

  GPT4ALL and OpenLLM

  RamishSiddiqui asked Sep 10, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  How are multiple API clients handled?

  geordie2020 started Sep 7, 2023 in General

  

  

  2
• 2 You must be logged in to vote

  💬

  Ideal Infrastructure

  jordanparker6 started Jun 18, 2023 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  Does openllm start support model sharding at inference?

  gbmarc1 asked Aug 28, 2023 in Q&A · Unanswered

  

  

  2