Discussions

• 1 You must be logged in to vote

  🙏

  lmdeploy有generator api可以只返回output ids而不是output string吗？

  haoxufd asked May 9, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💬

  How to load a model in bf16 format

  georgel2004 started May 1, 2024 in General · Closed

  

  

  1
• 1 You must be logged in to vote

  💬

  支持的transformers版本可以更新到4.39以上吗w

  IcyFeather233 started Apr 25, 2024 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  Does lmdeploy support llama3 deployment?

  Lisaiplus asked Apr 23, 2024 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  lmdeploy中TP设置成2时，当模型不能完全加载到一张显卡中，会对模型进行分层并加载到两张GPU中吗？

  hello-gary-2022 asked Apr 23, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  离线推理 pipeline 模式支持连续批处理吗？

  shell-nlp asked Apr 4, 2024 in Q&A · Closed · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  stream_infer input_embeddings

  JidongZhang-THU asked Dec 26, 2023 in Q&A · Answered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  How to custom <|SYS|> contents when using API? (after deployed)

  maxchiron asked Dec 19, 2023 in Q&A · Answered

  

  

  

  

  8
• 1 You must be logged in to vote

  🙏

  What's the difference between Persistent Batch and continuous batching?

  luweigen asked Nov 20, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  KV Cache int8 cooperate with Flash-attention2

  yhyu13 asked Nov 15, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  CUDA out of memory while Deploy

  anisingh1 asked Oct 5, 2023 in Q&A · Unanswered

  

  

  

  

  5
• 1 You must be logged in to vote

  🙏

  How do I remove the sharing of gradio in public url?

  supersuper1234 asked Sep 29, 2023 in Q&A · Unanswered

  

  

  

  

  2
• 1 You must be logged in to vote

  🙏

  MPI related undefined reference errors while lmdeploy build from source

  anisingh1 asked Sep 28, 2023 in Q&A · Closed · Unanswered

  

  

  0
• 2 You must be logged in to vote

  🙌

  如何使用 LMDeploy 把 InternLM-20B-4bit 部署为服务

  vansin started Sep 22, 2023 in Show and tell

  

  

  

  

  11
• 1 You must be logged in to vote

  💡

  Does LMdeploy support latest version of flash attention, page attention?

  ywangwxd started Sep 26, 2023 in Ideas

  

  

  2
• 1 You must be logged in to vote

  💬

  大模型的 2k和8k是哪个参数决定的？

  vansin started Sep 25, 2023 in General

  

  

  1
• 1 You must be logged in to vote

  🙏

  LMDeploy on mac platform

  anisingh1 asked Sep 20, 2023 in Q&A · Closed · Unanswered

  

  

  1
• 1 You must be logged in to vote

  💬

  How to deploy multiple models in the same workspace?

  amulil started Sep 4, 2023 in General · Closed

  

  

  1
• 1 You must be logged in to vote

  💬

  kv cache 可以关掉吗，我想每次模型重新生成答案，不使用历史数据？

  amulil started Aug 30, 2023 in General · Closed

  

  

  3
• 1 You must be logged in to vote

  💬

  Does LLama2 support dynamic NTK-RoPE？How to open it?

  amulil started Sep 4, 2023 in General · Closed

  

  

  1
• 1 You must be logged in to vote

  💬

  How to use multi node to deploy model?

  amulil started Sep 1, 2023 in General · Closed

  

  

  1
• 1 You must be logged in to vote

  🙏

  How to install lmdeploy.serve.openai?

  realhaik asked Aug 23, 2023 in Q&A · Unanswered

  

  

  

  

  14
• 1 You must be logged in to vote

  🙏

  Does LMDeploy support original meta LLaMA2?

  realhaik asked Aug 20, 2023 in Q&A · Unanswered

  

  

  3