Discussions

• 1 You must be logged in to vote

  🙏

  data type during inference?

  RanchiZhao asked Apr 15, 2024 in Q&A · Answered

  

  

  1
• 3 You must be logged in to vote

  🙏

  Is there any documentation on how to use the AutoGPTQ quantized model to train adapters?

  mzbac asked Jun 8, 2023 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  How can I use GPTQ+ LoRA method to finetune, and directly merge GPTQ and LoRA modules to int4, and do inference?

  RanchiZhao asked Apr 12, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  量化能不使用GPU吗

  lijihua2017 asked Apr 9, 2024 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Autogptq packages for Apple Silicon M3 with GPU.

  krasaee asked Nov 24, 2023 in Q&A · Unanswered

  

  

  

  

  9
• 1 You must be logged in to vote

  🙏

  ValueError: QuantLinear() does not have a parameter or a buffer named weight.

  ciliamadani asked Dec 27, 2023 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  AttributeError: 'LlamaAttention' object has no attribute 'qkv_proj'

  ciliamadani asked Dec 27, 2023 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  Installed auto_gptq but python does not recognize it?

  quantuan125 asked Jul 30, 2023 in Q&A · Unanswered

  

  

  1
• 2 You must be logged in to vote

  🙏

  How to estimate VRAM requirement for GPTQ Quantization?

  attkap asked Sep 27, 2023 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  💬

  核显能支持gptq吗

  yangzehan started Sep 8, 2023 in General

  

  

  0
• 1 You must be logged in to vote

  💬

  Cash grant?

  TikkunCreation started Sep 1, 2023 in General

  

  

  0
• 1 You must be logged in to vote

  💬

  New Own Dataset

  Okeke-Stephen started Aug 28, 2023 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  AutoGPTQ with TorchServe

  mapa17 asked Aug 12, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  How to speed up the model?

  VlaTal1 asked Jul 21, 2023 in Q&A · Unanswered

  

  

  

  

  23
• 11 You must be logged in to vote

  🙏

  Converting a the LLava model

  Don-Chad asked Jul 30, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  Is there a way to get the embeddings of a text ?

  ParisNeo started Jul 14, 2023 in General

  

  

  0
• 1 You must be logged in to vote

  🙏

  Why losses are higher for GPTJ than LLama?

  ri938 asked Jun 20, 2023 in Q&A · Closed · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  Does this Model Supports Transformers Pipeline and Sentence Transformers?

  fahadh4ilyas asked May 31, 2023 in Q&A · Unanswered

  

  

  0
• 0 You must be logged in to vote

  💡

  Which model else you want to be included in auto-gptq?

  PanQiWei started May 26, 2023 in Ideas · Closed

  

  

  0