Use merge_lora_weights.py Error:AttributeError: 'TinyLlavaPhiConfig' object has no attribute 'attention_bias'

[mengjiexu]

the error:
[2024-03-15 17:42:38,572] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect) Loading LLaVA from base model... Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Traceback (most recent call last): File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/scripts/merge_lora_weights.py", line 22, in <module> merge_lora(args) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/scripts/merge_lora_weights.py", line 8, in merge_lora tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name, device_map='cpu') File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/builder.py", line 118, in load_pretrained_model model = TinyLlavaLlamaForCausalLM.from_pretrained(model_base, low_cpu_mem_usage=True, config=cfg_pretrained, File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3594, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/language_model/llava_llama.py", line 46, in __init__ self.model = TinyLlavaLlamaModel(config) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/language_model/llava_llama.py", line 38, in __init__ super(TinyLlavaLlamaModel, self).__init__(config) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/llava_arch.py", line 32, in __init__ super(LlavaMetaModel, self).__init__(config) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 958, in __init__ [LlamaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)] File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 958, in <listcomp> [LlamaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)] File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 758, in __init__ self.self_attn = LLAMA_ATTENTION_CLASSES[config._attn_implementation](config=config, layer_idx=layer_idx) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 315, in __init__ self.q_proj = nn.Linear(self.hidden_size, self.num_heads * self.head_dim, bias=config.attention_bias) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/configuration_utils.py", line 265, in __getattribute__ return super().__getattribute__(key) AttributeError: 'TinyLlavaPhiConfig' object has no attribute 'attention_bias'

[JalorOo]

I have not encountered this problem, but I want to know why can you use lora to finetune the model? because I can't found the script. #23

[baichuanzhou]

the error: [2024-03-15 17:42:38,572] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect) Loading LLaVA from base model... Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Traceback (most recent call last): File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/scripts/merge_lora_weights.py", line 22, in <module> merge_lora(args) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/scripts/merge_lora_weights.py", line 8, in merge_lora tokenizer, model, image_processor, context_len = load_pretrained_model(args.model_path, args.model_base, model_name, device_map='cpu') File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/builder.py", line 118, in load_pretrained_model model = TinyLlavaLlamaForCausalLM.from_pretrained(model_base, low_cpu_mem_usage=True, config=cfg_pretrained, File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3594, in from_pretrained model = cls(config, *model_args, **model_kwargs) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/language_model/llava_llama.py", line 46, in __init__ self.model = TinyLlavaLlamaModel(config) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/language_model/llava_llama.py", line 38, in __init__ super(TinyLlavaLlamaModel, self).__init__(config) File "/home/xmj/mycipan3/smartahc_algorithm/projects/TinyLLaVABench/tinyllava/model/llava_arch.py", line 32, in __init__ super(LlavaMetaModel, self).__init__(config) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 958, in __init__ [LlamaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)] File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 958, in <listcomp> [LlamaDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)] File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 758, in __init__ self.self_attn = LLAMA_ATTENTION_CLASSES[config._attn_implementation](config=config, layer_idx=layer_idx) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 315, in __init__ self.q_proj = nn.Linear(self.hidden_size, self.num_heads * self.head_dim, bias=config.attention_bias) File "/home/xmj/anaconda3/envs/tinyllava/lib/python3.10/site-packages/transformers/configuration_utils.py", line 265, in __getattribute__ return super().__getattribute__(key) AttributeError: 'TinyLlavaPhiConfig' object has no attribute 'attention_bias'

Make sure your model name has ‘phi’ or ‘TinyLLaVA-3.1B’ in it.

[baichuanzhou]

I provided an example with lora finetuning in #23