You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Traceback (most recent call last):
File "/home/arda/kai/webui/text-generation-webui/modules/callbacks.py", line 61, in gentask
ret = self.mfunc(callback=_callback, *args, **self.kwargs)
File "/home/arda/kai/webui/text-generation-webui/modules/text_generation.py", line 375, in generate_with_callback
shared.model.generate(**kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/transformers/generation/utils.py", line 1575, in generate
result = self._sample(
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/transformers/generation/utils.py", line 2697, in _sample
outputs = self(
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/transformers/models/mistral/modeling_mistral.py", line 1157, in forward
outputs = self.model(
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/ipex_llm/transformers/models/mistral.py", line 148, in mistral_model_forward_4_36
return MistralModel.forward(
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/transformers/models/mistral/modeling_mistral.py", line 1042, in forward
layer_outputs = decoder_layer(
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/transformers/models/mistral/modeling_mistral.py", line 757, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/ipex_llm/transformers/models/mistral.py", line 528, in mistral_attention_forward_4_36
return forward_function(
File "/opt/anaconda3/envs/text-webui-upstream/lib/python3.9/site-packages/ipex_llm/transformers/models/mistral.py", line 806, in mistral_attention_forward_4_36_original
past_key_value.seen_tokens += key_states.shape[-2]
AttributeError: can't set attribute
Can siucceed for 4.38.2
@jason-dai@shane-huang Seems the upstream is continuously upgrading transformers and our implementation may fail for most models...
The text was updated successfully, but these errors were encountered:
This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.
Error for 4.39.3
Can siucceed for 4.38.2
@jason-dai @shane-huang Seems the upstream is continuously upgrading transformers and our implementation may fail for most models...
The text was updated successfully, but these errors were encountered: