You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
My own task or dataset (give details below)
Reproduction
If I load a fine-tuned BARTforConditionalGeneration model and then try to generate text with it, I run into the following error: This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (1024). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
Hi @vsocrates
Thanks for the issue !
You are getting that warning because the model's maximum positional embedding stops at 1024 tokens, some models have fixed positional embeddings where you can't exceed the maximum number of tokens by design (e.g. by having a nn.Embedding layer), for some models it is possible to exceed that, at your own risk as the model has not been trained to exceed that many tokens. If you are getting consistent / nice generations, I would say that there is nothing to worry about, otherwise you might need to use other models that support longer context length
Understood! Went through some other issues and and it looks like T5 might use relative position embeddings so in theory, should be able to extend beyond its max context length (512 tokens), but potentially with some loss of accuracy/weird generations, is that correct?
System Info
transformers
version: 4.40.2Who can help?
@ArthurZucker @younesbelkada @gante
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
If I load a fine-tuned
BARTforConditionalGeneration
model and then try to generate text with it, I run into the following error:This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (1024). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
Generation code:
I was under the impression that since the BART decoder generates autoregressively, there was no limit to its generation?
Expected behavior
Generation without a CUDA or out-of-bounds error with arbitrary length.
The text was updated successfully, but these errors were encountered: