You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Seems like the current RWKVWorldTokenizer on HuggingFace does not do truncation, even though I set truncation=True. Is this a deliberate decision? Is there some other way to do input truncation?
The text was updated successfully, but these errors were encountered:
Seems like the current RWKVWorldTokenizer on HuggingFace does not do truncation, even though I set truncation=True. Is this a deliberate decision? Is there some other way to do input truncation?
The text was updated successfully, but these errors were encountered: