You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I'm testing codellama, and I would like a guide on how to enable it to accept the sending of 100K input tokens. From what I understand, this is done by adjusting the max_seq_len and max_batch_size parameters, but I couldn't find more information on this. Please, if someone could guide me.
As an extra question, do you know what the default size it would accept is?
Thank you, and I will appreciate any help.
The text was updated successfully, but these errors were encountered:
Hi, I'm testing codellama, and I would like a guide on how to enable it to accept the sending of 100K input tokens. From what I understand, this is done by adjusting the
max_seq_lenandmax_batch_sizeparameters, but I couldn't find more information on this. Please, if someone could guide me.As an extra question, do you know what the default size it would accept is?
Thank you, and I will appreciate any help.
The text was updated successfully, but these errors were encountered: