-
Notifications
You must be signed in to change notification settings - Fork 258
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Asking about the return_scores during generation #1661
Comments
Hello, |
What do you mean by "Ctranslate2 calculate the sum of the highest score of each step"? What is the Num equal to? |
For example bs = 1. HF score: seq_len x 1 x vocab. Otherwise, Ctranslate2 have shape: 1: (max score in vocab) of token 1 + (max score in vocab) of token 2 + ... + (max score in vocab) of token seq_len . |
Thank you! Another question, according to the autoregressive score after linear layer and chain rule, why is |
In ctranslate2, the score after each step is the log-likelihood score. That's why we do the sum. |
Oh I see! Thanks so much! |
Sorry I don't find the parameter |
Hi, I'm new to ctranslate2, and I'm confused about the scores returned by generator.generate_batch() function. What's the coresponding meaning of the scores in the huggingface generate() function?
For example,
in hf generation:
But if I use ctranslate2, for example:
I will get a list of len 3 for step_results[0].scores
And I also noticed that there is another function in hf:
which is really different from the scores in
step_results
.So I have two questions here:
generated_outputs.scores
,transition_scores
andstep_results[0].scores
?Why is that? Are there parameters I'm not aware of?
The text was updated successfully, but these errors were encountered: