New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] 请问在使用VLLM测评模型humaneval时，batch_size 不同导致测评结果有区别是为什么？ #1097

Open

1 task

noforit opened this issue Apr 26, 2024 · 0 comments

Assignees

noforit commented Apr 26, 2024

Describe the feature

在batch_size 分别为128，64，16的情况下，deepseek 1.3B 的P@1 分别是31.71、30.49、29.27
请问这是为什么？

Will you implement it?

I would like to implement this feature and create a PR!

mm-assistant bot assigned tonysy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment