New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
关于llama3base版本的评测 #3447
Comments
使用 fewshot 的 template 试一下,我的结果是 |
可以展示一下脚本文件吗 |
|
python ./src/evaluate.py |
非常感谢 |
Reminder
Reproduction
CUDA_VISIBLE_DEVICES=0,1 python ../../src/evaluate.py
--model_name_or_path /public/model/Meta-Llama-3-8B
--template llama3
--finetuning_type lora
--task /public/zzy/LLaMA-Factory/evaluation/mmlu
--split validation
--lang en
--n_shot 5
--batch_size 4
评测结果:
Average: 23.74
STEM: 24.75
Social Sciences: 22.15
Humanities: 22.97
Other: 25.51
Expected behavior
为什么只有23.74
System Info
No response
Others
No response
The text was updated successfully, but these errors were encountered: