New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reproducing paper results #34
Comments
Hi. Can I check which LLaMa-7B checkpoint you use? |
I am using 'yahma/llama-7b-hf' |
Have you tried the copied version of We would try that kind of checkpoint these days to see if the results are reproducible in those available checkpoints. |
With the checkpoint you specified, I could replicate the metrics. Do you know what is the difference between those 2? I thought there is one LLama and the checkpoints should be the same |
I have no idea about this😢. |
I run LLM pruner with the command specified in the ReadMe to prune LLama-7B
I get the following results
Perplexities reported in Table 1 in the paper are WikiText2 - 19.09 and PTB - 34.21. Is there any reason for the difference in thses perplexities especially PTB? Thanks
The text was updated successfully, but these errors were encountered: