New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Add huggingface apply_chat_template #1098
[Feature] Add huggingface apply_chat_template #1098
Conversation
"""PPL Inferencer.""" | ||
# flake8: noqa | ||
# yapf: disable | ||
"""LL Inferencer.""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"""LL Inferencer.""" | |
"""LogLikelihood(LL) Inferencer.""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update the readme with more information about new HF classes, and give instructions on the --accelarator
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Motivation
Remove a large amount of redundant information from the HuggingFace model config or information that can be obtained from the HF cache, including but not limited to
chat_template
,max_seq_len
, and so on.Modification
Two new classes,
HuggingFaceBaseModel
andHuggingFacewithChatTemplate
are introduced inopencompass/models/huggingface_above_v4_33.py
.Most of the model configs are rewritten.
BC-breaking (Optional)
Passing
--accelerator
via cli will become not usable.