Is it possible to load LLM2Vec config?

[xiaoyuqian2]

Is it possible to load LLM2Vec encoder model config without loading the weights, like below?

config = AutoConfig.from_pretrained("McGill-NLP/LLM2Vec-Mistral-7B-Instruct-v2-mntp")
model = AutoModel.from_config(config)

This sounds like an uncommon request. But what I want to achieve is:

• load LLM2Vec model config
• download all weights needed (including the base model weights and the adapter weights) explicitly to a folder llm2vec_weights
• initialize the model later by running AutoModel.from_pretrained(llm2vec_weights, trust_remote_code=True).

Is it possible? If so, what are the weights I should download (e.g. for LLM2Vec encoder model with llama2-7b-chat as base)? Thank you!

[vaibhavad]

Hi @xiaoyuqian2,

Apologies for the late response. Do you want this setup to run the models offline? In that case, the steps detailed here will be an easier and simpler case.

If you have a different use case, then let me know and I will try to come up with steps to load the model following the constrains mentioned.

[vaibhavad]

Closing as it is stale. Feel free to re-open if you have any further questions.