Request for running Llama 2 with Nemo

[LouieBHLu]

Hi,

Is there any tutorial or example to run LLama2 with Nemo?

Thank you.

[titu1994]

The following tutorials may help with using LLama 2 models in NeMo -

Llama2 SFT - https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/llama2sft.html
Llama2 PEFT - https://docs.nvidia.com/nemo-framework/user-guide/latest/playbooks/llama2peft.html

[LouieBHLu]

Wow, thank you for your reply. This is very helpful.

[alex-ht]

Hi @titu1994
Why is it that the entire process does not require specifying a chat template?

NeMo/examples/nlp/language_modeling/tuning/conf/megatron_gpt_sft.yaml

Line 110 in 32e6302

prompt_template: "{input} {output}" # fstring to use for assistant prompt. Example: "Q: {input}\nA: {output}"

Do we need to specify the prompt_template: '[INST] {input} [/INST] {output}'?

[titu1994]

@cuichenx @arendu