Why can llama understand Chinese so much and can't reply directly in Chinese?

[XksA-me]

Why can't llama understand Chinese so much and can't reply directly in Chinese?

I tested Llama-2-7b-chat-hf again today.

Test using GPU platform: matpool.com
Memory usage:
Open 8BIT occupies 8G+, GPU utilization: 13-20%
If 8BIT is not enabled, it takes up 14G+, GPU utilization: 55-70%

[liltom-eth]

I believe this happened because of the system prompt. this is added to your chat text sending to llm. Try delete system prompt in UI or use Chinese system prompt.
The results might be better.

Lllama2 I believe not well supported on Chinese now since Chinese have more tokens.

I list a feature to support some Chinese llama2 models. Haven't got a chance to test it.

[XksA-me]

thanks for the reply. Useful, but limited.


Hope I can contribute for feature.

[liltom-eth]

@XksA-me do you want to contribute to this feature? Might need take a look at this repo Chinese-Llama-2-7b.

[liltom-eth]

@XksA-me welcome contributing your benchmark performance here.