Special stop tokens in stop_sequences doesn't work with GGUFs

[noisefloordev]

I'm trying to use special stop tokens in stop_sequences, but it's not recognizing them as a single token. I'm passing stop_sequence: ["<|im_start|>", "<|im_end|>"] to /api/extra/tokencount, but it doesn't stop on those tokens.

It looks like llama_tokenize is adding the BOS, and changing the third parameter of TokenizeString's llama_tokenize call to add_bos seems to fix this. I'm not sure if that's the right fix, since that parameter has some other effects.

[LostRuins]

There's a planned fix upcoming for this, this should work in the next version.

[LostRuins]

Hi, Should be fixed in the latest version