FIM/Infill changes vs. model support & GGUF regeneration etc.?

[ghchris2021]

I noticed the recent release notes about it and the following commit activity wrt. FIM / Infill and supporting more models e.g. CodeGemma having that capability but having various vocabulary symbols required to solicit it i.e.:
#6689
#6626

I haven't scrutinized the details but I have a couple questions:

Codellama (and I suppose its close derivative relations) was AFAICT the nominally originally supported model for
FIM. Now CodeGemma has been mentioned wrt. improving support as above.

Is there a summary of status of which models are presently supported for the FIM use case given the recent changes?
Obviously Codellama and codegemma. I'm not sure what prominent others may be "equivalent" to those because
of being derivatives / using the same vocabulary for the function.

And then I'm less sure about others e.g. the deepseek-coder variant below which mentions in its model card that
it was trained from scratch so I gather it may not be detected as compatible with the above?

https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base

Also after b2680 changed the gguf vocabulary is my conception correct that if I've previously downloaded other models
which might newly now support the FIM feature that I'd have to obtain re-converted GGUF models encoded using the new
codebase post b2680 in order to use the FIM (as opposed to any runtime rebuild / configuration tweaks + older GGUFs)?

Thanks! It's nice to see codegemma etc. getting support for this, it was on my list to try for IDE use.

Context:

b2680

gguf : add special tokens metadata for FIM/Infill (#6689)

This commit adds special token metadata for Fill-In-the-Middle
(FIM)/Infill to the GGUF model.

The motivation for this is that currently there is support for CodeLlama
but other models exist now like CodeGemma, but the different models use
different token ids for the special tokens and this commit allows for
supporting multiple models.

[ggerganov]

Is there a summary of status of which models are presently supported for the FIM use case given the recent changes?

All models that during conversion write special FIM tokens to the GGUF header should be supported:

llama.cpp/convert-hf-to-gguf.py

Lines 1306 to 1312 in 599ce84

	special_vocab = gguf.SpecialVocab(self.dir_model, load_merges=False,
	special_token_types = ['prefix', 'suffix', 'middle', 'eot'])
	special_vocab._set_special_token("prefix", 32007)
	special_vocab._set_special_token("suffix", 32008)
	special_vocab._set_special_token("middle", 32009)
	special_vocab._set_special_token("eot", 32010)
	special_vocab.add_to_gguf(self.gguf_writer)

llama.cpp/llama.cpp

Lines 4278 to 4281 in 599ce84

	{ LLM_KV_TOKENIZER_PREFIX_ID, vocab.special_prefix_id },
	{ LLM_KV_TOKENIZER_SUFFIX_ID, vocab.special_suffix_id },
	{ LLM_KV_TOKENIZER_MIDDLE_ID, vocab.special_middle_id },
	{ LLM_KV_TOKENIZER_EOT_ID, vocab.special_eot_id },

If support for a new model is needed, the convert scripts need to be updated accrodingly.

Deepseek models are still WIP: #5981