Support for MLLMs such as LLaVA?

[echo840]

❓Question

• If this is a question about the Core ML Frame work or Xcode, please ask your question in the Apple Developer Forum: https://developer.apple.com/forums/

Hello Coremltools Team,

I was just checking if It is possible for me to convert "https://github.com/haotian-liu/LLaVA/tree/main" model to support coreml?

It is an Image-to-Text model that combines a vision encoder and a large language model as a decoder.

Could you give me some guide to convert the model?

Thanks!!!!

[TobyRoseman]

Take a look at our User Guide page for converting PyTorch models.