[Model Request] Microsoft Phi-3 mini Instruct (Faster and better then LLama 3 8B)

[sebastienbo]

⚙️ Request New Models

• Link to an existing implementation (e.g. Hugging Face/Github):

• - Default version (faster) https://huggingface.co/microsoft/Phi-3-mini-4k-instruct

• - GGUF model: https://huggingface.co/PrunaAI/Phi-3-mini-128k-instruct-GGUF-Imatrix-smashed

• Is this model architecture supported by MLC-LLM? (the list of supported models) Yes

Additional context

This model is the replacement model of Phi-2 it is better on all the benchmarks then LLama 3 8B, while beeing much smaller. It's kinda of cool that this is even possible on a phone now. I hope it will be quickly available on Android MLC chat.

Note: phi-3 mini will have a bigger brother next month called "phi-3 small" that one will even compete with the 70billion model and better then GPT 3.5

ps: android bug: when a user downloaded a model, and wants to delete the downloaded model, the android application also deletes the Model from the list. This is really bad, because users can't re-download the model anymore . It doesn't re-download the supported models. Maybe you should foresee a possibility to refresh the list of supported models that you compiled?

[Hzfengsy]

There is a WIP PR: #2222

[sebastienbo]

Oh yes I see. I've been paying with phi 3 on my pc and it is insane how good this model is. It beats gpt 3.5 on everything I throw at it. How do they do this in such a small file. ps: there was a Jenkins error: pr- merge this commit cannot be build

…

On Sun, 28 Apr 2024, 07:40 Siyuan Feng, ***@***.***> wrote: There is a WIP PR: #2222 <#2222> — Reply to this email directly, view it on GitHub <#2238 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7MPQRGARYX57A75YD7BB3Y7SDWLAVCNFSM6AAAAABG37Q432VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBRGM2DAOBTGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[MasterJH5574]

We have merged the Phi support early this week, so let's close this issue.

[sebastienbo]

How can we run phi-3 ?I don't see it in my models list.
Is there a new apk file? Or is there a refresh button to load the new list of models within the app?