You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Additional context
I think there's minor concerns regarding the fact that Phi 3 has its MLP and QKV fused... This can be discussed in my PR I raised in detail
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Extend AutoGPTQ support for Microsoft's recently released Phi-3 models
Describe the solution you'd like
I have a PR ready and tested on my end to allow Phi-3-mini variants to be quantized using AutoGPTQ library.
Please see my PR here: #651
Additional context
I think there's minor concerns regarding the fact that Phi 3 has its MLP and QKV fused... This can be discussed in my PR I raised in detail
The text was updated successfully, but these errors were encountered: