The MixLoRA (LoRA + MoE) and its related improvements are available at mikecovlee/mlora.

[mikecovlee]

We have an actively developing fork of the official m-LoRA repository, focusing on LoRA + MoE and its related improvements, maintained by the authors of m-LoRA.
URL: https://github.com/mikecovlee/mlora

[mikecovlee]

We also provide the following features in this fork:

• Auto evaluation, including popular NLP tasks, QA tasks, etc.
• Training and evaluation of models for sequence classification tasks.
• Half precision training and inference.
• Built-in multi-LoRA inference.

These features will be added to this repository once they are stable and deemed to be of sufficient quality.