Efficient global optimization toolbox in Rust: bayesian optimization, mixture of gaussian processes, sampling methods
-
Updated
Jun 7, 2024 - Rust
Efficient global optimization toolbox in Rust: bayesian optimization, mixture of gaussian processes, sampling methods
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
pytorch implementation of grok
Surrogate Modeling Toolbox
Tutel MoE: An Optimized Mixture-of-Experts Implementation
[ICLR 2024 Spotlight] Code for the paper "Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy"
Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
PyTorch library for cost-effective, fast and easy serving of MoE models.
This is the official repository of the papers "Parameter-Efficient Transfer Learning of Audio Spectrogram Transformers" and "Efficient Fine-tuning of Audio Spectrogram Transformers via Soft Mixture of Adapters".
[Preprint] Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models
A library for easily merging multiple LLM experts, and efficiently train the merged LLM.
Repository for our paper "See More Details: Efficient Image Super-Resolution by Experts Mining"
MoE Decoder Transformer implementation with MLX
[arXiv'24] Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization
The idea to create the perfect LLM currently possible came to my mind because I was watching a YouTube on GaLore, the "sequel" to LoRa, and I realized how fucking groundbreaking that tech is. I was daydreaming about pretraining my own model, this (probably impossible to implement) concept is a refined version of that model.
[SIGIR'24] The official implementation code of MOELoRA.
[Paper][Preprint 2024] Mixture of Modality Knowledge Experts for Robust Multi-modal Knowledge Graph Completion
an LLM toolkit
Mistral and Mixtral (MoE) from scratch
Add a description, image, and links to the mixture-of-experts topic page so that developers can more easily learn about it.
To associate your repository with the mixture-of-experts topic, visit your repo's landing page and select "manage topics."