Colab Mixtral 8x7B Offloading Demo

Welcome to the Mixtral 8x7B offloading demo repository! This project aims to demonstrate the seamless execution of Mixtral-8x7B models on Colab or consumer desktops.

Getting Started

To explore and experience the demo, follow these steps:

1. Open the demo notebook located at ./notebooks/demo.ipynb.

2. Follow the instructions within the notebook to run the Mixtral-8x7B offloading demo.

Overview

This repository provides a hands-on experience for users interested in leveraging Mixtral-8x7B models for various applications. The demo notebook serves as a comprehensive guide, offering insights into the capabilities of Mixtral-8x7B and providing a platform for experimentation.

Requirements

Make sure you have the necessary dependencies installed before running the demo. Refer to the notebook for specific requirements.

Issues and Contributions

If you encounter any issues or have suggestions for improvement, feel free to open an issue or contribute to the project. We welcome collaboration from the community to enhance this demo.

Thank you for exploring the Mixtral 8x7B offloading demo! Enjoy experimenting with the capabilities of Mixtral-8x7B in your projects.

Refrence:

1)https://arxiv.org/abs/2312.17238

Upcoming application to demonstrate!