Got the same error. I tried to set PYTORCH_ENABLE_MPS_FALLBACK=1, but no luck. I am running M3 macbook air.

Attempting a few more things to see what might happen, hopefully, someone from the team can respond if this is just supposed to run on Nvida / windows

+1
The same error on MacBook Pro M1

+1 on m2

+1

Attempting a few more things to see what might happen, hopefully, someone from the team can respond if this is just supposed to run on Nvida / windows

thats a negative, i actually run into the same issue when i try to run it on windows. Windows Subsystem for Linux (WSL) works fine though. one thing to check would be to see if you have the proper CPU only installation of torch. i dont think GPU support is there for mac.

I identified the code that is forcing nccl in my issue, #132 One of the first things Llama.build() does is initialize torch distributed with a hard coded nccl initialization. https://pytorch.org/docs/stable/distributed.html

+1 on M3 Pro

Feel free to correct me if I'm wrong... but I'm pretty sure there is no official support for ARM Macs (M-series chips).