Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker on OsX does not support GPUs acceleration #5

Open
lecciovich opened this issue Nov 17, 2023 · 1 comment
Open

Docker on OsX does not support GPUs acceleration #5

lecciovich opened this issue Nov 17, 2023 · 1 comment

Comments

@lecciovich
Copy link

As stated in Ollama website, to run containers, Docker uses a VM on Mac.
It uses Apples own hypervisor, which doesn't provide GPU access.
So, Ollama on Docker on MacOS isn't not GPU accelerated.
The menu bar app version of Ollama,anyway, is GPU accelerated on Apple Silicon.

Are there other ways to accelerate an Ollama containers on OsX?
If answer is no, wait next Apple announcement on their hypervisor at WWDC next year.

@lecciovich lecciovich changed the title Docker compose on OsX does not support GPUs acceleration Docker on OsX does not support GPUs acceleration Nov 17, 2023
@valentimarco
Copy link
Collaborator

We can resolve this issue by setting the default url of ollama to http://0.0.0.0:11434

@valentimarco valentimarco mentioned this issue Feb 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants