Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Insufficient error handling for the ollama backend #81

Open
Vorlent opened this issue Mar 4, 2024 · 1 comment
Open

Insufficient error handling for the ollama backend #81

Vorlent opened this issue Mar 4, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@Vorlent
Copy link

Vorlent commented Mar 4, 2024

The backend works, but it is non-obvious how to get it working. There are two major problems with the error handling for the ollama backend.

  1. CORS prevents the application from talking with ollama
  2. The ollama model tag could not be found

These two error cases need a dedicated error message each.

The error handling should detect potential CORS issues if possible and tell the user to set the set the OLLAMA_ORIGINS env variable. It is also not obvious what the CORS values should be when you are running this locally. I couldn't get it working.
The error handling should also detect the case that the model does not exist or has not yet been downloaded. If possible, you could query ollama for a list of already downloaded models and show them in a dropdown with typeahead.

In general, the menu for setting up the ollama should be able to check ollama without the user having to leave the menu and entering a message for the chatbot. Perhaps you can put a button "Test ollama connection" button or something similar.

I don't want to give the impression that this project only has flaws by the way. Overall I am impressed with this project and find amica to be extremely enjoyable. When I create issues and point out bugs, it is because I am satisfied with everything except the bug in question. The ollama backend works, but my hardware is a bit slow when it comes to prompt ingestion (processing the initial prompt). Generating tokens even at a measily 7 tokens/s locally is fast enough to feed the text to speech engine on amica.arbius.ai.

@slowsynapse
Copy link
Collaborator

Nice idea. We could add the CORS issue detect, as CORS is a common annoying problem for a lot of debugging.

I also like the idea of expanding the olamma menu to directly display and change models.

@slowsynapse slowsynapse added the enhancement New feature or request label May 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants