Error loading model message [Solved]

[awkwardwaiter]

When I first attempt to "Ask your knowledge" I get this error message:
Error: Error invoking remote method ‘create-session’: Error: Failed to load model

• I am using Ubuntu Linux 22.04 on older Intel i5 CPU hardware
• Running Reor from the appimage
• I downloaded my model, Google's Gemma 2B-IT.gguf, to a local directory
• Model is in .GGUF format, used Git LFS
• During setup I selected that model. No other changes from defaults
• I quit the Reor app and started again, but get same Error message

Reor app displays the model name in Settings, so I thought setup configuration worked?

What should I do?

[samlhuillier]

Thank you @awkwardwaiter for mentioning this. Could you try running the app image from terminal and sending me the logs you see? I would also recommend you use the latest version.

[awkwardwaiter]

Thanks for responding. New version solved problem.

1. My CPU-only hardware so slow my computer keeps trying to help by asking if I want to Force Quit ;)
2. The Related Note functioned right away, so vector db was fine. It was just the "Ask" function that threw error before.
   Screenshot attached
   

[samlhuillier]

Ok thank you. You are saying it is only slow when you use a local llm? The related notes sidebar is fine in terms of performance?

[awkwardwaiter]

Yes I am running on Local LLM, CPU only and my CPU is ~5 years old. That's why I selected the Gemma 2B model, I wasn't sure Mistral 7B could work.

The related notes sidebar works fine. Thanks for making this app. I've been wanting to experiment with vector db and Local LLM.

Next I'm going to see if I can connect Reor to OpenAI, if I can find my key...

[awkwardwaiter]

Update: Connecting to OpenAI works great. Screenshots attached

1. local Gemma 2B response was 2 sentences

2. GPT 3.5 Turbo response is much longer. I also tried asking out-of-notes question, Who is Santa Claus, and OpenAI responded appropriately.
   

p.s. along with switching the LLM, I moved Reor Vault to same folder as Reor appimage. Then Reor re-indexed the Notes, which worked. With that Related notes sidebar takes some time to become available, ~20 seconds on my old hardware and a few notes. Not a problem, just an observation.