Can't Connect Ollama With Lobe-Chat

[boudywho]

I will start by saying, I am no professional :)

I'm having trouble linking Lobe-Chat and Ollama in my Docker setup. Both containers are running (Ollama responds to curl), but I cannot connect them through the Lobe-Chat settings.

I run them both by sudo docker start lobe sudo docker start ollama

Error:

{
  "error": {
    "cause": {
      "errno": -111,
      "code": "ECONNREFUSED",
      "syscall": "connect",
      "address": "127.0.0.1",
      "port": 11434
    }
  },
  "endpoint": "http://127.0.***.1:****/",
  "provider": "ollama"
}

[boudywho]

I even tried running sudo docker run -p 11434:11434 ollama/ollama then running sudo docker run -d -p 3210:3210 -e OLLAMA_PROXY_URL=https://127.0.0.1:11434/ lobehub/lobe-chat

Still didn't work, am I missing something here?

[arvinxx]

O I know, you just run ollama with docker. you should set the Proxy URL to your ollama docker container name, maybe like http://ollama/v1 ?

[boudywho]

Unfortunately, this didn't work

[boudywho]

I even tried installing ollama locally (not in docker) and tried connecting it with Lobe, still no luck

[boudywho]

Vistting http://127.0.1:11434 shows Ollama is running and when I do curl http://127.0.1:11434 it shows also Ollama is running

[boudywho]

[ahag93]

++, I also tried this with the last docker image on this date (v0.133.1):

(base) ➜  _WORKSPACE_ curl http://127.0.0.1:11434           
Ollama is running%                                                                                                                       
(base) ➜  _WORKSPACE_ docker rm -f 92dc68ae6278
92dc68ae6278
(base) ➜  _WORKSPACE_ docker run --rm -d  --network=host -e OLLAMA_PROXY_URL=http://127.0.0.1:11434/ lobehub/lobe-chat:v0.133.1
1d8dd366962cd68dae7b41a4e53138699701888129d1e389a67e0ddcf87d88de
(base) ➜  _WORKSPACE_ docker logs -f 1d8dd366962cd68dae7b41a4e53138699701888129d1e389a67e0ddcf87d88de                          
   ▲ Next.js 14.1.3
   - Local:        http://localhost:3210
   - Network:      http://0.0.0.0:3210

 ✓ Ready in 71ms
Route: [ollama] OllamaBizError: {
  headers: {
  content-length: '18',
  content-type: 'text/plain',
  date: 'Fri, 08 Mar 2024 09:25:46 GMT'
},
  stack: 'Error: 404 404 page not found\n    at X.generate (/app/.next/server/edge-chunks/686.js:4:1745)\n    at tF.makeStatusError (/app/.next/server/edge-chunks/686.js:4:13387)\n    at tF.makeRequest (/app/.next/server/edge-chunks/686.js:4:14306)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at async W.chat (/app/.next/server/edge-chunks/173.js:1:10559)\n    at async ei (/app/.next/server/edge-chunks/173.js:1:17810)\n    at async /app/.next/server/edge-chunks/212.js:6:64195\n    at async O.execute (/app/.next/server/edge-chunks/212.js:6:61088)\n    at async O.handle (/app/.next/server/edge-chunks/212.js:6:65462)\n    at async ey.handler (/app/.next/server/edge-chunks/212.js:7:31644)',
  status: 404
}

[arvinxx]

try with:

OLLAMA_PROXY_URL=http://0.0.0.0:11434/v1

[ahag93]

Thanks, but ollama doesn't expose an endpoint with /v1:

https://github.com/ollama/ollama/blob/main/docs/api.md
https://github.com/ollama/ollama/blob/main/api/client.go

[arvinxx]

please check: https://ollama.com/blog/openai-compatibility

[forge-oscar-caballero]

Hi, I faced this situation before, if you are using docker compose to run it please ensure to use this as your host url for Ollama:

version: '3.8'
 
services:
  lobe-chat:
    image: lobehub/lobe-chat
    container_name: lobe-chat
    restart: always
    ports:
      - '3210:3210'
    environment:
      OPENAI_API_KEY: sk-....
      OPENAI_PROXY_URL: https://api.openai.com/v1
      # ACCESS_CODE: lobe66
      OLLAMA_PROXY_URL: http://host.docker.internal:11434/v1

Let me know if this works for you.

[ahag93]

Hi, Thank you for this information, indeed there is this doc Openai compatibility that I have not seen, after testing the compose, I have not had convincing results so far:

I don't know if you paid any attention to this note for the doc:
OpenAI compatibility

Note: OpenAI compatibility is experimental and is subject to major adjustments including breaking changes. For fully-featured access to the Ollama API, see the Ollama Python library, JavaScript library and REST API.

thanks

[arvinxx]

I think it's fixed with 0.149.0. If there is any other issue, please let me know

[walk4rever]

No, the problem is still there.

[arvinxx]

@walk4rever what's the problem?