Ollama ResponseNotRead: Attempted to access streaming response content, without having called read().

[alpiyev99]

How to fix it?

[electriquo]

i am having the same problem. for more information, that's what i did

in one terminal window, spin up ollama container

$ docker run --rm -it -e OLLAMA_DEBUG=1 -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
time=2024-03-25T11:32:37.135Z level=INFO source=images.go:806 msg="total blobs: 0"
time=2024-03-25T11:32:37.135Z level=INFO source=images.go:813 msg="total unused blobs removed: 0"
time=2024-03-25T11:32:37.135Z level=INFO source=routes.go:1110 msg="Listening on [::]:11434 (version 0.1.29)"
time=2024-03-25T11:32:37.135Z level=INFO source=payload_common.go:112 msg="Extracting dynamic libraries to /tmp/ollama3513124258/runners ..."
time=2024-03-25T11:32:38.984Z level=INFO source=payload_common.go:139 msg="Dynamic LLM libraries [cpu cuda_v11]"
time=2024-03-25T11:32:38.984Z level=DEBUG source=payload_common.go:140 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-03-25T11:32:38.984Z level=INFO source=gpu.go:77 msg="Detecting GPU type"
time=2024-03-25T11:32:38.984Z level=INFO source=gpu.go:191 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-03-25T11:32:38.984Z level=DEBUG source=gpu.go:209 msg="gpu management search paths: [/usr/local/cuda/lib64/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/x86_64-linux-gnu/libnvidia-ml.so* /usr/lib/wsl/lib/libnvidia-ml.so* /usr/lib/wsl/drivers/*/libnvidia-ml.so* /opt/cuda/lib64/libnvidia-ml.so* /usr/lib*/libnvidia-ml.so* /usr/local/lib*/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libnvidia-ml.so* /usr/lib/aarch64-linux-gnu/libnvidia-ml.so* /opt/cuda/targets/x86_64-linux/lib/stubs/libnvidia-ml.so* /usr/local/nvidia/lib/libnvidia-ml.so* /usr/local/nvidia/lib64/libnvidia-ml.so*]"
time=2024-03-25T11:32:38.984Z level=INFO source=gpu.go:237 msg="Discovered GPU libraries: []"
time=2024-03-25T11:32:38.984Z level=INFO source=cpu_common.go:18 msg="CPU does not have vector extensions"
time=2024-03-25T11:32:38.984Z level=DEBUG source=amd_linux.go:263 msg="amdgpu driver not detected /sys/module/amdgpu"
time=2024-03-25T11:32:38.984Z level=INFO source=routes.go:1133 msg="no GPU detected"

i modified the shell_gpt configuration to ollama as described in docs and installed litellm and started shell_gpt in another terminal window

$ python --version
Python 3.12.2

$ cat requirements.txt
litellm==1.24.5
shell_gpt==1.4.0

$ sgpt "Who are you?"
...
│ in content                                                                                       │
│                                                                                                  │
│    567 │   @property                                                                             │
│    568 │   def content(self) -> bytes:                                                           │
│    569 │   │   if not hasattr(self, "_content"):                                                 │
│ ❱  570 │   │   │   raise ResponseNotRead()                                                       │
│    571 │   │   return self._content                                                              │
│    572 │                                                                                         │
│    573 │   @property                                                                             │
│                                                                                                  │
│ ╭───────────── locals ──────────────╮                                                            │
│ │ self = <Response [404 Not Found]> │                                                            │
│ ╰───────────────────────────────────╯                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ResponseNotRead: Attempted to access streaming response content, without having called `read()`.

in the ollama terminal, i got also an error

[GIN] 2024/03/25 - 11:32:47 | 404 |     337.875µs |      172.17.0.1 | POST     "/api/generate"

i also tried using shell_gpt version 1.3.0 and 1.3.1 as noted in #467 but the error remains.

[dcleri]

Same problem here.
I am using ollama/llama3

this is the output from ollama serve when running sgpt prompt:

ollama serve

time=2024-04-24T11:33:09.589+02:00 level=INFO source=images.go:817 msg="total blobs: 0"
time=2024-04-24T11:33:09.589+02:00 level=INFO source=images.go:824 msg="total unused blobs removed: 0"time=2024-04-24T11:33:09.589+02:00 level=INFO source=routes.go:1143 msg="Listening on 127.0.0.1:11434 (version 0.1.32)"
time=2024-04-24T11:33:09.589+02:00 level=INFO source=payload.go:28 msg="extracting embedded files" dir=/tmp/ollama548878943/runnerstime=2024-04-24T11:33:11.096+02:00 level=INFO source=payload.go:41 msg="Dynamic LLM libraries [cuda_v11 rocm_v60002 cpu cpu_avx cpu_avx2]"
time=2024-04-24T11:33:11.096+02:00 level=INFO source=gpu.go:121 msg="Detecting GPU type"time=2024-04-24T11:33:11.096+02:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*"time=2024-04-24T11:33:11.098+02:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama548878943/runners/cuda_v11/libcudart.so.11.0]"
time=2024-04-24T11:33:11.098+02:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama548878943/runners/cuda_v11/libcudart.so.11.0: your nvidia driver is too old or missing, please upgrade to run ollama"time=2024-04-24T11:33:11.098+02:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so"time=2024-04-24T11:33:11.099+02:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: []"
time=2024-04-24T11:33:11.099+02:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"time=2024-04-24T11:33:11.099+02:00 level=INFO source=amd_linux.go:50 msg="AMD Driver: 6.3.6"
time=2024-04-24T11:33:11.100+02:00 level=INFO source=amd_linux.go:88 msg="detected amdgpu versions [gfx1100 gfx1036]"time=2024-04-24T11:33:11.101+02:00 level=INFO source=amd_linux.go:121 msg="amdgpu [0] gfx1100 is supported"
time=2024-04-24T11:33:11.101+02:00 level=WARN source=amd_linux.go:116 msg="amdgpu [1] gfx1036 is not supported by /tmp/ollama548878943/rocm [gfx1030 gfx1100 gfx1101 gfx1102 gfx900 gfx906 gfx908 gfx90a gfx940 gfx941 gfx942]"time=2024-04-24T11:33:11.101+02:00 level=WARN source=amd_linux.go:118 msg="See https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides for HSA_OVERRIDE_GFX_VERSION usage"
time=2024-04-24T11:33:11.101+02:00 level=INFO source=amd_linux.go:263 msg="[0] amdgpu totalMemory 46064M"time=2024-04-24T11:33:11.101+02:00 level=INFO source=amd_linux.go:264 msg="[0] amdgpu freeMemory  46064M"
time=2024-04-24T11:33:11.101+02:00 level=INFO source=amd_common.go:54 msg="Setting HIP_VISIBLE_DEVICES=0"
[GIN] 2024/04/24 - 11:35:03 | 404 |    1.812173ms |       127.0.0.1 | POST     "/api/generate"
[GIN] 2024/04/24 - 11:35:59 | 404 |     114.635µs |       127.0.0.1 | POST     "/api/generate"
[GIN] 2024/04/24 - 11:39:40 | 404 |     124.072µs |       127.0.0.1 | POST     "/api/generate"