Custom CUDA Image for GPT4All GPU and CPU Support

[dpsalvatierra]

I went down the rabbit hole on trying to find ways to fully leverage the capabilities of GPT4All, specifically in terms of GPU via FastAPI/API. At this time, we only have CPU support using the tiangolo/uvicorn-gunicorn:python3.11 image and huggingface TGI image which really isn't using gpt4all.

While working with the Nvidia CUDA image, I encountered several limitations due to outdated components and lack of GLIBCXX_3.4.29 support, since the maintainer has stopped building the CudaGL images since last year - reason? on hold to improve CI/CD system : https://gitlab.com/nvidia/container-images/cudagl

To address this, I've taken the initiative to build a custom image with Nomic Vulkan support. The image is built on CUDA version 12.3.1 with Ubuntu 22.04, specifically tailored for x86_64 architecture with CUDA GL support.

### Build Command

```shell
./build.sh -d --image-name dsalvat1/cudagl --cuda-version 12.3.1 --os ubuntu --os-version 22.04 --arch x86_64 --cudagl --push
```

I've temporarily hosted this image on my Docker Hub (dsalvat1/cudagl) for testing and review. I would greatly appreciate your feedback on this solution and any suggestions for improvement.

Additionally, we should discuss the long-term plan for maintaining this image. Whether it remains hosted on my Docker Hub, wait for the maintainer to optimize the image or we transition to an official repository, I'm open to suggestions and willing to assist in its upkeep.

Finally, I am also including my branch which has the updated docker compose file and other improvements such as enabling streaming for Chat Completion.

Guess the ask is to use the branch I created called "fastapi-dev" and then merge once the maintainers are satisfied.

Todos:

• GPT4ALL keeps defaulting to CPU inference after the first successful GPU inference
• Embedding needs work
• Cleanup README file

https://github.com/dpsalvatierra/gpt4all/tree/fastapi-dev