llm-webui

Is a simple, fun project to run your own LLM chat using llama.cpp.

How to run

Clone the repo: https://github.com/robjsliwa/llama-cpp-python for Python bindings for llama.cpp
If you want to get latest version of llama.cpp, go to vendor folder and run git clone https://github.com/ggerganov/llama.cpp in there or update the hash whatever is easier for you.
Build docker image: docker build -t llama-server .
Run it with: docker run --rm -it -p 8000:8000 -v /home/data/datasets/wizard-vicuna:/models -e MODEL=/models/Wizard-Vicuna-13B-Uncensored.ggml.q8_0.bin llama-server
Install web ui with: npm install
Start web ui with: npm start

Note: You can find great models on Hugging Face here: https://huggingface.co/TheBloke/Wizard-Vicuna-13B-Uncensored-GGML/tree/main

Name		Name	Last commit message	Last commit date
Latest commit History 8 Commits
public		public
src		src
.gitignore		.gitignore
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
postcss.config.js		postcss.config.js
tailwind.config.js		tailwind.config.js
tsconfig.json		tsconfig.json