Pytriton with Gradio

A simple example of using pytriton inference server with Gradio.

Requirements

pip install  -r ./requirements.base.txt

dvc pull
dvc repro

Run pytriton inference server:

docker compose up --build -d

Check status. It must be (healthy).

docker compose ps

If you want to use GPU uncomment deploy section in docker compose and replace in the Dockerfile:

--mount=type=bind,source=./requirements.server.cpu.txt,target=./requirements.txt \

to

--mount=type=bind,source=./requirements.server.gpu.txt,target=./requirements.txt \

Run gradio app:

pip install -r ./requirements.client.txt

python ./client.py

Open UI by link.

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
.dvc		.dvc
models		models
.editorconfig		.editorconfig
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
client.py		client.py
convert_models.py		convert_models.py
docker-compose.yaml		docker-compose.yaml
dvc.lock		dvc.lock
dvc.yaml		dvc.yaml
query.json		query.json
requirements.base.txt		requirements.base.txt
requirements.client.txt		requirements.client.txt
requirements.server.base.txt		requirements.server.base.txt
requirements.server.cpu.txt		requirements.server.cpu.txt
requirements.server.gpu.txt		requirements.server.gpu.txt
server.py		server.py