You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
E0517 12:18:34.314931 164 model_lifecycle.cc:638] failed to load 'llama3_tensorrt_llm' version 1: Invalid argument: unable to find backend library for backend 'tensorrtllm', try specifying runtime on the mode configuration.
models with python, onnx loads correct.
How i can combine docker image for using both backends?
Im trying https://github.com/triton-inference-server/server/blob/main/docs/customization_guide/compose.md
to build onnx+python+tensorrtllm backends.
as mention in doc i do
and it builds, but when i start triton server.
E0517 12:18:34.314931 164 model_lifecycle.cc:638] failed to load 'llama3_tensorrt_llm' version 1: Invalid argument: unable to find backend library for backend 'tensorrtllm', try specifying runtime on the mode configuration.
models with python, onnx loads correct.
How i can combine docker image for using both backends?
failed, tensorrt not found
The text was updated successfully, but these errors were encountered: