Skip to content

Issues: triton-inference-server/tensorrtllm_backend

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

199 Open 189 Closed
199 Open 189 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Deepseek model streaming mode with Chinese character �? bug Something isn't working
#493 opened Jun 7, 2024 by activezhao
2 of 4 tasks
4
2x docker image size increase for trtllm: from 8.38 GB (24.03) to 18.46 GB (24.04) bug Something isn't working triaged Issue has been triaged by maintainers
#489 opened Jun 3, 2024 by lopuhin
2 of 4 tasks
@byshiue @ncomly-nvidia @AdamzNV
2
Grafana Dashboard (Feature Request) feature request New feature or request
#485 opened May 31, 2024 by hestabit-dev
4 tasks
@ncomly-nvidia @AdamzNV
missing nv_trt_llm_request_metrics from python backend bug Something isn't working
#479 opened May 28, 2024 by Hao-YunDeng
2 of 4 tasks
[Question] Best practises to track inputs and predictions? question Further information is requested triaged Issue has been triaged by maintainers
#475 opened May 24, 2024 by FernandoDorado
@byshiue
2
tensorrt_llm_bls disregards top_k / temperature setting bug Something isn't working
#472 opened May 23, 2024 by janpetrov
1 of 4 tasks
random_seed seems to be ignored (or at least inconsistent) for inflight_batcher_llm bug Something isn't working triaged Issue has been triaged by maintainers
#468 opened May 21, 2024 by dyoshida-continua
2 of 4 tasks
@byshiue
4
unexpected error when creating modelInstanceState: [json.exception.out_of_range.403] key 'name' not found bug Something isn't working triaged Issue has been triaged by maintainers
#467 opened May 21, 2024 by Godlovecui
2 of 4 tasks
@byshiue
4
[Bug] Zero temperature curl request affects non-zero temperature requests bug Something isn't working triaged Issue has been triaged by maintainers
#464 opened May 20, 2024 by Hao-YunDeng
2 of 4 tasks
@byshiue
3
Can you provide an example of a visual language model or multimodal model launch by triton server? triaged Issue has been triaged by maintainers
#463 opened May 20, 2024 by lzcchl
@byshiue
1
How to deploy one model instance across multiple GPUs to tackle the OOM problem? question Further information is requested triaged Issue has been triaged by maintainers
#462 opened May 16, 2024 by shil3754
@byshiue
1
decoding_mode top_k_top_p does not take effect for llama2 not same with huggingface triaged Issue has been triaged by maintainers
#461 opened May 16, 2024 by yjjiang11
1 of 4 tasks
@byshiue
3
Implement XC-Cache to improve long context inference performance
#460 opened May 15, 2024 by avianion
1
Tritonserver won't start up running Smaug 34b bug Something isn't working triaged Issue has been triaged by maintainers
#459 opened May 15, 2024 by workuser12345
2 of 4 tasks
@byshiue
1
two seemingly identical functions in the same file triaged Issue has been triaged by maintainers
#458 opened May 15, 2024 by dongluw
@byshiue
1
Mixtral 8x7-v0.1 Hangs after serving a few requests bug Something isn't working
#457 opened May 15, 2024 by aaditya-srivathsan
2 of 4 tasks
3
[tensorrt-llm backend] A question about launch_triton_server.py question Further information is requested triaged Issue has been triaged by maintainers
#455 opened May 13, 2024 by victorsoda
@byshiue
1
Example gpu_device_ids for multi-model usage? question Further information is requested
#448 opened May 9, 2024 by vnkc1
2 of 4 tasks
@byshiue @schetlur-nv
1
InFlightBatching seems not working need more info triaged Issue has been triaged by maintainers
#442 opened May 6, 2024 by larme
2 of 4 tasks
@byshiue
3
Deployement failed for BERT triaged Issue has been triaged by maintainers
#440 opened May 3, 2024 by vivekjoshi556
@byshiue
1
Deploying Mixtral-8x7B-v0.1 with Triton 24.02 on A100 (160GB) raises "Cuda Runtime (out of memory)" exception bug Something isn't working triaged Issue has been triaged by maintainers
#438 opened Apr 29, 2024 by kelkarn
2 of 4 tasks
@byshiue
2
GptManager’s scalability issues with input & output parameters feature request New feature or request
#437 opened Apr 28, 2024 by service-kit
@ncomly-nvidia @AdamzNV
1
Encountered an error in forward function: std::bad_cast bug Something isn't working
#435 opened Apr 26, 2024 by wangqy1216
1 of 4 tasks
@schetlur-nv
1
max_batch_size seems to have no impact on model performance bug Something isn't working triaged Issue has been triaged by maintainers
#429 opened Apr 23, 2024 by VitalyPetrov
3 of 4 tasks
@byshiue @kaiyux
8
Performance Issue with return_context_logits Enabled in TensorRT-LLM bug Something isn't working
#428 opened Apr 23, 2024 by gywlssww
2 of 4 tasks
ProTip! Updated in the last three days: updated:>2024-06-05.