Skip to content

Issues: deepjavalibrary/djl-serving

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

38 Open 61 Closed
38 Open 61 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Error running multimodel endpoints in sagemaker bug Something isn't working
#1911 opened May 15, 2024 by Najib-Haq
document the /invocations endpoint bug Something isn't working
#1905 opened May 14, 2024 by tenpura-shrimp
Better support prometheus metrics and/or allow custom prometheus metrics enhancement New feature or request
#1827 opened Apr 27, 2024 by glennq
DJL-TensorRT-LLM Bug: TypeError: Got unsupported ScalarType BFloat16 bug Something isn't working
#1816 opened Apr 25, 2024 by rileyhun
3
DJL-TRTLLM: Error while detokenizing output response of teknium/OpenHermes-2.5-Mistral-7B on Sagemaker bug Something isn't working
#1792 opened Apr 20, 2024 by omarelshehy
1
question to error model conversion process failed bug Something isn't working
#1785 opened Apr 17, 2024 by geraldstanje
Token accuracy not as expected for starcoderbase-15b model with rolling batch type vllm bug Something isn't working
#1720 opened Apr 2, 2024 by sreka
snap installer for djlbench doesn't work for arm64 platform bug Something isn't working
#1532 opened Feb 6, 2024 by snadampal
Plan to use Attention Sinks? enhancement New feature or request
#1470 opened Jan 10, 2024 by spring1915
1
add support for nacos of spring cloud alibaba enhancement New feature or request
#1436 opened Dec 31, 2023 by litongjava
Streaming with rolling batch for starcoderbase model not working bug Something isn't working
#1352 opened Nov 30, 2023 by prgawade
3
Deep learning engine not found: MPI Error bug Something isn't working
#1351 opened Nov 30, 2023 by prgawade
3
Appropriate whitespace missing in streaming output for Llama2, Mistral models bug Something isn't working
#1272 opened Nov 5, 2023 by Najib-Haq
4
Issue with serving modes section (documentation) bug Something isn't working
#1236 opened Oct 30, 2023 by segundovolante
3
Add benchmark results or link to key features section enhancement New feature or request
#1234 opened Oct 28, 2023 by segundovolante
Ability to transform model outputs in DJL Serving enhancement New feature or request
#1214 opened Oct 23, 2023 by rachitchauhan43
3
djl lmi images with vllm and hf quantizaton support bug Something isn't working
#1188 opened Oct 16, 2023 by Nagarajj
rolling batch does not work bug Something isn't working
#1173 opened Oct 10, 2023 by prgawade
1
Does deepspeed engine support Llama2 question-answering task? bug Something isn't working
#1106 opened Sep 25, 2023 by YunTaoYoung
1
tensor parallelism across multiple GPU's
#1092 opened Sep 18, 2023 by samanthvishwas
1
What backend does djl-serving default to for rolling_batch? bug Something isn't working
#1075 opened Sep 12, 2023 by yudhiesh
3
How to use inference of multiple models
#1042 opened Aug 30, 2023 by polarisunny
5
Listener timed out in: 30.0 s bug Something isn't working
#950 opened Jul 13, 2023 by yuxin7
3
Stochastic algorithm PR testing behaves different from local testing, for a fixed random seed. bug Something isn't working
#913 opened Jul 6, 2023 by KexinFeng
Are there any plans to expose the skip_special_tokens parameter in the future, driven by serving configuration?
#900 opened Jul 5, 2023 by extraleo
1
ProTip! Exclude everything labeled bug with -label:bug.