Skip to content

Issues: xorbitsai/inference

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

257 Open 455 Closed
257 Open 455 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Support spare embedding of bge-m3
#1471 opened May 10, 2024 by yefangok v0.11.0
Request Limits参数对stream=true情况下无法限制并发数 question Further information is requested
#1467 opened May 10, 2024 by gggauss v0.11.0
如何控制大模型返回结果？ question Further information is requested
#1465 opened May 9, 2024 by yinghaodang v0.11.0
The actor is dead because its worker process has died. Worker exit type: SYSTEM_ERROR Worker exit detail: Worker unexpectedly exits with a connection error code 2. End of file. There are some potential root causes. (1) The process is killed by SIGKILL by OOM killer due to high memory usage. (2) ray stop --force is called. (3) The worker is crashed unexpectedly due to SIGSEGV or other unexpected errors.
#1464 opened May 9, 2024 by insistence-essenn v0.11.0
本地注册加载ChatGLM3-6B模型，回答异常 gpu question Further information is requested
#1463 opened May 9, 2024 by LASTLINEK v0.11.0
What's the path to download the model? question Further information is requested
#1459 opened May 9, 2024 by ciaoyizhen v0.11.0
Have anyone sucessed install from source code?
#1455 opened May 8, 2024 by sunhaha123 v0.11.0
1
How should I enable vllm distributed inference on multiple single GPU nodes? gpu question Further information is requested
#1452 opened May 8, 2024 by Jie2GG v0.11.0
EHN: the model list which support tool_calls should get from BUILTIN_LLM_MODEL_TOOL_CALL_FAMILIES
#1446 opened May 8, 2024 by mikeshi80 v0.11.0
FEAT: support deepseek v2 chat and coder feature
#1440 opened May 7, 2024 by GXKIM v0.11.0
3
CUDA call failed lazily at initialization with error: device >= 0 && device < num_gpus INTERNAL ASSERT FAILED at gpu
#1439 opened May 7, 2024 by wangyongpenga v0.11.0
3
start a image model and terminate it， then restart it will hit {"detail": "Not Found"} error when open it
#1438 opened May 7, 2024 by GodfreyGuo v0.11.0
ENH: Support stream_options for OpenAI create chat completion enhancement New feature or request
#1433 opened May 7, 2024 by qinxuye v0.11.0
[BUG] The fetch request of the rerank model cannot be handled correctly bug Something isn't working
#1432 opened May 7, 2024 by wongdi v0.11.0
6
服务Docker重启之后，如何恢复之前运行的模型 question Further information is requested
#1431 opened May 7, 2024 by yeyupiaoling v0.11.0
1
模型不支持 Model not supported, name: fuzimingcha_v1.0, format: pytorch, size: 6, quantization: 8-bit question Further information is requested
#1430 opened May 6, 2024 by zjx140 v0.11.0
2
OSError: [WinError 126] 找不到指定的模块。 Error loading "D:\anaconda\Lib\site-packages\torch\lib\shm.dll" or one of its dependencies.
#1428 opened May 6, 2024 by zhaoyuchen1128 v0.11.0
Unhandled exceptions of restful api lead to server hang gpu
#1424 opened May 3, 2024 by frostyplanet v0.11.0
I saw the configuration file of Yi-9b in the source code, but I can't use it.
#1421 opened May 2, 2024 by moyudaduizhang v0.11.0
1
Qwen1.5-chat 72B int4 4卡(V100) 推理过程中token数到10k会报错OOM question Further information is requested
#1419 opened Apr 30, 2024 by EthanD4869 v0.11.0
1
QUESTION support pipeline parallel inference question Further information is requested
#1418 opened Apr 30, 2024 by zhengxingmao v0.11.0
1
加载自定义模型 qwen1.5 0.5B chat
#1415 opened Apr 30, 2024 by GXKIM v0.11.0
4
FEAT: support modifying registered custom model configs feature
#1411 opened Apr 29, 2024 by ConleyKong v0.11.0
1
Chatglm3模型聊天Response中token消耗为-1
#1407 opened Apr 29, 2024 by dengpan1 v0.11.0
2
NCCL Error when replica=2 gpu
#1404 opened Apr 29, 2024 by yoke233 v0.11.0
2
ProTip! Exclude everything labeled bug with -label:bug.