Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

parallel sampling with vLLM #370

Open
VictorOdede opened this issue Oct 26, 2023 · 5 comments · May be fixed by #409
Open

parallel sampling with vLLM #370

VictorOdede opened this issue Oct 26, 2023 · 5 comments · May be fixed by #409
Assignees
@daspartho
Labels
feat/enhancement New feature or request good first issue Good for newcomers improvement SWAG

Comments

@VictorOdede
Copy link
Collaborator

https://vllm.readthedocs.io/en/latest/
@VictorOdede VictorOdede added feat/enhancement New feature or request $50 improvement SWAG good first issue Good for newcomers and removed $50 labels Oct 31, 2023
@rohinish404
Copy link

Hey, this issue seems interesting! How can i get started with it?

@daspartho daspartho changed the title Batch decoding with vLLM parallel sampling with vLLM Nov 27, 2023
@VictorOdede VictorOdede mentioned this issue Nov 27, 2023
Closed
@daspartho daspartho self-assigned this Nov 27, 2023
@daspartho daspartho linked a pull request Nov 29, 2023 that will close this issue
Draft
@kanyshai20
Copy link

Hi. Can I work on this?

@DAM8410
Copy link

DAM8410 commented Dec 12, 2023

How do I go about getting assigned, starting, working and finishing an issue so that I may get credit for it?
Thank you

@lucylililiwang
Copy link

Hi! Can I also please work on this issues? Thank you!

@lucylililiwang
Copy link

Hi, just have one question, when doing parallel sampling, should we combine the test_llm_vm.py with the parallel sampling for more efficiency? Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@daspartho daspartho

Labels
feat/enhancement New feature or request good first issue Good for newcomers improvement SWAG
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add parallel sampling using vllm daspartho/LLM-VM
Update README.md VictorOdede/LLM-VM
6 participants
@VictorOdede @DAM8410 @daspartho @kanyshai20 @rohinish404 @lucylililiwang