Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In chat mode, add a "generate more" button when reaching the max token limit #156

Open
TheLapinMalin opened this issue Mar 3, 2024 · 1 comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@TheLapinMalin
Copy link

Is your feature request related to a problem? Please describe.
In chat mode, the model sometimes outputs the max number of tokens (512/512) but hasn't finished answering yet.

Setting num predict chat to -1 doesn't work with the model I'm using, and increasing the limit to 1024-2048 tokens just pushes the issue little bit further.

Describe the solution you'd like
It would be there a way to complete/extend the answer with additional tokens. (another batch of 512 for instance)

Describe alternatives you've considered
n/a

Additional context
n/a

@rjmacarthy rjmacarthy added the enhancement New feature or request label Mar 16, 2024
@nviraj
Copy link

nviraj commented Apr 17, 2024

First and foremost, thanks for the excellent extension! I started using it today, and I already love it.

To add to this request, I noticed that when I request to continue for long code, it creates a new code block with messed-up indentation.

For this feature, I propose taking inspiration from textgen webui's instruct mode. In that mode, pressing the 'continue' button appends to the existing code block or message, rather than starting a new one.

@rjmacarthy rjmacarthy added help wanted Extra attention is needed good first issue Good for newcomers labels May 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants