Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong validations on Parameters in TGI python library #1913

Open
3 of 4 tasks
Jason-CKY opened this issue May 17, 2024 · 0 comments
Open
3 of 4 tasks

Wrong validations on Parameters in TGI python library #1913

Jason-CKY opened this issue May 17, 2024 · 0 comments

Comments

@Jason-CKY
Copy link
Contributor

Jason-CKY commented May 17, 2024

System Info

NA

Information

  • Docker
  • The CLI directly

Tasks

  • An officially supported command
  • My own modifications

Reproduction

Frequency penalty is validated to be strictly positive in the python library type validation, but is validated to be in the range [-2.0, 2.0] in the actual webserver. This means that i can send a REST API request to TGI with frequency penalty < 0 but will run into validation error when doing it via the text generation client

from text_generation import Client

endpoint_url = "https://YOUR_ENDPOINT.endpoints.huggingface.cloud"

client = Client(endpoint_url)
text = client.generate("Why is the sky blue?", frequency_penalty=-1.0).generated_text
print(text)
# ' Rayleigh scattering'

Expected behavior

No validation errors for frequency_penalty in the range [-2.0, 2.0]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant