Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic includes URLs with validation errors which aren't useful for the LLM #509

Open
mwildehahn opened this issue Mar 18, 2024 · 2 comments · May be fixed by #662
Open

Pydantic includes URLs with validation errors which aren't useful for the LLM #509

mwildehahn opened this issue Mar 18, 2024 · 2 comments · May be fixed by #662

Comments

@mwildehahn
Copy link
Contributor

mwildehahn commented Mar 18, 2024

Is your feature request related to a problem? Please describe.

Pydantic includes a URL by default to look up more information on the error which isn't useful for the LLM and also takes up token space.

1 validation error for Boop
name
  Value error, 'bla' is not a valid boop. [type=value_error, input_value='bla', input_type=str]
    For further information visit https://errors.pydantic.dev/2.6/v/value_error

Describe the solution you'd like
It'd be nice to have more control over what happens here. Pydantic let's you dump the errors if you catch the exception via errors( and then specify include_url. It feels like the default should be to not include that.

Describe alternatives you've considered
Just giving more control over the reask flow could let people solve this themselves.

@Mr-Ruben
Copy link

Totally agree.

Throwing data+garbage to the Model is not very smart/economical.

I tried to open the discussion here regarding that, but it seems like only Issues are attended (or maybe I don't know how to use Github :o)

@jxnl
Copy link
Owner

jxnl commented Mar 23, 2024

happy to take a PR

ssonal added a commit to ssonal/instructor that referenced this issue May 12, 2024
Fixes jxnl#509

Pydantic includes error URLs in ValidationErrors which means extra
tokens used in LLM calls.

Pydantic relies on an environment variable to control this setting and
is on by default. https://github.com/pydantic/pydantic-core/blob/e1fc99dd3207157aad77defc20ab6873fd268b5b/python/pydantic_core/_pydantic_core.pyi#L818

This change sets the environment variable dynamically when the package
is invoked.
@ssonal ssonal linked a pull request May 12, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants