Update pydantic code to fix warnings #3600

msabramo · 2024-05-12T17:12:38Z

Before (pydantic==2.7.1, pydantic_core==2.18.2):

$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
========================== 10 passed, 2 skipped, 48 warnings in 9.83s ===========================

After (pydantic==2.7.1, pydantic_core==2.18.2):

$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
=========================== 10 passed, 2 skipped, 1 warning in 9.89s ============================

After (pydantic==1.10.15, pydantic_core not installed):

$ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py
...
=========================== 10 passed, 2 skipped, 1 warning in 9.13s ============================

Note that the 1 remaining warning is fixed by #3602.

Before: ```shell $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ====================================== test session starts ====================================== platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0 rootdir: /Users/abramowi/Code/OpenSource/litellm plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0 asyncio: mode=Mode.STRICT collected 12 items litellm/tests/test_proxy_server.py s..........s [100%] ======================================= warnings summary ======================================== ../../../Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: 25 warnings /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/pydantic/_internal/_config.py:284: PydanticDeprecatedSince20: Support for class-based `config` is deprecated, use ConfigDict instead. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ warnings.warn(DEPRECATION_MESSAGE, DeprecationWarning) litellm/proxy/_types.py:225 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:312 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:312: PydanticDeprecatedSince20: `pydantic.config.Extra` is deprecated, use literal values instead (e.g. `extra='allow'`). Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ extra = Extra.allow # Allow extra fields litellm/proxy/_types.py:315 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:315: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:344 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:344: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:391 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:391: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:460 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:460: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:480 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:480: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:493 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:493: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:538 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:538: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:575 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:575: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:870 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:870: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:897 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:897: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/proxy/_types.py:918 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:918: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) litellm/utils.py:43 /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources # type: ignore litellm/tests/test_proxy_server.py::test_bedrock_embedding litellm/tests/test_proxy_server.py::test_chat_completion litellm/tests/test_proxy_server.py::test_chat_completion_azure litellm/tests/test_proxy_server.py::test_chat_completion_optional_params litellm/tests/test_proxy_server.py::test_embedding litellm/tests/test_proxy_server.py::test_engines_model_chat_completions litellm/tests/test_proxy_server.py::test_health litellm/tests/test_proxy_server.py::test_img_gen litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead. warnings.warn(message, DeprecationWarning) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ========================== 10 passed, 2 skipped, 48 warnings in 9.83s =========================== ``` After: ```shell $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ====================================== test session starts ====================================== platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0 rootdir: /Users/abramowi/Code/OpenSource/litellm plugins: anyio-4.3.0, asyncio-0.23.6, mock-3.14.0 asyncio: mode=Mode.STRICT collected 12 items litellm/tests/test_proxy_server.py s..........s [100%] ======================================= warnings summary ======================================== litellm/utils.py:43 /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources # type: ignore litellm/tests/test_proxy_server.py::test_bedrock_embedding litellm/tests/test_proxy_server.py::test_chat_completion litellm/tests/test_proxy_server.py::test_chat_completion_azure litellm/tests/test_proxy_server.py::test_chat_completion_optional_params litellm/tests/test_proxy_server.py::test_embedding litellm/tests/test_proxy_server.py::test_engines_model_chat_completions litellm/tests/test_proxy_server.py::test_health litellm/tests/test_proxy_server.py::test_img_gen litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure /Users/abramowi/Library/Caches/pypoetry/virtualenvs/litellm-Fe7WjZrx-py3.12/lib/python3.12/site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead. warnings.warn(message, DeprecationWarning) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ========================== 10 passed, 2 skipped, 10 warnings in 9.62s =========================== ```

vercel · 2024-05-12T17:12:43Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 13, 2024 11:51pm

because I made a lot of fairly mindless changes to pydantic code to fix warnings and I don't want git blame to give people the impression that I know more about this code than I do.

The httpx package emits some warnings when we run the tests in `test_proxy_server.py`: ```shell $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ... litellm/tests/test_proxy_server.py::test_bedrock_embedding litellm/tests/test_proxy_server.py::test_chat_completion litellm/tests/test_proxy_server.py::test_chat_completion_azure litellm/tests/test_proxy_server.py::test_chat_completion_optional_params litellm/tests/test_proxy_server.py::test_embedding litellm/tests/test_proxy_server.py::test_engines_model_chat_completions litellm/tests/test_proxy_server.py::test_health litellm/tests/test_proxy_server.py::test_img_gen litellm/tests/test_proxy_server.py::test_openai_deployments_model_chat_completions_azure .../site-packages/httpx/_client.py:680: DeprecationWarning: The 'app' shortcut is now deprecated. Use the explicit style 'transport=WSGITransport(app=...)' instead. warnings.warn(message, DeprecationWarning) ... ``` Fixing this requires updating starlette, but the version of FastAPI in use prevents that. So I updated FastAPI to update starlette to fix the httpx warnings. When run in combination with the changes in BerriAIGH-3600, the tests pass and only emit 1 warning! ```shell $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ====================================== test session starts ====================================== platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0 rootdir: /Users/abramowi/Code/OpenSource/litellm plugins: anyio-4.3.0, mock-3.14.0 collected 12 items litellm/tests/test_proxy_server.py s..........s [100%] ======================================= warnings summary ======================================== litellm/utils.py:43 /Users/abramowi/Code/OpenSource/litellm/litellm/utils.py:43: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources # type: ignore -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== 10 passed, 2 skipped, 1 warning in 8.81s ============================ ```

by trying to use `importlib.resources` first and falling back to `pkg_resources` if that fails. With this and the changes in BerriAIGH-3600 and BerriAIGH-3601, the tests pass with **zero warnings**!! 🎉 🎉 ```shell abramowi at marcs-mbp-3 in ~/Code/OpenSource/litellm (msabramo/fix-pydantic-warnings●●) $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ====================================== test session starts ====================================== platform darwin -- Python 3.12.3, pytest-7.4.4, pluggy-1.5.0 rootdir: /Users/abramowi/Code/OpenSource/litellm plugins: anyio-4.3.0, mock-3.14.0 collected 12 items litellm/tests/test_proxy_server.py s..........s [100%] ================================= 10 passed, 2 skipped in 9.24s ================================= ```

krrishdholakia · 2024-05-13T15:42:01Z

litellm/proxy/_types.py

@@ -216,7 +217,7 @@ class LiteLLMPromptInjectionParams(LiteLLMBase):
    llm_api_system_prompt: Optional[str] = None
    llm_api_fail_call_string: Optional[str] = None

-    @root_validator(pre=True)
+    @model_validator(mode="before")


i believe this is pydantic v2 only? @msabramo

openai sdk supports v1 pydantic - https://github.com/openai/openai-python/blob/906ece0e45235cb170bc062902182eb8b55698b9/pyproject.toml#L12

Any way we can fix the warnings without breaking compatibility

Hi @krrishdholakia! Ok, take a look at #3611. That fixes just the root_validator warnings in pydantic v2, while still working in pydantic v1.

I went ahead and merged #3611 into this PR, so there's just one PR and I ensured there are no merge conflicts.

pydantic v1 uses `root_validator` and pydantic v2 uses `model_validator`. pydantic v2 emits a warning when `root_validator` is used. E.g.: ``` litellm/proxy/_types.py:225 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) ``` This change eliminates those warnings with pydantic v2, while retaining compatibility with pydantic v1. pydantic 2.7.1 before ``` $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ... litellm/proxy/_types.py:225 /Users/abramowi/Code/OpenSource/litellm/litellm/proxy/_types.py:225: PydanticDeprecatedSince20: Pydantic V1 style `@root_validator` validators are deprecated. You should migrate to Pydantic V2 style `@model_validator` validators, see the migration guide for more details. Deprecated in Pydantic V2.0 to be removed in V3.0. See Pydantic V2 Migration Guide at https://errors.pydantic.dev/2.7/migration/ @root_validator(pre=True) ... ========================== 10 passed, 2 skipped, 39 warnings in 8.67s =========================== ``` pydantic 2.7.1 after ``` $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ... ========================== 10 passed, 2 skipped, 27 warnings in 9.85s =========================== ``` pydantic 1.10.5 after ``` $ poetry run pip install 'pydantic<2' ... Successfully installed pydantic-1.10.15 $ env -i PATH=$PATH poetry run pytest litellm/tests/test_proxy_server.py ... =========================== 10 passed, 2 skipped, 1 warning in 8.13s ============================ ```

I have it in a separate PR: BerriAI#3611

These don't work in pydantic v1.

in `litellm/types/router.py` I didn't replace the ones that have `allow_arbitrary_types` because changing those seems to break pydantic v1 compatibility.

pydantic v2 warns about using a Config class. But without this, pydantic v1 will raise an error: RuntimeError: no validator found for <class 'openai.Timeout'>, see `arbitrary_types_allowed` in Config Putting arbitrary_types_allowed = True in the ConfigDict doesn't work in pydantic v1. So we mostly use model_config = ConfigDict(...) and then only use the Config class with arbitrary_types_allowed = True for pydantic v1.

…idator' into msabramo/fix-pydantic-warnings

krrishdholakia · 2024-05-13T21:30:12Z

litellm/tests/test_streaming.py

@@ -2214,15 +2214,15 @@ class ToolCalls(BaseModel):

 class Delta(BaseModel):
    role: str
-    content: Optional[str]
+    content: Optional[str] = None


@msabramo any reason why the pydantic PR also involves setting these values to None?

We've seen issues during streaming when setting default values like this

@krrishdholakia Nah, no particular reason. I think those came from me running this:

bump-pydantic litellm

I just went and reverted those changes in bdb995f.

@krrishdholakia

because @krrishdholakia said [here](BerriAI#3600 (comment)) that they were problematic in the past.

krrishdholakia · 2024-05-14T01:00:14Z

Awesome - is this ready for review? @msabramo

msabramo · 2024-05-14T03:07:21Z

Awesome - is this ready for review? @msabramo

Yep, go ahead!

krrishdholakia · 2024-05-14T05:00:36Z

LGTM!

Will take care of fixing any ci/cd issues on main

krrishdholakia · 2024-05-14T05:00:50Z

Thanks for all your contributions @msabramo

[Fix] Revert #3600 #3600

vercel bot deployed to Preview May 12, 2024 17:13 View deployment

vercel bot deployed to Preview May 12, 2024 17:18 View deployment

msabramo force-pushed the msabramo/fix-pydantic-warnings branch from 4e5c950 to 1137560 Compare May 12, 2024 17:18

vercel bot deployed to Preview May 12, 2024 17:18 View deployment

msabramo force-pushed the msabramo/fix-pydantic-warnings branch from 1137560 to 274872c Compare May 12, 2024 17:20

vercel bot deployed to Preview May 12, 2024 17:20 View deployment

Add my commit to .git-blame-ignore-revs

abe2514

because I made a lot of fairly mindless changes to pydantic code to fix warnings and I don't want git blame to give people the impression that I know more about this code than I do.

msabramo force-pushed the msabramo/fix-pydantic-warnings branch from 274872c to abe2514 Compare May 12, 2024 17:21

vercel bot deployed to Preview May 12, 2024 17:22 View deployment

msabramo mentioned this pull request May 12, 2024

Update FastAPI to update starlette to fix warnings #3601

Merged

msabramo mentioned this pull request May 12, 2024

Fix pkg_resources warning #3602

Merged

krrishdholakia reviewed May 13, 2024

View reviewed changes

msabramo added 3 commits May 13, 2024 10:23

Revert root_validator => model_validator change

abd90e6

I have it in a separate PR: BerriAI#3611

Revert litellm/types/router.py changes

eacafa7

These don't work in pydantic v1.

vercel bot deployed to Preview May 13, 2024 17:35 View deployment

model_config = ConfigDict instead of model_config = dict

4ccdbfd

vercel bot deployed to Preview May 13, 2024 17:39 View deployment

Replace some Config classes with model_config

f233cde

in `litellm/types/router.py` I didn't replace the ones that have `allow_arbitrary_types` because changing those seems to break pydantic v1 compatibility.

vercel bot deployed to Preview May 13, 2024 18:06 View deployment

vercel bot deployed to Preview May 13, 2024 18:25 View deployment

Merge branch 'msabramo/pydantic_replace_root_validator_with_model_val…

73541b1

…idator' into msabramo/fix-pydantic-warnings

vercel bot deployed to Preview May 13, 2024 18:27 View deployment

msabramo mentioned this pull request May 13, 2024

Change pydantic root_validator to model_validator to eliminate warnings during pytest #3611

Closed

msabramo requested a review from krrishdholakia May 13, 2024 18:32

Merge branch 'main' into msabramo/fix-pydantic-warnings

1efa2e6

vercel bot deployed to Preview May 13, 2024 18:38 View deployment

krrishdholakia reviewed May 13, 2024

View reviewed changes

Revert = None changes

bdb995f

because @krrishdholakia said [here](BerriAI#3600 (comment)) that they were problematic in the past.

vercel bot deployed to Preview May 13, 2024 23:51 View deployment

krrishdholakia merged commit 2c867ea into BerriAI:main May 14, 2024
2 checks passed

msabramo deleted the msabramo/fix-pydantic-warnings branch May 14, 2024 16:58

ishaan-jaff mentioned this pull request May 16, 2024

[Fix] Revert #3600 https://github.com/BerriAI/litellm/pull/3600 #3664

Merged

ishaan-jaff added a commit that referenced this pull request May 16, 2024

Merge pull request #3664 from BerriAI/litellm_revert_3600

b3d472f

[Fix] Revert #3600 #3600

krrishdholakia mentioned this pull request May 16, 2024

Fix pytest warnings #3416

Closed

lj-wego mentioned this pull request May 16, 2024

Fix warnings from pydantic #3670

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update pydantic code to fix warnings #3600

Update pydantic code to fix warnings #3600

msabramo commented May 12, 2024 •

edited

vercel bot commented May 12, 2024 •

edited

krrishdholakia May 13, 2024

msabramo May 13, 2024 •

edited

msabramo May 13, 2024

krrishdholakia May 13, 2024

msabramo May 13, 2024 •

edited

krrishdholakia commented May 14, 2024

msabramo commented May 14, 2024

krrishdholakia commented May 14, 2024

krrishdholakia commented May 14, 2024

Update pydantic code to fix warnings #3600

Update pydantic code to fix warnings #3600

Conversation

msabramo commented May 12, 2024 • edited

vercel bot commented May 12, 2024 • edited

krrishdholakia May 13, 2024

Choose a reason for hiding this comment

msabramo May 13, 2024 • edited

Choose a reason for hiding this comment

msabramo May 13, 2024

Choose a reason for hiding this comment

krrishdholakia May 13, 2024

Choose a reason for hiding this comment

msabramo May 13, 2024 • edited

Choose a reason for hiding this comment

krrishdholakia commented May 14, 2024

msabramo commented May 14, 2024

krrishdholakia commented May 14, 2024

krrishdholakia commented May 14, 2024

msabramo commented May 12, 2024 •

edited

vercel bot commented May 12, 2024 •

edited

msabramo May 13, 2024 •

edited

msabramo May 13, 2024 •

edited