✨ Add support for Pydantic v2 (#9816)

✨ Add support for Pydantic v2 (#9816)

* ✨ Pydantic v2 migration, initial implementation (#9500)

* ✨ Add compat layer, for Pydantic v1 and v2

* ✨ Re-export Pydantic needed internals from compat, to later patch them for v1

* ♻️ Refactor internals to use new compatibility layers and run with Pydantic v2

* 📝 Update examples to run with Pydantic v2

* ✅ Update tests to use Pydantic v2

* 🎨 [pre-commit.ci] Auto format from pre-commit.com hooks

* ✅ Temporarily disable Peewee tests, afterwards I'll enable them only for Pydantic v1

* 🐛 Fix JSON Schema generation and OpenAPI ref template

* 🐛 Fix model field creation with defaults from Pydantic v2

* 🐛 Fix body field creation, with new FieldInfo

* ✨ Use and check new ResponseValidationError for server validation errors

* ✅ Fix test_schema_extra_examples tests with ResponseValidationError

* ✅ Add dirty-equals to tests for compatibility with Pydantic v1 and v2

* ✨ Add util to regenerate errors with custom loc

* ✨ Generate validation errors with loc

* ✅ Update tests for compatibility with Pydantic v1 and v2

* ✅ Update tests for Pydantic v2 in tests/test_filter_pydantic_sub_model.py

* ✅ Refactor tests in tests/test_dependency_overrides.py for Pydantic v2, separate parameterized into independent tests to use insert_assert

* ✅ Refactor OpenAPI test for tests/test_infer_param_optionality.py for consistency, and make it compatible with Pydantic v1 and v2

* ✅ Update tests for tests/test_multi_query_errors.py for Pydantic v1 and v2

* ✅ Update tests for tests/test_multi_body_errors.py for Pydantic v1 and v2

* ✅ Update tests for tests/test_multi_body_errors.py for Pydantic v1 and v2

* 🎨 [pre-commit.ci] Auto format from pre-commit.com hooks

* ♻️ Refactor tests for tests/test_path.py to inline pytest parameters, to make it easier to make them compatible with Pydantic v2

* ✅ Refactor and udpate tests for tests/test_path.py for Pydantic v1 and v2

* ♻️ Refactor and update tests for tests/test_query.py with compatibility for Pydantic v1 and v2

* ✅ Fix test with optional field without default None

* ✅ Update tests for compatibility with Pydantic v2

* ✅ Update tutorial tests for Pydantic v2

* ♻️ Update OAuth2 dependencies for Pydantic v2

* ♻️ Refactor str check when checking for sequence types

* ♻️ Rename regex to pattern to keep in sync with Pydantic v2

* ♻️ Refactor _compat.py, start moving conditional imports and declarations to specifics of Pydantic v1 or v2

* ✅ Update tests for OAuth2 security optional

* ✅ Refactor tests for OAuth2 optional for Pydantic v2

* ✅ Refactor tests for OAuth2 security for compatibility with Pydantic v2

* 🐛 Fix location in compat layer for Pydantic v2 ModelField

* ✅ Refactor tests for Pydantic v2 in tests/test_tutorial/test_bigger_applications/test_main_an_py39.py

* 🐛 Add missing markers in Python 3.9 tests

* ✅ Refactor tests for bigger apps for consistency with annotated ones and with support for Pydantic v2

* 🐛 Fix jsonable_encoder with new Pydantic v2 data types and Url

* 🐛 Fix invalid JSON error for compatibility with Pydantic v2

* ✅ Update tests for behind_a_proxy for Pydantic v2

* ✅ Update tests for tests/test_tutorial/test_body/test_tutorial001_py310.py for Pydantic v2

* ✅ Update tests for tests/test_tutorial/test_body/test_tutorial001.py with Pydantic v2 and consistency with Python 3.10 tests

* ✅ Fix tests for tutorial/body_fields for Pydantic v2

* ✅ Refactor tests for tutorial/body_multiple_params with Pydantic v2

* ✅ Update tests for tutorial/body_nested_models for Pydantic v2

* ✅ Update tests for tutorial/body_updates for Pydantic v2

* ✅ Update test for tutorial/cookie_params for Pydantic v2

* ✅ Fix tests for tests/test_tutorial/test_custom_request_and_route/test_tutorial002.py for Pydantic v2

* ✅ Update tests for tutorial/dataclasses for Pydantic v2

* ✅ Update tests for tutorial/dependencies for Pydantic v2

* ✅ Update tests for tutorial/extra_data_types for Pydantic v2

* ✅ Update tests for tutorial/handling_errors for Pydantic v2

* ✅ Fix test markers for Python 3.9

* ✅ Update tests for tutorial/header_params for Pydantic v2

* ✅ Update tests for Pydantic v2 in tests/test_tutorial/test_openapi_callbacks/test_tutorial001.py

* ✅ Fix extra tests for Pydantic v2

* ✅ Refactor test for parameters, to later fix Pydantic v2

* ✅ Update tests for tutorial/query_params for Pydantic v2

* ♻️ Update examples in docs to use new pattern instead of the old regex

* ✅ Fix several tests for Pydantic v2

* ✅ Update and fix test for ResponseValidationError

* 🐛 Fix check for sequences vs scalars, include bytes as scalar

* 🐛 Fix check for complex data types, include UploadFile

* 🐛 Add list to sequence annotation types

* 🐛 Fix checks for uploads and add utils to find if an annotation is an upload (or bytes)

* ✨ Add UnionType and NoneType to compat layer

* ✅ Update tests for request_files for compatibility with Pydantic v2 and consistency with other tests

* ✅ Fix testsw for request_forms for Pydantic v2

* ✅ Fix tests for request_forms_and_files for Pydantic v2

* ✅ Fix tests in tutorial/security for compatibility with Pydantic v2

* ⬆️ Upgrade required version of email_validator

* ✅ Fix tests for params repr

* ✅ Add Pydantic v2 pytest markers

* Use match_pydantic_error_url

* 🎨 [pre-commit.ci] Auto format from pre-commit.com hooks

* Use field_serializer instead of encoders in some tests

* Show Undefined as ... in repr

* Mark custom encoders test with xfail

* Update test to reflect new serialization of Decimal as str

* Use `model_validate` instead of `from_orm`

* Update JSON schema to reflect required nullable

* Add dirty-equals to pyproject.toml

* Fix locs and error creation for use with pydantic 2.0a4

* Use the type adapter for serialization. This is hacky.

* 🎨 [pre-commit.ci] Auto format from pre-commit.com hooks

* ✅ Refactor test_multi_body_errors for compatibility with Pydantic v1 and v2

* ✅ Refactor test_custom_encoder for Pydantic v1 and v2

* ✅ Set input to None for now, for compatibility with current tests

* 🐛 Fix passing serialization params to model field when handling the response

* ♻️ Refactor exceptions to not depend on Pydantic ValidationError class

* ♻️ Revert/refactor params to simplify repr

* ✅ Tweak tests for custom class encoders for Pydantic v1 and v2

* ✅ Tweak tests for jsonable_encoder for Pydantic v1 and v2

* ✅ Tweak test for compatibility with Pydantic v1 and v2

* 🐛 Fix filtering data with subclasses

* 🐛 Workaround examples in OpenAPI schema

* ✅ Add skip marker for SQL tutorial, needs to be updated either way

* ✅ Update test for broken JSON

* ✅ Fix test for broken JSON

* ✅ Update tests for timedeltas

* ✅ Fix test for plain text validation errors

* ✅ Add markers for Pydantic v1 exclusive tests (for now)

* ✅ Update test for path_params with enums for compatibility with Pydantic v1 and v2

* ✅ Update tests for extra examples in OpenAPI

* ✅ Fix tests for response_model with compatibility with Pydantic v1 and v2

* 🐛 Fix required double serialization for different types of models

* ✅ Fix tests for response model with compatibility with new Pydantic v2

* 🐛 Import Undefined from compat layer

* ✅ Fix tests for response_model for Pydantic v2

* ✅ Fix tests for schema_extra for Pydantic v2

* ✅ Add markers and update tests for Pydantic v2

* 💡 Comment out logic for double encoding that breaks other usecases

* ✅ Update errors for int parsing

* ♻️ Refactor re-enabling compatibility for Pydantic v1

* ♻️ Refactor OpenAPI utils to re-enable support for Pydantic v1

* ♻️ Refactor dependencies/utils and _compat for compatibility with Pydantic v1

* 🐛 Fix and tweak compatibility with Pydantic v1 and v2 in dependencies/utils

* ✅ Tweak tests and examples for Pydantic v1

* ♻️ Tweak call to ModelField.validate for compatibility with Pydantic v1

* ✨ Use new global override TypeAdapter from_attributes

* ✅ Update tests after updating from_attributes

* 🔧 Update pytest config to avoid collecting tests from docs, useful for editor-integrated tests

* ✅ Add test for data filtering, including inheritance and models in fields or lists of models

* ♻️ Make OpenAPI models compatible with both Pydantic v1 and v2

* ♻️ Fix compatibility for Pydantic v1 and v2 in jsonable_encoder

* ♻️ Fix compatibility in params with Pydantic v1 and v2

* ♻️ Fix compatibility when creating a FieldInfo in Pydantic v1 and v2 in utils.py

* ♻️ Fix generation of flat_models and JSON Schema definitions in _compat.py for Pydantic v1 and v2

* ♻️ Update handling of ErrorWrappers for Pydantic v1

* ♻️ Refactor checks and handling of types an sequences

* ♻️ Refactor and cleanup comments with compatibility for Pydantic v1 and v2

* ♻️ Update UploadFile for compatibility with both Pydantic v1 and v2

* 🔥 Remove commented out unneeded code

* 🐛 Fix mock of get_annotation_from_field_info for Pydantic v1 and v2

* 🐛 Fix params with compatibility for Pydantic v1 and v2, with schemas and new pattern vs regex

* 🐛 Fix check if field is sequence for Pydantic v1

* ✅ Fix tests for custom_schema_fields, for compatibility with Pydantic v1 and v2

* ✅ Simplify and fix tests for jsonable_encoder with compatibility for Pydantic v1 and v2

* ✅ Fix tests for orm_mode with Pydantic v1 and compatibility with Pydantic v2

* ♻️ Refactor logic for normalizing Pydantic v1 ErrorWrappers

* ♻️ Workaround for params with examples, before defining what to deprecate in Pydantic v1 and v2 for examples with JSON Schema vs OpenAPI

* ✅ Fix tests for Pydantic v1 and v2 for response_by_alias

* ✅ Fix test for schema_extra with compatibility with Pydantic v1 and v2

* ♻️ Tweak error regeneration with loc

* ♻️ Update error handling and serializationwith compatibility for Pydantic v1 and v2

* ♻️ Re-enable custom encoders for Pydantic v1

* ♻️ Update ErrorWrapper reserialization in Pydantic v1, do it outside of FastAPI ValidationExceptions

* ✅ Update test for filter_submodel, re-structure to simplify testing while keeping division of Pydantic v1 and v2

* ✅ Refactor Pydantic v1 only test that requires modifying environment variables

* 🔥 Update test for plaintext error responses, for Pydantic v1 and v2

* ⏪️ Revert changes in DB tutorial to use Pydantic v1 (the new guide will have SQLModel)

* ✅ Mark current SQL DB tutorial tests as Pydantic only

* ♻️ Update datastructures for compatibility with Pydantic v1, not requiring pydantic-core

* ♻️ Update encoders.py for compatibility with Pydantic v1

* ⏪️ Revert changes to Peewee, the docs for that are gonna live in a new HowTo section, not in the main tutorials

* ♻️ Simplify response body kwargs generation

* 🔥 Clean up comments

* 🔥 Clean some tests and comments

* ✅ Refactor tests to match new Pydantic error string URLs

* ✅ Refactor tests for recursive models for Pydantic v1 and v2

* ✅ Update tests for Peewee, re-enable, Pydantic-v1-only

* ♻️ Update FastAPI params to take regex and pattern arguments

* ⏪️ Revert tutorial examples for pattern, it will be done in a subsequent PR

* ⏪️ Revert changes in schema extra examples, it will be added later in a docs-specific PR

* 💡 Add TODO comment to document str validations with pattern

* 🔥 Remove unneeded comment

* 📌 Upgrade Pydantic pin dependency

* ⬆️ Upgrade email_validator dependency

* 🐛 Tweak type annotations in _compat.py

* 🔇 Tweak mypy errors for compat, for Pydantic v1 re-imports

* 🐛 Tweak and fix type annotations

* ➕ Update requirements-test.txt, re-add dirty-equals

* 🔥 Remove unnecessary config

* 🐛 Tweak type annotations

* 🔥 Remove unnecessary type in dependencies/utils.py

* 💡 Update comment in routing.py

---------

Co-authored-by: David Montague <35119617+dmontagu@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* 👷 Add CI for both Pydantic v1 and v2 (#9688)

* 👷 Test and install Pydantic v1 and v2 in CI

* 💚 Tweak CI config for Pydantic v1 and v2

* 💚 Fix Pydantic v2 specification in CI

* 🐛 Fix type annotations for compatibility with Python 3.7

* 💚 Install Pydantic v2 for lints

* 🐛 Fix type annotations for Pydantic v2

* 💚 Re-use test cache for lint

* ♻️ Refactor internals for test coverage and performance (#9691)

* ♻️ Tweak import of Annotated from typing_extensions, they are installed anyway

* ♻️ Refactor _compat to define functions for Pydantic v1 or v2 once instead of checking inside

* ✅ Add test for UploadFile for Pydantic v2

* ♻️ Refactor types and remove logic for impossible cases

* ✅ Add missing tests from test refactor for path params

* ✅ Add tests for new decimal encoder

* 💡 Add TODO comment for decimals in encoders

* 🔥 Remove unneeded dummy function

* 🔥 Remove section of code in field_annotation_is_scalar covered by sub-call to field_annotation_is_complex

* ♻️ Refactor and tweak variables and types in _compat

* ✅ Add tests for corner cases and compat with Pydantic v1 and v2

* ♻️ Refactor type annotations

* 🔖 Release version 0.100.0-beta1

* ♻️ Refactor parts that use optional requirements to make them compatible with installations without them (#9707)

* ♻️ Refactor parts that use optional requirements to make them compatible with installations without them

* ♻️ Update JSON Schema for email field without email-validator installed

* 🐛 Fix support for Pydantic v2.0, small changes in their final release (#9771)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Sebastián Ramírez <tiangolo@gmail.com>
* 🔖 Release version 0.100.0-beta2

* ✨ OpenAPI 3.1.0 with Pydantic v2, merge `master` (#9773)

* ➕ Add dirty-equals as a testing dependency (#9778)

➕ Add dirty-equals as a testing dependency, it seems it got lsot at some point

* 🔀 Merge master, fix valid JSON Schema accepting bools (#9782)

* ⏪️ Revert usage of custom logic for TypeAdapter JSON Schema, solved on the Pydantic side (#9787)

⏪️ Revert usage of custom logic for TypeAdapter JSON Schema, solved on Pydantic side

* ♻️ Deprecate parameter `regex`, use `pattern` instead (#9786)

* 📝 Update docs to deprecate regex, recommend pattern

* ♻️ Update examples to use new pattern instead of regex

* 📝 Add new example with deprecated regex

* ♻️ Add deprecation notes and warnings for regex

* ✅ Add tests for regex deprecation

* ✅ Update tests for compatibility with Pydantic v1

* ✨ Update docs to use Pydantic v2 settings and add note and example about v1 (#9788)

* ➕ Add pydantic-settings to all extras

* 📝 Update docs for Pydantic settings

* 📝 Update Settings source examples to use Pydantic v2, and add a Pydantic v1 version

* ✅ Add tests for settings with Pydantic v1 and v2

* 🔥 Remove solved TODO comment

* ♻️ Update conditional OpenAPI to use new Pydantic v2 settings

* ✅ Update tests to import Annotated from typing_extensions for Python < 3.9 (#9795)

* ➕ Add pydantic-extra-types to fastapi[extra]

* ➕ temp: Install Pydantic from source to test JSON Schema metadata fixes (#9777)

* ➕ Install Pydantic from source, from branch for JSON Schema with metadata

* ➕ Update dependencies, install Pydantic main

* ➕ Fix dependency URL for Pydantic from source

* ➕ Add pydantic-settings for test requirements

* 💡 Add TODO comments to re-enable Pydantic main (not from source) (#9796)

* ✨ Add new Pydantic Field param options to Query, Cookie, Body, etc. (#9797)

* 📝 Add docs for Pydantic v2 for `docs/en/docs/advanced/path-operation-advanced-configuration.md` (#9798)

* 📝 Update docs in examples for settings with Pydantic v2 (#9799)

* 📝 Update JSON Schema `examples` docs with Pydantic v2 (#9800)

* ♻️ Use new Pydantic v2 JSON Schema generator (#9813)

Co-authored-by: David Montague <35119617+dmontagu@users.noreply.github.com>
* ♻️ Tweak type annotations and Pydantic version range (#9801)

* 📌 Re-enable GA Pydantic, for v2, require minimum 2.0.2 (#9814)

* 🔖 Release version 0.100.0-beta3

* 🔥 Remove duplicate type declaration from merge conflicts (#9832)

* 👷‍♂️ Run tests with Pydantic v2 GA (#9830)

👷 Run tests for Pydantic v2 GA

* 📝 Add notes to docs expecting Pydantic v2 and future updates (#9833)

* 📝 Update index with new extras

* 📝 Update release notes

---------

Co-authored-by: David Montague <35119617+dmontagu@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Pastukhov Nikita <diementros@yandex.ru>