Support defining configuration and stream schemas using Pydantic
Pydantic is the most popular data validation and serialization library for Python at the moment. It is used by the new-ish and increasingly popular web framework FastAPI and by other frameworks wanting to support reliable data validation and serialization (like odmantic for MongoDB and pydantic-sqlalchemy).
I propose leveraging Pydantic to allow the SDK user to define a plugin's configuration and inline stream schemas using a known, powerful and well-documented library. The implementation would look something like the following:
from pydantic import BaseModel class BaseSchema(BaseModel): # Created by the SDK developers to support the specifics of Singer schemas class TapTestConfig(BaseSchema): # Plugin config created by the SDK user class ExampleStreamSchema(BaseSchema): # Stream schema created by the SDK user
BaseSchemaneeds to extend pydantic to make sure schemas are Singer-compatible: https://pydantic-docs.helpmanual.io/usage/schema/#schema-customization
- Initial discussion for pydantic use: https://gitlab.com/meltano/singer-sdk/-/issues/10#note_519595958