Skip to content
Snippets Groups Projects

Draft: Use pydantic to represent configs. Rework constructors

Closed Cal Pratt requested to merge cpratt34/yaml-loader into master
  1. May 24, 2023
    • Cal Pratt's avatar
      Add missing remote-storage option · b625e97e
      Cal Pratt authored
      b625e97e
    • Cal Pratt's avatar
      Mypy found bugs! Hooray! · dfe62538
      Cal Pratt authored
      dfe62538
    • Cal Pratt's avatar
      fix connect-args · b3340d19
      Cal Pratt authored
      b3340d19
    • Cal Pratt's avatar
      Add s3 storage constructor · ebf0fadf
      Cal Pratt authored
      ebf0fadf
    • Cal Pratt's avatar
      flake8 · feac4093
      Cal Pratt authored
      feac4093
    • Cal Pratt's avatar
      Use allow_population_by_field_name=True · 268ff01f
      Cal Pratt authored
      268ff01f
    • Cal Pratt's avatar
      Use pydantic to represent configs. Rework constructors · d513796a
      Cal Pratt authored
      The way configurations are represented today makes the loading of the server
      hard to reason about and to debug. The setup of the yaml constructors and
      json schemas allows room for errors, as there is no way to validate that the
      json schema accurately reflects interface of the yaml constructor.
      Additionally, yaml.add_constructor does not allow us to do things like use
      context managers for the creation of external resource components, as there
      is no easy way to inject context into the loading logic.
      
      This change tries to achieve two things:
      
      1) Replacing the schemas with pydantic models.
      
         This will allow us to achieve greater type safety when converting from
         the yaml representation into the actual objects. I started off this
         refactoring by first taking each schema and coverting it, slowly adding
         all of the missing references to configs which I found along the way.
      
      2) Separate the actual class construction from the yaml loading the.
      
         Given a pydantic model representing the configs we can create the server
         instances in a much more controlled manner. To ensure we do not create
         duplicates of objects, a context-var based wrapper has been applied to
         the individual constructor methods. This pattern should allow us to more
         easily add new configs, and opens up the door to add context to the
         constructor calls. e.g. an ExitStack for context manager based resources.
      
      Notes: ruamel.yaml provides a better alternative to PyYAML. It allows
      setting the constructors on a yaml loader instance rather than the module
      itself, meaning constructors do not bleed into the global state, and will
      not interfer with other loaders in multithreaded contexts.
      d513796a
Loading