Draft: Move deps to pyproject.toml. Update Deps. Regen Protos.
Description
Use the latest stable configurations for packaging the buildgrid project by leveraging pyproject.toml and setup.cfg to configure package layouts.
As it stands, all of the requirements contents had been duplicated in the main requirements.in file, leaving most of the CI setup redundant by installing the main .in file and the individual extras .in files. This MR leaves the behavior mostly untouched to begin the process of properly defining the extras with pyproject.toml and setup.cfg. It uses pip's recursive dependency feature to construct the "all" extra, removing the need for some of the more complex pip-tools usage.
The protobuf generation utility has been replaced by tox -e protos
which now supports creating pyi
stub files for the pb2_grpc
modules as well. A pb2_grpc_aio
module is also created which will
allow creating entities using the asyncio fixtures from grpc.aio. The
majority of the code changes from this MR are simply from regenerating
the protos.
Note: There were a number of unused json and yaml files in the protos packages which I imaging were generated from previous protoc calls. These files have been removed.
The utility for regeneration of the requirements.txt file has also been
replaced in this work. You can now use tox -e requirements-append
to add
newly defined dependencies from the pyproject.toml
into requirements.txt
(keeping old pins the same) or use tox -e requirements-refresh
to upgrade
all dependencies to the latest versions (ignoring the previous pins).
Note: Updates to some of the libraries exposed more type errors that I have marked ignored. We should revisit them at some point.
Sadly, pylint
does not seem to support the new protobuf generation, failing
to resolve imports. I have switched the linting to instead use flake8
,
which runs much faster and covers a wider number of use-cases. We may also
choose to later extend flake8
with plugins, which I have done to continue
creating the codeclimate.json
file, using the flake8-gl-codeclimate
plugin.
Finally, I've updated the docs to point to these new tools for future devs.