better templates storage/sync handling
Currently templates location used in sync_templates
is controlled with APPLICATION_TEMPLATES_DIR
path setting which points to a fixed path in the repository. This is not very flexible and requires working with full repository when modifying templates. There are several ways to make things more flexible here:
- make
APPLICATION_TEMPLATES_DIR
read from env - make
APPLICATION_TEMPLATES_DIR
a list of paths - additionally to this setting, use entry point (i.e.
baserow.data.templates
initially introduced in !2326 (merged)) to point to module data that stores template dumps. This allows to inject templates without modifying anything in the code.
sample entry_point (baserow.data.templates
):
[project.entry-points."baserow.data.templates"]
# the entry point expects at least one entry in
# key='module.name'. keys are arbitrary.
basic='baserow.data.templates.basic'
extra='baserow.data.templates.extra'
tells that in baserow.data.templates.basic
and baserow.data.templates.extra
modules should reside package data with templates.
$ ls -1 baserow/data/templates/
__init__.py
some_template.json
some_template.zip
...
Later on, in code we need just to read entry points, and for each item in entry point list files with given mask. The code below extends templates
list, which can be used for templates synchronization:
# helper functions that handle
from importlib.metadata import entry_points
from importlib.resources import files
templates = []
# list entry points defined in other packages
# note: this is python3.11 syntax. pre-3.11 won't allow `group=` keyword arg.
entry_points(group='baserow.data.templates')
# for a package that provides own templates, it has to have
# 'baserow.data.templates' entry point defined in pyproject.toml (or setup.py):
#
# [project.entry_points."baserow.data.templates"]
# templates="mypackage.baserow.templates"
#
# `mypackage.baserow.templates` module's directory should contain json/zip
# files with templates
for ep in eps.get("baserow.data.templates") or []:
# importlib.resources.files(module_name) will return pathlib.Path that points to a directory for a module.
# for a directory
templates.extend(list(files(ep.module).glob('*.json')))
One note here: reading files from several locations may cause duplicates (multiple files with the same name but in different directories). Some mitigation here would be to use vendor prefix in template name (i.e. mypackage.baserow.templates.MyTemplate
).
Using entry point technique allows to do several things:
- split direct dependency between code in repository from templates data
- allow to split templates into smaller groups (template packages)
- allow to distribute templates separately from code
- allow external developers to provide their own packages with templates in similar way as plugins.