Re-factor `src/fgen/templates` and `src/fgen/templator.py`
The problem
As we've added support for more things, our templating has become increasingly complicated. The high-level summary is: we need to work out a way to clarify what is going on with our templates.
Definition of "done"
-
We have refactored src/fgen/templates
andsrc/fgen/templator.py
in a way that we are happy with. This is extremely vague, the reason for this is below.
Additional context
The templating is actually a really difficult problem. I don't think we should tackle this lightly, and it looks like we're going to get away with our current implementation for at least a little while. It's just that our current implementation will be increasingly difficult to maintain, the longer we leave this untidied.
In my opinion, the wrapper generation is really difficult because there are couplings between the high-level package structure and details. For example, to write the fortran wrapper for a method that takes in a derived type as an argument and returns a derived type, you need to know where these derived types come from in the overall package structure. So even at the level of an individual line, you sometimes need to know things about the top-level structure. I think this just makes things hard.
My suggestion for a way through this: we try and move all of the logic and the data model to Python, because the logic and data model are way easier to write, understand and test in Python. Then we thin out the jinja accordingly, only using jinja where templating really makes our lives easier (and avoiding logic in the jinja as much as possible).
A structure something like the below makes sense to me (and will also have implications for #2):
-
Package
- structure of the package. Holds all the modules and knows about the dependency tree between the modules (helps pull the dependency logic out of jinja, where it currently lives). Having an object like this would also help us with #2 (where I think we could write something that can write the python wrappers, fortran wrappers, fortran managers, CMakeLists.txt file and pyproject.toml file).- I would love this to have methods like:
-
generate_python_wrapper(module: str) -> str
: generates the Python wrapper for a module within the package -
get_python_wrapper_file(module: str, directory: Path) -> Path
: gets the path in which to write the Python wrapper file (assumption here being that the directory to write in is specified by the user (via some mechanism we'd have to work out), but the filename follows some internal rules)- the combo of
generate_python_wrapper
andget_python_wrapper_file
then just leaves the trivial steps of formatting the generated wrapper and writing it in the desired output path
- the combo of
-
generate_fortran_wrapper(module: str) -> str
: generates the Fortran wrapper for a module within the package -
get_fortran_wrapper_file(module: str, directory: Path) -> Path
: same idea as forget_python_wrapper_file
- same idea as the Python wrapper re just combining the generate method with the get path method, before formatting and writing
-
generate_fortran_manager(module: str) -> str
: generates the Fortran lifecycle manager for a module within the package -
get_fortran_manager_file(module: str, directory: Path) -> Path
: same idea as forget_python_wrapper_file
- same idea as the Python wrapper re just combining the generate method with the get path method, before formatting and writing
-
generate_cmakelists() -> str
: generates theCMakeLists.txt
file for the package (also happy to not handle auto-generating build stuff, I know writing these can be tricky) -
get_cmakelists_file(directory: Path) -> Path
: same idea as forget_python_wrapper_file
- same idea as the Python wrapper re just combining the generate method with the get path method, before formatting and writing
-
generate_pyproject_toml() -> str
: generates thepyproject.toml
file for the package (also happy to not handle auto-generating build stuff, I know writing these can be tricky) -
get_pyproject_toml_file(directory: Path) -> Path
: same idea as forget_python_wrapper_file
- same idea as the Python wrapper re just combining the generate method with the get path method, before formatting and writing
-
- I would love this to have methods like:
-
Modules
,Attributes
etc. - basically as they are now -
Add methods onto the
FortranDataType
's for auto-generating some wrapper stuff. The type of fortran data defines how we wrap the things (we're seeing this in our templates, where attributes and methods are split based on the fortran type). It makes sense to move this logic next to the data it relates to. We'll need a lot of subtly different methods on ourFortranDataType
(or maybe it makes more sense if each data type gets its own class where the generation functions live, anyway) to handle the different ways we need to pass this data e.g.generate_python_facing_input_parameter_line
,generate_python_value_retrieval_code
,generate_python_to_fortran_passing_code
etc. Those methods might themselves use jinja templates, but I think this way of splitting it will make it easier to see what is going on.
See also discussion here: !79 (comment 1814507924)