Python scripts infrastructure
This came in !235 (closed).
When working on Python scripts, after editing some file under
python/, the changes will not be reflected in the script generated in
out/bin/ after a new make when
sudo make install was previously run.
Steps to reproduce:
$ git status On branch master Your branch is up to date with 'origin/master'. nothing to commit, working tree clean $ # Edit files. $ git diff diff --git a/python/convertrules.py b/python/convertrules.py index 5316839a8d..66a7222857 100644 --- a/python/convertrules.py +++ b/python/convertrules.py @@ -26,7 +26,7 @@ import sys import lilylib -NOT_SMART = "\n" + _ ("Not smart enough to convert %s.") + "\n" +NOT_SMART = "\n" + _ ("AAA Not smart enough to convert %s.") + "\n" UPDATE_MANUALLY = _ ("Please refer to the manual for details, and update manually.") + "\n" FROM_TO = _ ("%s has been replaced by %s") + "\n" diff --git a/scripts/convert-ly.py b/scripts/convert-ly.py index 7dd9bcc0f9..8a707e8b43 100644 --- a/scripts/convert-ly.py +++ b/scripts/convert-ly.py @@ -36,6 +36,7 @@ gettext.install ('lilypond', '@[email protected]') import lilylib as ly import convertrules +print(convertrules.NOT_SMART) lilypond_version_re_str = '\\\\version *\"([0-9.]+)"' lilypond_version_re = re.compile (lilypond_version_re_str) $ cd build $ make Making python/out/convertrules.pyc.dummy (py compile) Making scripts/out/convert-ly (sed) Making scripts/out/convert-ly.1 $ out/bin/convert-ly Not smart enough to convert %s. Usage: convert-ly [OPTION]... FILE [etc.]
A little debugging shows that when
convert-ly is run,
['/home/jean/repos/lilypond/build/out/lib/lilypond/current/python', '/home/jean/repos/lilypond/build/out/share/lilypond/current/python', '/usr/local/lib/lilypond/2.21.3/python', '/usr/local/share/lilypond/2.21.3/python', '/home/jean/repos/lilypond/build/scripts/out', '/home/jean/repos/lilypond/build/out/bin', '/home/jean/repos/lilypond/python', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/jean/.local/lib/python3.8/site-packages', '/home/jean/repos/frescobaldi', '/home/jean/repos/matplotlib/lib', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages']
convertrules module is taken from
/usr/local/share/lilypond/2.21.3/python/convertrules.py (which is made clear by printing
convertrules.__file__). The relocate preamble apparently inserts path entries in the wrong place:
/usr/local/share/ is taking priority over
/home/jean/repos/lilypond/python/. In fact,
sudo make uninstall errors out, yet fixes the problem (that is,
out/bin/convert-ly outputs a message starting with "AAA").
This does of course not happen when running the scripts directly (that is,
python3 scripts/convert-ly.py instead of
Still, it is an argument for stopping
@ substitutions and running Python scripts directly from source, reading the
VERSION file at runtime. To get rid of
@[email protected], we'd need to put everything under either
build/, and use relative imports. Apart from changing paths to the build scripts in make files, the tricky part is how all this will we handled at installation. GUB apparently expects the relocate preamble so it'd need a fix, and additionnally,
sudo make install should put symlinks to the scripts in
/usr/bin/. On Windows, we'll need wrappers written in Python (or bat) so the story is complicated.
Another option would be to ship
pip (which are pure Python tools) and use them at installation to create the appropriate executables.
Yet another solution would be to split these scripts out of the installers entirely and distribute them on PyPI. Thus, they could evolve with PyPI dependencies such as python-midi. Given that Frescobaldi can easily specify them as dependencies (since it already has some dependencies on PyPI like python-ly), this relies on the assumption that users who run them in terminals are capable enough to run
pip install abc2ly for example.
In short: this needs reflection.