Python scripts infrastructure
This came in !235 (closed).
When working on Python scripts, after editing some file under python/
, the changes will not be reflected in the script generated in out/bin/
after a new make when sudo make install
was previously run.
Steps to reproduce:
$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
$ # Edit files.
$ git diff
diff --git a/python/convertrules.py b/python/convertrules.py
index 5316839a8d..66a7222857 100644
--- a/python/convertrules.py
+++ b/python/convertrules.py
@@ -26,7 +26,7 @@ import sys
import lilylib
-NOT_SMART = "\n" + _ ("Not smart enough to convert %s.") + "\n"
+NOT_SMART = "\n" + _ ("AAA Not smart enough to convert %s.") + "\n"
UPDATE_MANUALLY = _ ("Please refer to the manual for details, and update manually.") + "\n"
FROM_TO = _ ("%s has been replaced by %s") + "\n"
diff --git a/scripts/convert-ly.py b/scripts/convert-ly.py
index 7dd9bcc0f9..8a707e8b43 100644
--- a/scripts/convert-ly.py
+++ b/scripts/convert-ly.py
@@ -36,6 +36,7 @@ gettext.install ('lilypond', '@localedir@')
import lilylib as ly
import convertrules
+print(convertrules.NOT_SMART)
lilypond_version_re_str = '\\\\version *\"([0-9.]+)"'
lilypond_version_re = re.compile (lilypond_version_re_str)
$ cd build
$ make
Making python/out/convertrules.pyc.dummy (py compile)
Making scripts/out/convert-ly (sed)
Making scripts/out/convert-ly.1
$ out/bin/convert-ly
Not smart enough to convert %s.
Usage: convert-ly [OPTION]... FILE [etc.]
A little debugging shows that when convert-ly
is run, sys.path
is
['/home/jean/repos/lilypond/build/out/lib/lilypond/current/python',
'/home/jean/repos/lilypond/build/out/share/lilypond/current/python',
'/usr/local/lib/lilypond/2.21.3/python',
'/usr/local/share/lilypond/2.21.3/python',
'/home/jean/repos/lilypond/build/scripts/out',
'/home/jean/repos/lilypond/build/out/bin',
'/home/jean/repos/lilypond/python',
'/usr/lib/python38.zip',
'/usr/lib/python3.8',
'/usr/lib/python3.8/lib-dynload',
'/home/jean/.local/lib/python3.8/site-packages',
'/home/jean/repos/frescobaldi',
'/home/jean/repos/matplotlib/lib',
'/usr/local/lib/python3.8/dist-packages',
'/usr/lib/python3/dist-packages']
and the convertrules
module is taken from /usr/local/share/lilypond/2.21.3/python/convertrules.py
(which is made clear by printing convertrules.__file__
). The relocate preamble apparently inserts path entries in the wrong place: /usr/local/share/
is taking priority over /home/jean/repos/lilypond/python/
. In fact, sudo make uninstall
errors out, yet fixes the problem (that is, out/bin/convert-ly
outputs a message starting with "AAA").
This does of course not happen when running the scripts directly (that is, python3 scripts/convert-ly.py
instead of build/out/bin/convert-ly
).
Still, it is an argument for stopping @
substitutions and running Python scripts directly from source, reading the VERSION
file at runtime. To get rid of @RELOCATE_PREAMBLE@
, we'd need to put everything under either python/
or build/
, and use relative imports. Apart from changing paths to the build scripts in make files, the tricky part is how all this will we handled at installation. GUB apparently expects the relocate preamble so it'd need a fix, and additionnally, sudo make install
should put symlinks to the scripts in /usr/bin/
. On Windows, we'll need wrappers written in Python (or bat) so the story is complicated.
Another option would be to ship setuptools
and pip
(which are pure Python tools) and use them at installation to create the appropriate executables.
Yet another solution would be to split these scripts out of the installers entirely and distribute them on PyPI. Thus, they could evolve with PyPI dependencies such as python-midi. Given that Frescobaldi can easily specify them as dependencies (since it already has some dependencies on PyPI like python-ly), this relies on the assumption that users who run them in terminals are capable enough to run pip install abc2ly
for example.
In short: this needs reflection.