Portmod issueshttps://gitlab.com/portmod/portmod/-/issues2023-12-31T23:35:10Zhttps://gitlab.com/portmod/portmod/-/issues/428Unable to trace flag when flag is enabled in IUSE, present in use.alias.yaml,...2023-12-31T23:35:10ZJacob BriggsUnable to trace flag when flag is enabled in IUSE, present in use.alias.yaml, and package is brought in as a dependency[flag]## Summary
Enabling a USE flag by default by way of `+foo` in `IUSE` causes an `unable to trace flag` exception if the flag is also present in `use.alias.yaml` and the package is being merged as a dependency of another package which req...## Summary
Enabling a USE flag by default by way of `+foo` in `IUSE` causes an `unable to trace flag` exception if the flag is also present in `use.alias.yaml` and the package is being merged as a dependency of another package which requires a use flag enabled.
## Steps to Reproduce
Add a `+` in front of any USE flag in any package's `IUSE` as long as that USE flag is also present in `use.alias.yaml`. Attempt to merge a package that depends on it and also specifies any USE flag of that package to be enabled.
_some-package_ `IUSE="+foo"`
_package-depending_ `RDEPEND="some-package[bar]"`
_use.alias.yaml_ `foo:baz`
## Current Behaviour
Raises an `unable to trace flag` exception and disables all use flags not explicitly enabled by _package-depending_ `RDEPEND`
<details>
<summary>Inquisitor</summary>
```plaintext
$ inquisitor test-install openmw *pybuild
Initialized prefix test0
Adding repository openmw to /home/jacobi/.config/portmod/test0/portmod.conf
Available profile symlink targets (openmw):
[0] default/openmw/2.0 (deprecated)
[1] default/openmw/2.0/morrowind (deprecated)
[2] default/openmw/2.0/morrowind-bm (deprecated)
[3] default/openmw/2.0/morrowind-tb (deprecated)
[4] default/openmw/2.0/morrowind-tb-bm (deprecated)
[5] openmw/3.0 (stable)
[6] openmw/3.0/morrowind-tb-bm (stable)
[7] openmw/3.1 (experimental)
[8] openmw/3.1/morrowind-tb-bm (experimental)
Using profile /home/jacobi/.local/share/portmod/repos/openmw/profiles/default/openmw/2.0
Testing configuration USE="-rebirth"
Adding flag -rebirth to virtual/siege-at-firemoth-1.0::openmw in /home/jacobi/.config/portmod/test0/package.use
Calculating Dependencies…
Done!
Removing flag -rebirth from virtual/siege-at-firemoth-1.0::openmw in /home/jacobi/.config/portmod/test0/package.use
Removing directory /home/jacobi/.config/portmod/test0
Removing directory /home/jacobi/.local/share/portmod/test0/var
Removing prefix test0 from prefix file
Traceback (most recent call last):
File "/usr/lib/python3.11/site-packages/portmod/_deps/__init__.py", line 517, in resolve
source_trace, _ = find_conflict(flag_config + formula.clauses)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/portmod/_deps/__init__.py", line 218, in find_conflict
raise DepError("Internal error: Unable to find conflict!")
portmod._deps.DepError: Internal error: Unable to find conflict!
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python-exec/python3.11/inquisitor", line 8, in <module>
sys.exit(main())
^^^^^^
File "/usr/lib/python3.11/site-packages/portmod/_cli/inquisitor.py", line 655, in main
args.func(args, repo_root, err)
File "/usr/lib/python3.11/site-packages/portmod/_cli/inquisitor.py", line 524, in test_install_cli
test_install(load_file(file))
File "/usr/lib/python3.11/site-packages/portmod/_cli/inquisitor.py", line 467, in test_install
merge([package.ATOM], select=False, io=CLIMerge())
File "/usr/lib/python3.11/site-packages/portmod/merge.py", line 311, in merge
transactions = resolve(
^^^^^^^^
File "/usr/lib/python3.11/site-packages/portmod/_deps/__init__.py", line 523, in resolve
raise Exception(message) from error
Exception: Internal Error: Unable to trace flag adamantium-armor for package meta-misc/umopp-naturalized-1.0::openmw
```
</details>
<details>
<summary>Portmod</summary>
```
$ portmod testing merge virtual/siege-at-fortmoth
Calculating Dependencies…
Done!
WARNING: Internal Error: Unable to trace flag aoe-arrows for package meta-misc/umopp-naturalized-1.0::openmw
WARNING: Internal Error: Unable to trace flag adamantium-armor for package meta-misc/umopp-naturalized-1.0::openmw
WARNING: Internal Error: Unable to trace flag entertainers for package meta-misc/umopp-naturalized-1.0::openmw
WARNING: Internal Error: Unable to trace flag bitter-coast-sounds for package meta-misc/umopp-naturalized-1.0::openmw
```
</details>
## Portmod and Prefix Information
<details>
<summary>prefix info</summary>
```plaintext
Portmod 2.7.3
Python 3.11.7 (main, Dec 28 2023, 03:09:37) [GCC 13.2.1 20230826]
Repositories:
LocalRepo(name='python'
sync_type='git'
sync_uri='https://gitlab.com/portmod/python.git'
priority=-999
location='~/.local/share/portmod/repos/python'
auto_sync=True)
Timestamp: 2023-05-12 07:41:53-04:00
Head commit: 7ee2e2a5da781e41a7cee0e972aa2f4ff6d0b1fa
LocalRepo(name='openmw'
sync_type='git'
sync_uri='https://gitlab.com/portmod/openmw-mods.git'
priority=-1000
location='~/.local/share/portmod/repos/openmw'
auto_sync=False)
Timestamp: 2023-12-29 15:36:42-06:00
Head commit: dd7b6a2fb9f72bcf8e12380b89a649c0b2fc2916
LocalRepo(name='meta'
sync_type='git'
sync_uri='https://gitlab.com/portmod/meta.git'
priority=-1000
location='~/.local/share/portmod/repos/meta'
auto_sync=True)
Timestamp: 2023-03-08 20:50:56-05:00
Head commit: c38df0c0376116472b5abb3a2db2a0c4d2f79702
base/morrowind: 1.6.1820-r3 USE="bloodmoon tribunal"
bin/7z: 22.01
modules/configtool: 0.11.2 USE="-grass" MAP="normal specular terrain_normal terrain_specular"
ACCEPT_KEYWORDS = openmw
ACCEPT_LICENSE = * -@EULA
ARCH = openmw
ARCH_VERSION = None
INSTALL_DEST = pkg/{CATEGORY}/{PN}
MODULEPATH = modules
OPENMW_CONFIG_DIR = ~/.config/openmw:~/.var/app/org.openmw.OpenMW/config/openmw/
PLATFORM = linux
PORTMOD_MIRRORS = https://gitlab.com/portmod-mirrors/openmw/-/raw/master/
PYTHONPATH = ~/.local/share/portmod/testing/lib/python
TEXTURE_SIZE = min
USE = -atlasgen atlas bloodmoon sunrays tribunal
TMP_DIR = /tmp/portmod
CACHE_DIR = ~/.cache/portmod
CONFIG_DIR = ~/.config/portmod/testing
ROOT = ~/.local/share/portmod/testing
```
</details>
The packages I used for testing are from my forked branch of the repo, but I don't see any reason why this issue wouldn't pop up on the main branch given the same conditions
## Possible fixes
I unfortunately am not familiar enough with Rust or Python to be of much further usehttps://gitlab.com/portmod/portmod/-/issues/427Tie search MultiSelect prompt into Tantivy2023-11-03T22:12:32ZPopeRigbypoperigby@mailbox.orgTie search MultiSelect prompt into TantivyI'd like to be able to fetch the next `n` items in the Tantivy index when I scroll down in the prompt generated by `portmod <prefix> search <term>`. Not sure if this is possible, but it might be a good idea to look into.I'd like to be able to fetch the next `n` items in the Tantivy index when I scroll down in the prompt generated by `portmod <prefix> search <term>`. Not sure if this is possible, but it might be a good idea to look into.https://gitlab.com/portmod/portmod/-/issues/426Use clap for CLI2023-11-03T16:27:35ZPopeRigbypoperigby@mailbox.orgUse clap for CLI> I can't think of any major arguments against clap. It would probably need a CLI entry point that just calls directly into a rust extension "main" function (I'm not sure if a pure rust executable would integrate well with python's packa...> I can't think of any major arguments against clap. It would probably need a CLI entry point that just calls directly into a rust extension "main" function (I'm not sure if a pure rust executable would integrate well with python's packaging system, but we could possibly do that and use pyo3 to call into portmod as a library). I think it would be necessary to use both the builder and derive APIs, since I don't think you can have runtime-generated subcommands with the derive API. That said, I think most of our CLI is underneath the generated subcommands, but maybe you can re-use the derive structures for the parts underneath (e.g. one Merge subcommand via a struct which gets added to all the prefix subcommands added through the builder).
Note to self: Prefix subcommands need to be built with builder API, probably.PopeRigbypoperigby@mailbox.orgPopeRigbypoperigby@mailbox.orghttps://gitlab.com/portmod/portmod/-/issues/425CI Warnings2023-10-29T17:30:13ZBenjamin Wingerbmw@disroot.orgCI WarningsThere are lots of CI warnings being produced, particularly in the windows CI E.g.: https://ci.appveyor.com/project/portmod/portmod/builds/47724074/job/i5p5hjupymm9k8jv
We should probably treat those as errors so they are more evident, a...There are lots of CI warnings being produced, particularly in the windows CI E.g.: https://ci.appveyor.com/project/portmod/portmod/builds/47724074/job/i5p5hjupymm9k8jv
We should probably treat those as errors so they are more evident, and address them.https://gitlab.com/portmod/portmod/-/issues/424Windows language identifiers2023-10-29T17:30:13ZBenjamin Wingerbmw@disroot.orgWindows language identifiersThe following error is very common on windows. Evidently the interface we're using to get language information is not providing the expected type of language identifier.
```
C:\projects\portmod\portmodlib\l10n.py:52: UserWarning: User l...The following error is very common on windows. Evidently the interface we're using to get language information is not providing the expected type of language identifier.
```
C:\projects\portmod\portmodlib\l10n.py:52: UserWarning: User language identifier "English_United States" could not be parsed
result = l10n_lookup(default, msg_id, kwargs)
```https://gitlab.com/portmod/portmod/-/issues/423Windows test hangs2023-10-29T17:30:13ZBenjamin Wingerbmw@disroot.orgWindows test hangsWindows tests keep hanging, so I'm disabling them for now (!595). It seems to be an issue with cleaning up test files.Windows tests keep hanging, so I'm disabling them for now (!595). It seems to be an issue with cleaning up test files.https://gitlab.com/portmod/portmod/-/issues/422Can't init prefix2023-09-04T07:59:37ZBart VaesCan't init prefix## Summary
Installed latest portmod via pip3 from gitlab : pip3 install git+https://gitlab.com/portmod/portmod
After installing portmod, try to create new prefix with "init" sub-command and openmw arch, gives "Reference at 'refs/heads/ma...## Summary
Installed latest portmod via pip3 from gitlab : pip3 install git+https://gitlab.com/portmod/portmod
After installing portmod, try to create new prefix with "init" sub-command and openmw arch, gives "Reference at 'refs/heads/master' does not exist" error.
Using openSUSE Tumbleweed linux, python version = 3.11.4.
## Steps to Reproduce
Install portmod via pip3 install git+https://gitlab.com/portmod/portmod
Execute: portmod init openmw openmw
## Current Behaviour
<details><summary>Output</summary>
```
portmod init openmw openmw
Before a prefix can be used, it needs package repositories, and a configuration profile
Portmod's only hardcoded package repository, the meta repository, serves primarily
to introduce other repositories, so you will need to select repositories from those
which match your architecture.
For many architectures, there is only one such repository, in which case it will be selected automatically
A profile provides a set of configuration options for a prefix
Profiles are provided by repositories, so the available profiles may depend on the repositories you choose
Available Repositories
[0] openmw: The OpenMW Mod Package Repository (stable https://gitlab.com/portmod/openmw-mods.git) *
The only repository available for arch openmw is openmw. Adding…
portmod openmw select repo add openmw
portmod sync openmw
Syncing repo openmw…
Traceback (most recent call last):
File "/home/bart/.local/lib/python3.11/site-packages/portmod/_cli/main.py", line 335, in main
args.func(args)
File "/home/bart/.local/lib/python3.11/site-packages/portmod/_cli/init.py", line 70, in init
sync([repos[x] for x in selected])
File "/home/bart/.local/lib/python3.11/site-packages/portmod/sync.py", line 49, in sync
current = gitrepo.head.commit
^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/git/refs/symbolic.py", line 226, in _get_commit
obj = self._get_object()
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/git/refs/symbolic.py", line 219, in _get_object
return Object.new_from_sha(self.repo, hex_to_bin(self.dereference_recursive(self.repo, self.path)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/git/refs/symbolic.py", line 159, in dereference_recursive
hexsha, ref_path = cls._get_ref_info(repo, ref_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/git/refs/symbolic.py", line 210, in _get_ref_info
return cls._get_ref_info_helper(repo, ref_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/site-packages/git/refs/symbolic.py", line 193, in _get_ref_info_helper
raise ValueError("Reference at %r does not exist" % ref_path)
ValueError: Reference at 'refs/heads/master' does not exist
ERROR: Reference at 'refs/heads/master' does not exist
```
</details>
## Portmod and Prefix Information
```
portmod --version
Portmod 2.6.3.dev16+gc03ec39
```
</details>
## Possible fixeshttps://gitlab.com/portmod/portmod/-/issues/421Git clone not working during merge on WSL2023-06-03T23:05:03ZEriranGit clone not working during merge on WSLWhen trying to merge openmw i-heart-vanilla or i-heart-vanilla-directors-cut metamods, cloning Git repository fails during their installation. This is currently happening on Pelagiad font (https://github.com/Isaskar/Pelagiad). I have set...When trying to merge openmw i-heart-vanilla or i-heart-vanilla-directors-cut metamods, cloning Git repository fails during their installation. This is currently happening on Pelagiad font (https://github.com/Isaskar/Pelagiad). I have set up my own SSH credentials for Github and `git clone git@github.com:Isaskar/Pelagiad.git` on its own works without any issues.
I am using WSL Ubuntu 22.04 in Windows 10 Pro. The subsystem is fairly fresh as I reset it while attempting to solve this
I could not figure out if I was missing some OS packages or other dependencies that the Portmod Git class uses. Any clue what could be causing this?
```
portmod openmw merge i-heart-vanilla-directors-cut
...
>>> Cleaned up /mnt/c/games/PortmodTmp/portmod/media-fonts/dejavu-2.37
>>> Starting installation of media-fonts/pelagiad-0_p20170803
>>> Unpacking package…
Cloning into 'Pelagiad'...
fatal: unable to access 'https://github.com/Isaskar/Pelagiad.git/': Could not resolve host: github.com
Traceback (most recent call last):
File "/mnt/c/games/OpenMWPortMod/var/db/common/git/git-1.0-r1.pybuild", line 45, in src_unpack
self.execute(f"git clone {repo}")
File "/home/erik/.local/lib/python3.10/site-packages/portmodlib/execute.py", line 34, in execute
proc = subprocess.run( # nosec B603
File "/usr/lib/python3.10/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'clone', 'https://github.com/Isaskar/Pelagiad.git']' returned non-zero exit status 128.
Successfully merged 25 packages before failure.
Error occurred when attempting to merge media-fonts/pelagiad-0_p20170803::openmw
ERROR: The unpack phase of the pybuild /home/erik/.local/share/portmod/repos/openmw/media-fonts/pelagiad/pelagiad-0_p20170803.pybuild failed.
Do not report this error message by itself. There will be another error message from the sandboxed code for the actual problem, probably directly above this one.
```https://gitlab.com/portmod/portmod/-/issues/420Whitelisting of module updates to config files can hide bugs2023-05-12T12:07:51ZBenjamin Wingerbmw@disroot.orgWhitelisting of module updates to config files can hide bugsI'm not positive, but for configtool#16 if a user whitelisted `settings.cfg` the first time when it was empty, the issue wouldn't have been visible, and then they wouldn't have had to manually review subsequent changes where progressivel...I'm not positive, but for configtool#16 if a user whitelisted `settings.cfg` the first time when it was empty, the issue wouldn't have been visible, and then they wouldn't have had to manually review subsequent changes where progressively more data would be written. That would have hidden the issue until it became out of control and harder to diagnose.
I'm not convinced the convenience of whitelisting files outweighs the potential for similar problems. Users should be paying attention to what portmod is doing, and even though output is still displayed for whitelisted files, it makes it easier to ignore without a prompt.https://gitlab.com/portmod/portmod/-/issues/419Fixing Test Fixtures2023-04-10T01:47:01ZBenjamin Wingerbmw@disroot.orgFixing Test FixturesThe tests using fixtures at the moment aren't really using them properly, since most of information is global state and changes when other tests are run and we're just depending on the global state instead of actually using the "fixed" d...The tests using fixtures at the moment aren't really using them properly, since most of information is global state and changes when other tests are run and we're just depending on the global state instead of actually using the "fixed" data produced by the fixture. I guess since tests use a different directory every time setup_env sets up, it might be possible to share the file data more globally and just switch the global variables to point at a particular test directory when a particular test runs (e.g. have each fixture setup files and return the test directory path, then have each test using it call a function which updates the globals to refer to the stuff in the test directory. We could then use just a few shared test environments which are shared between the test modules rather than having separate ones for each module).
This could speed tests up somewhat since we could setup test environments less frequently, and share fixtures across the entire test session, instead of setting up custom fixtures in each test suite.
Note that tests which install stuff as part of the test probably should be more isolated, or at least should use separate fixtures from the ones which are supposed to actually be read-only and clean up after themselves when they are done.
This is relatively low priority, but could speed up running tests noticeably (and could considerably speed up the windows CI given that it's package installation (more particularly sandbox setup) which is so slow, which this aims to de-duplicate).https://gitlab.com/portmod/portmod/-/issues/417GUI write features2023-04-17T18:31:21ZBenjamin Wingerbmw@disroot.orgGUI write featuresCurrent state is that a basic read-only GUI has been implemented with searching and display of installed packages. The parts which modify stuff are already in-progress by @poperigby as far as I know, and include (not exhaustive):
- [ ] ...Current state is that a basic read-only GUI has been implemented with searching and display of installed packages. The parts which modify stuff are already in-progress by @poperigby as far as I know, and include (not exhaustive):
- [ ] Prefix creation (analogous to `portmod init`)
- [ ] Merging packages (`portmod <prefix> merge`)
- [ ] Synching repositories (`portmod sync`)
- [ ] Modifying use flags (`portmod <prefix> use`, though see below)
- [ ] Updating protected files (`portmod cfg-update`)
I think it's worth noting that configuration editing (including use flags, but also `portmod.conf`, `repos.cfg`, `package.accept_keywords`, etc.) can already be done in a text editor, and while a builtin editor might be helpful if done well, it's probably lower priority than the rest, and a reasonable compromise for now could be buttons for opening those files in an external editor.PopeRigbypoperigby@mailbox.orgPopeRigbypoperigby@mailbox.orghttps://gitlab.com/portmod/portmod/-/issues/415Mixing superclass versions2023-03-31T20:50:11ZBenjamin Wingerbmw@disroot.orgMixing superclass versionsI don't think this really works at the moment with `Pybuild1` and `Pybuild2`, but it occurred to me that for things like the `MW` class in the openmw repo, individual packages which inherit from it should be able to choose which pybuild ...I don't think this really works at the moment with `Pybuild1` and `Pybuild2`, but it occurred to me that for things like the `MW` class in the openmw repo, individual packages which inherit from it should be able to choose which pybuild version they implement. Even when it comes to Pybuild1, the main issue was getting everything to use the new configtool VFS rather than portmod's deprecated builtin one, and fundamentally there's no problem with a package using Pybuild1 for installation, as long as it also inherits from `MW` and lets that set up the json config files which configtool needs. Admittedly there are other issues with `Pybuild1`, so it's not the best example, so I'm going to use the not yet implemented `Pybuild3` below instead.
E.g.
```python
class Pybuild2:
def src_install(self):
# For the purposes of this, doesn't do anything
pass
class Pybuild3(Pybuild2):
def src_install(self):
# For the purposes of this, doesn't do anything
pass
class MW(Pybuild2):
def src_install(self):
...
class Package(Pybuild3, MW):
...
```
For the purposes of this, Pybuild3 being a subclass of Pybuild2, but not the other way around, works well. It means that MW should target the lowest supported version, and then if you try to use something older with it you should get an mro error.
On the other hand, it means that Pybuild3's (empty) `src_install` gets priority over `MW`'s. This isn't necessarily bad, but it's unhelpful if all you want Pybuild3 for is, for example, #363.
This could be more or less solved if Pybuild3 was to call `super().src_install()`, however this isn't necessarily desirable if there are multiple install-related functions in the mro. E.g. if Pybuild3 were to include a new simplified default install (might be useful so that #395 can do something out of the box), then both that, and the install code from `MW` would be run, which might not be very helpful either. But we can probably work around that as long as the interfaces are distinct (e.g. if the default install from Pybuild3 uses a field which is left empty, then it won't be an issue). It's not like it's not already possible to install things in multiple ways and ignore when they have conflicting interfaces.
Keeping things tightly-focused will help here too. `MW` isn't a very good example as it both does installation, and sets up config files for configtool, and provides custom unpack functionality, and you can't separate any of them at the moment. Most of this could be solved if those things were separated and added on top of your choice of `Pybuild3` or `Pybuild2`, though even there we can run into MRO issues depending on the superclasses and their order.
And documentation is crucial the more options are provided. If every mod package inherits from just `MW` and possibly a mixin like `Git` or `NexusMod`, then it's relatively straightforward, but the more options there are the more it's going to be necessary to document specifically what each class does, and what they would be compatible with. I think there used to be something mentioned on the wiki, but I think it would be much more helpful to provide generated APIdocs (and there could also be guides on the wiki when helpful).Pybuild3https://gitlab.com/portmod/portmod/-/issues/414Displaying source of use flag changes2023-03-28T21:08:03ZBenjamin Wingerbmw@disroot.orgDisplaying source of use flag changesThere is a field `oldvalue` on the `UseDep` class, which used to be used to display when an automated change would override something which is already set. It's no longer actually being set anywhere, but it also wan't very helpful as it ...There is a field `oldvalue` on the `UseDep` class, which used to be used to display when an automated change would override something which is already set. It's no longer actually being set anywhere, but it also wan't very helpful as it didn't display where the flag was set from.
It should be replaced with a system that tracks the source, so it can annotate changes with "Set via global USE in portmod.conf", "Set via global USE in profile", "Set via package.use", etc.https://gitlab.com/portmod/portmod/-/issues/413Maybe make `get_total_download_size` return a more human readable size?2023-04-04T20:00:27ZPopeRigbypoperigby@mailbox.orgMaybe make `get_total_download_size` return a more human readable size?In !576, I made `get_total_download_size` return bytes. For the GUI, I created a [script](https://gitlab.com/portmod/portmod/-/blob/50e6f717261c36fbb3f117072a9efb3b3c154304/portmod/_gui/human_bytes.py) that would convert the bytes to a m...In !576, I made `get_total_download_size` return bytes. For the GUI, I created a [script](https://gitlab.com/portmod/portmod/-/blob/50e6f717261c36fbb3f117072a9efb3b3c154304/portmod/_gui/human_bytes.py) that would convert the bytes to a more human readable format (KiB, MiB, or GiB.)
But, I was thinking that it should probably be included with `get_total_download_size` (having two separate functions to convert to a human readable format for the GUI and CLI doesn't make a whole lot of sense), or we should at least have a helper function that converts the bytes.
I can do this by the way, I just wanted to get your opinion on this admittedly trivial issue before I created an MR.PopeRigbypoperigby@mailbox.orgPopeRigbypoperigby@mailbox.orghttps://gitlab.com/portmod/portmod/-/issues/409Licenses of bundled dependencies2023-03-17T17:53:06ZBenjamin Wingerbmw@disroot.orgLicenses of bundled dependenciesWhen thinking about pyinstaller (#36) again for dealing with installation particularly on windows (https://gitlab.com/portmod/portmod/-/merge_requests/541#note_1311813366), it occurred to me that distributing pyinstaller builds will requ...When thinking about pyinstaller (#36) again for dealing with installation particularly on windows (https://gitlab.com/portmod/portmod/-/merge_requests/541#note_1311813366), it occurred to me that distributing pyinstaller builds will require double-checking the licenses for everything being bundled.
Furthermore, we should have been doing this already for the pre-built wheels, as they bundle Rust dependencies. It looks like [cargo-bundle-licenses](https://github.com/sstadick/cargo-bundle-licenses) and [cargo-about](https://github.com/EmbarkStudios/cargo-about) can be used for this purpose.
[pip-licenses](https://pypi.org/project/pip-licenses/) looks like it would work for python dependencies.https://gitlab.com/portmod/portmod/-/issues/408Importing/exporting of mod lists2023-03-16T16:12:40ZPopeRigbypoperigby@mailbox.orgImporting/exporting of mod listsI've found myself wanting to share mod lists a few times, but (as far as I know) this currently requires either creating a shell script to run a series of Portmod commands (not very cross-platform portable, and tedious) or creating a met...I've found myself wanting to share mod lists a few times, but (as far as I know) this currently requires either creating a shell script to run a series of Portmod commands (not very cross-platform portable, and tedious) or creating a metapackage (not beginner friendly, and time consuming.)
I was thinking it would be cool if Portmod could generate a file with the list of the all the installed packages (excluding system packages), and their selected use flags. Then for somebody on the import side, Portmod could just go through the file line-by-line to install each package with the right use flag.https://gitlab.com/portmod/portmod/-/issues/407Pybuild String Fields2023-03-31T21:56:13ZBenjamin Wingerbmw@disroot.orgPybuild String FieldsThere are a bunch of fields in `BasePybuild` which should be moved to `FullPybuild` and avoided internally. I'd rather use structured data which displays as a string than manipulate strings internally, and to that end a Version class was...There are a bunch of fields in `BasePybuild` which should be moved to `FullPybuild` and avoided internally. I'd rather use structured data which displays as a string than manipulate strings internally, and to that end a Version class was introduced some time ago, and the current string-based Atom class will also be replaced (#397).
They can't be removed entirely as the packaging environment depends on them being set on `Pybuild1` and `Pybuild2`. It might also make sense to have them be a property of `Pybuild2` specifically rather than `FullPybuild`.
Initially this includes `PV`, `PR` and `PVR`, but will also include `P`, `PN`, `CPN`, `CATEGORY`, `PF` and `CP` with the implementation of #397.https://gitlab.com/portmod/portmod/-/issues/406Download Directory for WSL2023-03-03T15:44:46ZBenjamin Wingerbmw@disroot.orgDownload Directory for WSLIt would be easier to set the download directory to use the host Windows system's one if it was configurable in `portmod.conf`, rather than only as an environment variable. We could add `DOWNLOADS` to the list of config variables exporte...It would be easier to set the download directory to use the host Windows system's one if it was configurable in `portmod.conf`, rather than only as an environment variable. We could add `DOWNLOADS` to the list of config variables exported as environment variables, which should let it be set either by the environment or in `portmod.conf`.
Since some environment variables are shared with the Windows system, it might also be possible to use a windows-specific environment variable to detect downloads whether or not we're in WSL on windows (`$HOMEPATH/Downloads` might work), but I suppose that depends on the default contents of [WSLENV](https://devblogs.microsoft.com/commandline/share-environment-vars-between-wsl-and-windows/).https://gitlab.com/portmod/portmod/-/issues/405Environment Variables Overriding Config Settings2023-03-17T17:16:42ZBenjamin Wingerbmw@disroot.orgEnvironment Variables Overriding Config SettingsThis (title) is mentioned in the docs, and I think it might have once been true, but environment variables are currently used as input to the profile, so if something is set in the profile (or the user's `portmod.conf`) it can't really b...This (title) is mentioned in the docs, and I think it might have once been true, but environment variables are currently used as input to the profile, so if something is set in the profile (or the user's `portmod.conf`) it can't really be overridden by an environment variable.
Similarly, the variables that "accumulate" like `USE` will only apply when set as environment variables if the flag isn't already being set in the profile, which is confusing and requires users knowing about profile internals to understand certain behaviours.
At minimum, the docs should be fixed to reflect the current reality (i.e. set everything in `portmod.conf`, and that environment variables may be overridden).
It would also be possible to only set things in the profile as defaults, but that's a little messy, and there isn't really a way to write a function in the profile that could simplify it (though portmod could provide such a function to the profile environment). E.g.
```python
try:
OPENMW_CONFIG_DIR
except:
OPENMW_CONFIG_DIR = "..."
```
Environment variables could also be separated from configuration variables, and an `environ` dictionary be provided to the profile to let it do things like modify `PATH` in the sandbox. The config could then be merged with the modified `environ` object's environment variables (with the latter taking priority) when environment variables are passed to the sandbox. (This should be done with a new "profile_version" setting in `layout.conf`, to make sure that the change doesn't break existing profiles).https://gitlab.com/portmod/portmod/-/issues/399Default working directory2023-04-25T19:06:11ZBenjamin Wingerbmw@disroot.orgDefault working directoryThe current behaviour is documented incorrectly [here](https://portmod.readthedocs.io/en/v2.5.6/dev/pybuild/pybuild.html#pybuild.Pybuild2.S). In reality, `Pybuild2.S` falls back to `Pybuild2.P` if there are no sources (enabled) in `SRC_U...The current behaviour is documented incorrectly [here](https://portmod.readthedocs.io/en/v2.5.6/dev/pybuild/pybuild.html#pybuild.Pybuild2.S). In reality, `Pybuild2.S` falls back to `Pybuild2.P` if there are no sources (enabled) in `SRC_URI`, but the working directory will not.
For the moment, the documentation at the minimum needs to be updated. The code should also be corrected such that the working directory falls back to `Pybuild2.P` to match `Pybuild2.S`, as I'm fairly sure that won't break anything. It may also be useful to correct or create a more helpful error in the case that the working directory does not exist.
This behaviour is somewhat different in the [PMS section 9.1.1](https://projects.gentoo.org/pms/7/pms.html#x1-860009.1.1), primarily since `S` does not default to the first source's extracted path there, however it specifies that an error should be raised by the package manager if the directory does not exist prior to phase function execution (with certain helpful conditions), and I think that would be useful to provide more helpful error messages, rather than just having errors arise from the directory not existing in the packaging environment.
I also think that for Pybuild3 the `S` variable should be renamed to something more intuitive. It's worth noting that `WORKDIR` is very similar but has a subtly different meaning, and they may need to both be renamed.
- [ ] Update docs
- [ ] Fix working directory to match `S`
- [ ] Error message prior to phase function if working directory doesn't exist
- [ ] Pybuild3 renamePybuild3