Project direction 2020: base on Alpine stable, get serious about supported devices
This is somewhat of a follow-up to last year's big project direction post, but this time I did not come up with it all on my own. Quite the contrary, the core team has a monthly phone call now, in which we talk one or two hours about moving postmarketOS forward and this is basically what we came up with. In addition, and probably because of that fact, the ideas are more technically mature and ready to be implemented. Let me just write it all together in a big picture, and fill in the blanks as necessary. Further below are detailed TODOs, which will link to separate issues, as we approach them. Get comfortable in your chair and grab a drink, here goes a long-read!
Where we are now
Since the last direction post, we had a huge amount of commits in pmaports.git and pmbootstrap.git obviously, but I'm not going to recount those here as this would blow up this issue. But to focus on what is important for the overall project direction, the biggest change has been, that I am not building the binary packages at home anymore - they are built on sourcehut with bpo. Previously all merge requests would only be merged by yours truly (as only I could see whether builds fail or not after merging), but now the build logs are transparently online and we have more people merging the MRs - this allows the whole project to scale (see Alpine stable goal below) and me to work less on MRs and more on tasks like this one.
We have dropped several UIs that were not well maintained and all remaining UIs have maintainers.
On the hardware side, both the Pinephone and the Librem 5 are delivering the first prototypes to people who ordered them these days (afaik Librem 5 has already, and Pinephone is either in December or January). Both phones are made for Linux distributions, running a (close to) mainline kernel, free software where possible, and not just Android with a downstream kernel fork and tons of blobs like your regular smartphone. This is great for postmarketOS, because it makes a daily driver goal more realistic. More on that below.
Notably the Pinephone expects to be ready for daily usage around March 2020 (that's what I've gathered from a recent interview in the Linux Unplugged Podcast), which is a reasonable timeframe for getting postmarketOS in shape for daily usage - at least for basic tasks!
Road to daily driver in 2020
Base on Alpine's stable releases
Moved to milestone: Alpine stable
Devices in separate repository
After discussion below, this idea was abandoned: devices will not go into a separate repository for now.
Devices should go into a separate repository. This allows using any device with both the latest channels, or with edge (as long as they don't depend on device-independent features in edge). Without the downside of keeping all devices duplicated in two or even three branches at the same time.
Each device should have the minimum compatible postmarketOS channels, that it was tested with, in its deviceinfo, so let's introduce a new variable:
In case a device does depend on a feature that is only available in edge, it would simply put the next version there (latest + 1), so if latest is at 3.10, this would be 3.11. For consistency with pmbootstrap.git and pmaports.git, let's call the new repository pmdevices.git.
Get serious about supported devices
- Moved to milestone: device categorization
- Not sure about the deviceinfo variable, we have a directory structure now that indicates the category
Right now we don't have any devices that we claim are actually "supported" on the homepage. We do have two devices, that are highlighted however, and these are the Nokia N900 and the Nexus 5. Both have great mainline support and what is possible with postmarketOS on them is impressive considering the downstream kernel and nonfree userspace drivers they come from. However, in the current state, these ports are not usable as daily drivers. So let's take a look at the up-and-coming devices from Pine64 and Purism:
- Librem 5
Since they are made for running Linux, and we can collaborate and build on work from related Linux smartphone projects, it should be reasonable to get the following features working:
- Calls (not for tablet)
- SMS (not for tablet)
- Usable phone UI (Plasma Mobile, possibly Phosh)
- Battery charging
"Nice to have" (points above must definitively be prioritized!):
So we're going to approach this from a realistic angle. Start with two or three supported devices, by name the two Pine64 devices (team members should have enough of those and be able to maintain them), and if we find somebody who has one and is able to maintain it, the Librem 5 (possibly @craftyguy?). If in doubt, just start with two or even just the PinePhone, but have that one working reliably with a small feature set and expand from there.
Let's add a new deviceinfo variable, where we can see if a device is officially supported or not. Think of all other devices like the AUR - still useful, but sometimes broken and you need to know what you are doing when using these. (Unsupported devices should also have this variable in the future, so it's explicit.)
I hope that in the future, we can expand the supported devices to some of our favorite old-but-gold ones like the Nexus 5. But this one specifically doesn't have audio working, so it's out of question for now. Having postmarketOS working as daily driver for at least one phone would be a game changer, which is likely to lead to new hackers showing up that eventually lead to making old devices supported. So while the device selection is tiny, consider this a catalyst for the whole project that will let it grow a lot more, for all devices - after all, the whole architecture of postmarketOS is designed to have as few device specific code as possible!
More sophisticated approach, that allows more than two values ("supported, "unsupported") is in #16 (closed).
Behind the scenes, it should clone pmdevices.git if it is missing (just like it clones pmaports.git) and add a symlink in the local clone of pmaports.git, so that pmaports/device and pmaports/firmware point to the new repository. This is consistent with how we handled the split of pmaports.git from pmbootstrap/aports to its own repository.
If pmaports.git is on the master branch, or on a known channel branch, it would ask the user which channel they would like to use, along with the available choices parsed from channels.cfg. On the first run, we recommend "latest" (that's the stable one regular users should be using).
During device selection, this should show a warning if the selected device is unsupported.
So we have yet-another-repository that developers, but also regular users, need to keep updated. Let's make it a little bit easier for both and provide a shorthand command for "git pull" in both pmaports.git and pmdevices.git.
This should check first, if the repository in question is on "master" or a channel branch, and only then run "git pull". Regular users will benefit most from this change, because then they don't need to familiarize with where the repositories are and how to update them - just run one command.
Basically a health check of your pmbootstrap setup, which would most importantly check if you need to update your pmaports.git and pmdevices.git repositories, and if so, directly point the user at "pmbootstrap pull". We had the idea for this, before the concept of pmdevices.git, but with the big picture at hand, this seems like the perfect addition to make the transition to the new repository as painfree as possible.
This should also run at the end of "pmbootstrap init" and also show if the device is supported or not.
Full issue: pmbootstrap#1829 (closed).
Again, see milestones for the up-to-date progress.
Here is what needs to be done in chronological order (unlike the above, which is thematically sorted to show what it builds up to). We'll add related issues in the future, for now I'll just add checkboxes unless we have an existing issue, so I can finish this post today.
pmbootstrap pullas shorthand for
pmaports.gitclone, if it is on
masterbranch (pmbootstrap#1858 (closed))
pmbootstrap statusto display whether pmaports.git must be updated, run it at the end of pmbootstrap init (pmbootstrap#1829 (closed))
pmbootstrap: support pmdevices.git (merge together with split, see below)
- pmbootstrap init: clone pmdevices.git and create "device" and "firmware" symlinks in pmaports.git clone
- pmbootstrap init: check pmdevices.git version from something like pmaports.cfg, just like we do for pmaports (so it can have a min pmbootstrap version, and we can require a min pmdevices.git version in pmbootstrap)
- pmbootstrap pull: also pull pmdevices.git
- pmbootstrap status: also display status of pmdevices.git
- pmbootstrap: create release
- require new pmaports version
pmaports: split repository into
- adjust CI in both repos
- bump pmaports version in pmaports.cfg
- require new pmbootstrap version in pmaports.cfg
- bpo: start building new packages when commits get pushed to pmdevices.git
- publish a blog post about this change (I'd like to get smaller, more frequent ones out)
Stable branch in pmaports.git (depends on pmdevices.git)
- pmaports: possibly get rid of all branches except for master (put into other repository and create wiki page documenting where they are now) - let's discuss this first
- pmaports: branch master to new branch named like current Alpine stable (3.11 probably at this point?)
pmaports: create "channels.cfg" branch and "channels.cfg" file, with
- pmaports: set up permissions, so that nobody can force-push to channels.cfg and 3.11 branches, just like master
- pmbootstrap: allow branch selection (default to "master" for now) (pmbootstrap#1855 (closed))
- pmbootstrap: check for apk version: this check exists already, but from now on it should check for different versions depending on the channel branch
- bpo: properly support building from multiple branches (build.postmarketos.org#63 (closed))
- bpo: build devices and firmware only from edge channel, use symlinks in all other channels
- bpo: configure to build latest channel branch (this will take some time...)
- pmbootstrap pull: also pull pmaports.git, when on channel branch from channels.cfg
- pmbootstrap: write postmarketOS channel (3.11) into /etc/os-release, instead of pmbootstrap version (1.x)
pmbootstrap: aportgen: add
deviceinfo_min_postmarketos_channelto each device, with CI test
deviceinfo_min_postmarketos_channelwhen selecting device and channel
- pmbootstrap: init: default to "latest" channel (!)
- document branch switching workflow from above in wiki
- blog post
Supported device selection (could be worked on in parallel to both above)
(see #16 (closed))
pmbootstrap: add support for
pmbootstrap: generate new devices into
consider adding a new(maybe later, don't see a use case right now)
deviceinfo_statusproperty or somehow generate it automatically
pmbootstrap init: show warning if device is in
testing(preferably a different one)
- pmbootstrap status: show device status
wiki: document requirements for
update wiki: separate devices by status (
- bonus (could be done later): display devices differently, e.g.
main: compare specifications
community: compare status, eventually display SoC
testing: just a plain list of devices(?)
- bonus (could be done later): display devices differently, e.g.
prepare homepage update (merge as one MR)
- pmdevices.git and stable branch changes from above must be completed first
- update homepage, so it doesn't feature the N900 and Nexus 5 anymore. Replace with devices that are supported.
- get rid of "phone calls don't work" (or reword it... should at least work on the supported devices)
- new blog post
pmbootstrap: add support for
- device should be selectable with an immediate error in pmbootstrap
- packages should not get built
- pmbootstrap: add some way to set a reason for unmaintained status, and display it when selecting the device
- make sure that upgrading always works properly (post-install scripts should not fail, provide upgrade paths if subpackages changed like with mesa in !896 etc.)
Next up for me is writing a short blog post related to build.postmarketos.org being in production. I wanted to get this issue written down first, so I could link to it from there, in case some readers are curious about what we have planned. After that, I'll switch between making necessary bpo improvements, and starting to implement the first points from this list (besides doing other necessary admin work).
The list above reflects how I would implement it. If somebody is interested in helping out, it would be great and certainly speeds up the process. Pick one of the easy starting points, like "pmbootstrap: add
pmbootstrap pull as shorthand"... above, or "pmbootstrap: aportgen: add
deviceinfo_officially_supported", then create an issue in which you state that you will work on it, and link to it from here (I'll try to do the same, so we don't have duplicate work). But please make sure that you don't take too long with it if it's blocking the next task.
Thanks for reading, and thanks to everybody for all the awesome contributions that lead us here!
Looking forward to comments.
- changed wording from release to channel where appropriate
- "firmware" dir also needs to move to pmdevices.git
- mention that we need something like pmaports.cfg in pmdevices.git (so we can check min pmbootstrap and pmdevices.git versions in pmbootstrap, just like we do with pmaports)
- bpo: add TODO for building devices/firmware only for one release and using symlinks
- add "Streamlining" section
- link to #16 (closed)
EDIT 2020-02-25: (by @Minecrell)
- rewrite steps for "supported device selection"
- rewrote regarding new milestones