Don't allow ssh password login
On postmarketOS, we tend to have numerical passwords because that's what the phone shells accept. Those passwords also tend to be quite short. At the same time, phones tend to connect to random public network and their modems will work out of the box more often. I'd say the device is pretty exposed, or at least more than usual Alpine installation that can hide behind the network firewall.
Given that ssh is activated by default and that if someone guess the ssh username+password he can get root by typing the password twice, I'd say it's pretty dangerous, too. And you wouldn't even notice because you don't usually monitor the logs on your phone, IMO.
It looks like a serious issue to me, and I think we should not allow such a risk. One suggestion would be to disallow password login. If someone did not ask pmbootstrap to copy his/her public key; well, he has no access to ssh. I'd say he won't ever loose access to his device, because he can still access the phone physically, but it may be annoying (some ui like mate don't provide a virtual keyboard and you need a terminal to install one, and some don't provide a terminal either, like sway).
The best would be to ask a confirmation before checking the password, as it would remove any brute force attempt.
That may require a fix to sshd or creating (finding?) a pam module. This could be used for inspiration.
The second way would need a patch to sshd's config to use PAM plus a patch to his PAM config (/etc/pam.d/sshd).