That would set the SONAME to libidn2.so.4, you might want to play with the numbers a bit to bump it just to libidn2.so.1, but it's just a number, it just needs to be different.
We should rather bring back these symbols as compat symbols if applications refer to them.
However, the question is why _idn2_punycode_decode is referenced by so many binaries. Last time we looked at it, it was just an accidental export, without any actual users outside libidn2. To me, it looks like there is another issue involved.
I just came to the same conclusion and restored those symbols in the 2.2.0-2 Debian package. Well, it could be the linker doing something wrong, but it's the Ubuntu 18.04, so fairly recent.
Searching for this function, it seems that gnutls is using it if compiled with libidn2 earlier than 2.0. As this is not an exported function, maybe gnutls is to blame here, and we should drop support for libidn2 < 2.0 there.
If you drop support for libidn2 < 2.0, you need to bump the SOVERSION. It's not friendly to the downstream users to drop the symbols that were in the public ABI and not bump the SOVERSION for own convenience. Regardless of the blame, either the symbols should be restored or the SONAME needs to change.
I understand this is inconvenient, but this will also hit anybody who does partial upgrades from Ubuntu 18.04 (LTS) to the next Ubuntu LTS. Debian seems to be on the safe side, as Debian doesn't support skipping releases when upgrading and stretch had 0.16, buster has 2.0.5 and bullseye will have 2.2.0.
We can bring back the symbols as compat symbols. If I recall correctly, a first attempt failed due to a musl limitation and the difficulty of recognizing musl at build time. I can try to come up with a different patch which works everywhere in late September. (I think we need to rename the internal functions and have _idn2_punycode_decode and _idn2_punycode_encode call them, without using a symbol alias.)
@oerdnj I am not so into downstream package organization, so please forgive me my somewhat naive question: What exactly is the problem ? If you want to upgrade a libidn2 0.16 environment, you have to to upgrade libgnutls + libidn2 in one step. You cannot upgrade either one of them. Isn't that kind of dependency tracking quite normal, or what do I miss ?
The problem is that the lockstep upgrade as you described it can happen only when there's a SOVERSION bump... :-) or somebody would have to manually declare Conflict between the old GnuTLS and new libidn2 package (or vice versa). It's not that easy to automatically deal with disappeared public symbols.
And when you declare the conflict, it also means that you can't ever partially upgrade your system to get new libidn2 f.e. for backporting libidn2 users (f.e. BIND 9, or php-pecl-http) unless you backport everything that get's broken too, which can turn into a mess (e.g. why would you get new GnuTLS from the BIND 9 repository when BIND 9 doesn't even use GnuTLS...).
All that would be fine if the SONAME/SOVERSION was changed, because would be able to co-install libidn2-0 and f.e. libidn2-1, or you just keep the backwards compatibility. You are neither the first project nor the last one that would fall into the trap "but those were private", once it's part of the ABI, people will use any symbols they found there.
I am normally totally pro automatism. In this case it might be a good idea to manually edit the package dependencies and add conflicts. The reason is that nobody should use libidn2 0.16 any more - there have been some serious (security) bugs being fixed meanwhile (not tracked as CVE). So forcing a user to upgrade libidn2 is a pro-active security step (when upgrading libgnutls). Also vice-versa... libgnutls fixes bugs every now and then.
JFTR I think that SONAME bump was a right call. It causes some disturbances for the distributions as everything needs to be rebuilt, but we (in Debian) deal with that all the time, so it's not a big hassle.
I would like to keep it easy... is a 2.3.0 release with a SONAME bump appropriate ? I would also move from Unicode 6.3 to 11.0. I would not re-introduce the dropped symbols as we have secure replacement functions since 2.0.0.
Is there anything that speaks against this approach ?
I think that's OK, I have reintroduced the symbols for Debian 2.2.0 package, so that's on the safe side now, and when the SONAME gets bumped in 2.3.0, I will drop the patch.
I would like to keep it easy... is a 2.3.0 release with a SONAME bump appropriate ? I would also move from Unicode 6.3 to 11.0. I would not re-introduce the dropped symbols as we have secure replacement functions since 2.0.0.
Is there anything that speaks against this approach ?
My point is that the damage is already done, and will be fixed one way or another. I do not see the point for a soname bump at this point. A soname bump has several problems as it creates two streams of the library, one before the bump and one after. Those will be maintained separately by distributions, causing a lot of trouble (e.g., enterprise distros which ship the current libidn2 will most likely never upgrade).
I think keeping the soname fixed, and introducing a policy to not break it is the right way forward. We already have an ABI check introduced towards that goal, so future issues are eliminated.
Ddin't have time today, but will look into how to thin-layer around to_unicode API to provide the two removed symbols/functions. So we provide them in a safe way.
I think we would have to fork libidn2 for glibc's sake if it ever bumps soname again. This seems a little bit pointless.
All we need to do is to rename the internal uses of the removed functions, and then we can safely introduce a compatibility symbol. That is pretty easy. (The previous attempt without renaming ran into limitations of the musl dynamic loader.)
I currently have Internet trouble, so I do not know when I will be able to work on this.
!82 (closed) is a rehash of the old proposal for a compatibility symbol, which we rejected as too complex at the time. I had almost forgotten about it, but found it again just in time …