Skip to content

Cache build is not deterministic anymore

Hi,

In Tails we ship a compiled binary cache for our AppArmor policy (to improve boot time) and the ISO/USB images we distribute have to build reproducibly. This all worked smoothly so far until Tails 6.x, based on Debian 12, with AppArmor 3.0.8-3.

We're now working on porting Tails to Debian 13, which has AppArmor 4.1.0-1. And we face a regression in this area, which is going to block our initial release candidate, that's currently scheduled for August 7.

When we compile our policy like this:

apparmor_parser \
   --write-cache \
   --skip-kernel-load \
   --verbose \
   --add \
   /etc/apparmor.d

… at least 2 of among the cupsd, Evince, Pidgin, and Totem profiles. The Evince one always differs (and IIRC it's one of the most complex and slowest to compile).

Here's a smaller reproducer that always yields a different binary cache on my Debian sid system:

for n in 1 2; do
   apparmor_parser --write-cache --skip-kernel-load --add --cache-loc "/tmp/$n" /etc/apparmor.d/usr.bin.evince
done
diff -Naur /tmp/{1,2} | diffstat

I've tried to compare the binary cache with diffoscope but the binary diff does not help me at all. Perhaps it would be more readable for someone else?

On the corresponding Tails issue I've listed a few things that I've tried already, and that did not help:

  • --skip-read-cache
  • --jobs=1
  • faketime -f "(date --utc --date="(dpkg-parsechangelog --show-field=Date)" '+%Y-%m-%d %H:%M:%S')"
  • --optimize=0
  • --optimize=compress-small
  • --optimize=equiv

Any other workaround I could try?

Any idea what change in the parser could have made the cache build non-deterministic?

Sorry my IRC access is broken at the moment so I can't conveniently join you there.

cc @jjohansen who's usually on top of things for this kind of problem <3