usb_install_and_upgrade.mdwn 23 KB
Newer Older
[[!meta title="USB install and upgrade"]]

[[!toc levels=3]]

5 6 7 8 9
Current state of things

Tails [[ships hybrid ISO images|contribute/design/hybrid_ISO]]. dd'ing
such a file to a USB stick produces a bootable Tails system and is
[[well documented|doc/installing_onto_a_usb_stick/linux]], but it
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
requires unfriendly command-line usage and overwrites any previously
existing partition table.

The current state of things implies two distasteful facts:

1. A given USB stick currently must be dedicated to Tails only;
   considering the ever-growing size of such devices, the amount of
   wasted bytes is not a minor one.
2. A given USB stick cannot be used to host both a Tails system *and*
   a persistent storage volume, which makes it hard to provide
   friendly integration of [[upcoming persistence
   features|todo/persistence]] for end-user.

Such user unfriendliness conflicts with Tails [[specified
goals|contribute/design]] and must be fixed.

Tails developers's avatar
Tails developers committed
27 28
Installing specification

One should be able to easily install Tails to a USB storage device:

* by cloning an existing Tails system that is running from CD or USB;
33 34 35 36 37 38
* running a non-Tails operating system, from a downloaded ISO image.

The installation target may be an existing partition whose content
must be left untouched. The choice between possible destination
devices or partitions must be graphically proposed amongst the
available removable storage devices.

40 41
Moreover, at installation time, one should be allowed to setup an
encrypted storage space that can be used by the [[upcoming persistence
features|todo/persistence]]; earliest versions of the installation
43 44
program must at least leave room for it so that a later version can be
make use of it. Whether this storage space should better be a second
45 46 47
encrypted partition or a file-backed encrypted FS shall be researched
and decided upon. If the partition way is chosen, the installer
software must have partitioning capabilities.

49 50 51 52 53 54 55

[[!tag todo/code]]

* [[what is left to do|usb_install_and_upgrade/todo]]
* [[what we've done so far|usb_install_and_upgrade/done]]
* [[notes about GPT|usb_install_and_upgrade/gpt]]
Tails developers's avatar
Tails developers committed
57 58 59 60 61 62 63 64

Our plan is to:

1. release something useful based on the work we've already done on
   the basis of Fedora liveusb-creator
2. probably start (again...) from Ubuntu's usb-creator, that has a
   much saner codebase; [[usb_install_and_upgrade/future_spec]]
65 66 67 68 69 70

Our work on liveusb-creator is in a Git repository:

- anonymous, read-only: `git://`
- developers read-write: `ssh://`

71 72

Frontend UI
75 76

77 78 79
### Overview

Unfortunately, no single existing piece of software satisfies the
above specification. It seems like everyone writes their own USB
81 82 83
installer; after the following round-up has been made, this is no
surprise at all: since such tools are designed to be dead easy for the
non-technical end-user, every author selects the few features that are
the most important ones for their intended audience, and generally
85 86
writes a very ad-hoc piece of software. Therefore, every Live system
project whose needs are slightly different from the others needs to
write its own USB installer, possibly reinventing more or less wheels
88 89 90 91 92 93 94 95 96 97 98 99 100 101
along the road.

We need to see if we can turn one of those tools into something that
satisfies our specification; sadly, proceeding this way *without
forking the existing tool* would probably need more initial
development time than writing a new installer from scratch; we are not
even sure any upstream author of such a software (good enough to be a
possible viable basis) would want to make it more generic and maintain
it this way; on the other hand, if we avoid forking or writing from
scratch, we also avoid maintaining a Tails-specific tool on the long

### Existing tools

102 103
[[!tag todo/test]]

104 105 106
Wikipedia has a [[!wikipedia List of tools to create Live USB systems]]
we should look at.

* `live-installer` udeb and its `live-installer-launcher` desktop
108 109 110 111 112 113 114 115
  companion allows to run the Debian Installer from the bootloader;
  paraphrasing the manual: the "Live" Debian Installer flavour will
  proceed in an identical fashion to the normal installation but at
  the actual package installation stage, instead of using
  `debootstrap` to fetch and install packages, the "live" filesystem
  image is copied to the target. After this stage, the Debian
  Installer continues as normal, installing and configuring items such
  as bootloaders and local users, etc. This must be tested.

Tails developers's avatar
Tails developers committed
117 118 119 120 121 122
  > Not usable without a lot of work: there is way too many
  > questions asked, the graphical interface overlaps the
  > panels (hence unreachable buttons), it tries to create
  > a new user account, etc. Forget it, even if it was nice
  > to have partman handy to configure partitions and
  > bootloaders.

124 125 126
* `live-boot`'s `todisk=DEVICE` boot parameter could be used to copy
  the entire read-only media to the specified device; the process
  could then be:

128 129 130 131 132 133 134 135 136
  1. boot from CD
  2. use some documentation or GUI to partition the USB stick and take
     note of its device name
  3. reboot; at syslinux prompt add the `todisk=DEVICE` parameter
     using the previously noted device name; then let the system boot
     once from the USB stick, and use some documentation or GUI to
     install a bootloader on the USB stick. This step might be
     automated in live-boot.
  4. reboot and eject the CD: the system should now boot from USB.
137 138 139 140

  This only works for the "clone existing system" usecase, though.

* Ubuntu's [usb-creator](
141 142
   - supposed to replicate exactly the CD to USB
   - how does it handle partitions?

144 145
* [UNetbootin](
   - works on Windows and GNU/Linux
146 147
   - has a plugin architecture that only allows rebranding and
     pre-selection of the available distributions list
148 149
   - included in squeeze and sid, easily backported to Lenny
   - works like a charm to install an ISO image to USB, but the
150 151 152 153 154
     syslinux menu has the unetbootin logo instead of Debian's one;
     the installed syslinux defaults to the first menu entry (in
     Tails case, Arabic); are the other bits of syslinux menu (Debian
     Live help and so on) here?
   - part of Debian
   - non-destructive install: does not format the chosen partition
   - no support for partitioning nor persistent storage's avatar committed
   - works from a running Live CD; specify /dev/cdrom as ISO diskimage
158 159
   - how would upgrading work?

160 161 162 163 164 165 166 167 168
* [Ubuntu LiveUSB](
   - (a bit too) simplistic
   - dedicated to "create a bootable Live USB medium from a running
     Ubuntu Live CD"
   - depending on sources, it partitions USB stick with one single
     800MB partition or one partition using the whole device; in any
     case, it unconditionally overwrites the device's content
   - able to upgrade without re-partitioning the device?
   - [launchpad](

170 171 172 173
* aptosid's `aptosid-on-a-stick` needs an input ISO (check again), and
  lacks partitioning capabilities, but it still deserves being looked
  at more thoroughly; see the

175 176
* Fedora's `livecd-iso-to-disk`:
   - [manual](
177 178 179 180 181
   - [Gitweb](
   - cleanly and defensively written in Bash
   - supports creating a file-backed filesystem for `$HOME`,
     LUKS-encrypted by default, in the same partition where the Live
     system is installed
   - uses parted

184 185 186
* Fedora's [liveusb-creator](
   - works on Windows and GNU/Linux
   - "non-destructive install", which means?
   - supports FAT and EXT filesystems
   - persistent storage creation: file-backed filesystem, no
189 190 191 192
     partitioning capabilities
   - actively developed (as of early 2011)
   - written in Python
   - input: ISO file or real CD
193 194 195 196
   - output: extracts ISO's content to the partition
   - does not use `livecd-iso-to-disk`
   - needs to be slightly adapted/extended to support non-Fedora Live
     systems, e.g. some directory names must be made configurable
197 198 199
   - sidux ships a modified version:
      - [.deb](,
      - [source](

amnesia's avatar
amnesia committed
201 202
* [RIP’s mkusb]( and its

204 205 206 207
* SUSE Studio Image Writer([[!wikipedia SUSE Studio ImageWriter]],
  [KIWI homepage](, [source
   - supports Windows and GNU/Linux
208 209
   - persistent storage creation - that is?

#### Dismissed

* [FUSBi](
   - modified from UNetbootin; what are its enhancements?
214 215
   - supports Windows and GNU/Linux
   - no release in 2009 nor 2010

217 218 219 220 221 222 223 224 225 226 227
* Incognito's `root_overlay/usr/sbin/create-usb` seems quite nice, as
  it allows using an existing fat32 partition.
   - no partition capabilities

* Liberté Linux' `burn-usb` creates a FAT32 partition for data and
  an ext2 partition where it copies the system to. It installs Grub on
  the USB stick.

* [Mandriva Seed](;a=summary)
   - supports Windows and GNU/Linux
   - fork of Fedora's liveusb-creator; what are its enhancements?
228 229 230 231 232 233 234 235 236 237 238 239

* Moblin's [Image
   - written in Python
   - simple dd wrapper with proper sanity checks

* [usb-imagewriter](
   - seems like a clone of the good old rawwrite with no more feature
     at all

* <>: Windows installer for Ubuntu.

240 241 242 243 244 245 246 247 248 249 250
* [makebootfat](
  command line utility able to create bootable USB disks
  using the FAT filesystem and syslinux
   - available in Debian
   - able to autodetect/format/populate the USB disk in a
     single step without any user interaction
   - able to create disk images which are compatible with all the
     three standards USB-FDD, USB-HDD and USB-ZIP at the same time
   - if it supported partitioning as well, and supported Windows, it
     could be used to do most of the low-level work

251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308
Booting methods

### Overview

Tails [[ships hybrid ISO images|contribute/design/hybrid_ISO]]. dd'ing
such a file to a USB stick produces a bootable Tails system but it
overwrites any previously existing partition table. This method is
therefore not suitable for us since it does not satisfy the
"non-destructive upgrade process" requirement.

Two alternatives booting methods have been investigated:

1. **Boot ISO file**: Setup a bootloader on the USB stick that is able
   to boot an ISO image dropped into the right directory of it. Even
   early boot files (kernel, initramfs) are loaded from *inside* the
   ISO file. The bootloader installed on the USB stick can either
   directly boot the kernel from inside the ISO file or chainload the
   ISO image's bootloader. In the first case, until the
   [[todo/boot_menu]] task is completed, the installed boot loader
   must present a boot menu that at least allows the user to choose
   her preferred language.

2. **Copy ISO's content**: Copy needed files (mainly the kernel,
   initramfs and SquashFS image that live in the `live/` directory)
   from the ISO filesystem to the destination partition, then setup a
   bootloader that uses files in that directory. The end result is
   more or less equivalent to dd'ing the ISO image to the USB stick,
   but achieved with a non-destructive process.

### Grub2

#### Grub2's loopback support

Debian Live now supports Grub2 and `fromiso=`. The grub.cfg that is
generated inside the binary image could be copied, adapted and used to
(loopback) boot an ISO dropped on a USB stick.

Here is how to test this.

0. Build a Debian Live iso or iso-hybrid image with a working
   `live-boot` installed in it, i.e. at least 2.0.12 (warning: Tails
   0.6 series ship a buggy version).
1. Create a ext2 filesystem on a USB stick partition.
2. Copy a Tails ISO file **at the root** of the USB stick partition.
3. Run `ln -s . boot` at the root of the (mounted) USB stick partition.
4. Install Grub2 on the USB stick MBR, using the USB stick partition
   as `--root-directory`.
5. Create a `grub.cfg` file in there and try a menu entry like this:

          menuentry 'test' --class debian --class gnu-linux --class gnu --class os {
                insmod part_msdos
                insmod ext2
                insmod loopback
                insmod iso9660
                set root='(hd0,msdos1)'
                search --no-floppy --fs-uuid --set b51975cd-c5ef-4b32-996a-2c8206516917
                loopback loop /binary.iso
309 310
                linux   (loop)/live/vmlinuz boot=live fromiso=/dev/sda1/binary.iso config
                initrd  (loop)/live/initrd.img
311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334


* `insmod ext2` is because the partition that hosts Grub2 and the ISO
  is a ext2 one
* `(hd0,msdos1)` and the search UUID match this partition as well,
  from Grub2 point-of-view
* `/dev/sda1` is a path to this partition block device from inside the
  Live initrd
* `/binary.iso` is the path to the ISO file, relative to the root of
  the partition
* `/live/vmlinuz1` and `/live/initrd1.img` are respectively the path
  to the kernel and ramdisk relative to the root of the ISO filesystem
* `boot=live` enables `live-boot` and thus `fromiso=`
* `config` enables `live-config`

This might fails at kernel load time with following error:

    error: invalid magic number

In that case you probably need a newer Grub2 (see [[!debbug 543924]]
for details).

Tails developers's avatar
Tails developers committed
335 336

337 338 339
1. On some versions of Tails, this breaks the [[emergency
   shutdown|contribute/design/memory_erasure]] on media removal.
2. This is by no means a supported installation method. E.g. the
Tails developers's avatar
Tails developers committed
340 341
   kernel parameters shall be manually updated when replacing the ISO
   with a newer one.
Tails developers's avatar
Tails developers committed

343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389
#### Security matters

In the above example, the Grub2 configuration file must contain
information about the partition where the ISO file is stored.

Doing this is feasible at install time, and ensures the ISO will be
loaded from the right partition, rather than from a rogue one on the
host system's hard-disk.

Other Live systems that don't face the same threat model as Tails tend
to prefer using automagic ISO loopback support so that Grub2 itself
finds a partition where a properly named ISO is stored, e.g. using
[GRML's patches against
that add the `findiso=` boot parameter (by the way, does it respect
`live-media=removable` ?); here is how they boot things up:

- GRML (`grml-rescueboot` flavor) [now supports booting using Grub2
- comments of
- Super Grub Disk's
  [loopback.cfg]( might
  be [going

#### Grub2 combined with syslinux' memdisk

This is used by some in combination with Grub2, in place of its
loopback support. The [memdisk
page]( on the
syslinux wiki tells the whole low-level story and more.


- GRML (`grub-imageboot` flavor) [now supports booting using
  according to this blog post, support for doing this is in live-boot
  2.0.14-1. The memdiskfind/phram/mtdblock approach this blog post
  talks of would ease cloning an existing Tails system, since it is
  supposed to provide a `/dev/mtdblock0` device, which should be the
  .ISO image. See the [details on the memdisk wiki
- a [blog
  with a snippet for `grub.d` to automate booting ISO images from
  Grub2, using syslinux' memdisk

391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413
### extlinux and memdisk

According to Philip Hands, who proposed this solution on [[!debbug
534887]], it is possible to use extlinux merely to chainload the
image's own bootloader.


Various scripts, such as [Benjamin FOURTICQ's
aim at adding a persistent home-rw partition to a Debian Live disk
image, generally using losetup and parted. They don't suit Tails
specification as they depend on knowing in advance the size of the
persistent volume the user wants to setup and obviously cannot setup
encryption on it in place of the user.

On the other hand,
[udisks]( provides a
D-Bus interface that makes it easy to programmatically partition a
storage device and create filesystems in there without messing with
low-level details; its LUKS encryption support probably is the killer
feature that makes it the perfect library for the job.
414 415

Tails developers's avatar
Tails developers committed

Tails developers's avatar
Tails developers committed
418 419 420 421 422 423
One should also be able to easily and non-destructively upgrade a
given Tails system installed on a USB storage device.
Non-destructively means not touching any partition or file other than
the ones managed by the Tails installation system. Proceeding this way
is needed to make it possible using a single USB stick for the Tails
system *and* some persistent storage space.
424 425 426
The upgrade issue is actually twofold: network burden and writing the upgraded
live system on the USB stick.

Tails developers's avatar
Tails developers committed
427 428
Full upgrade

Tails developers's avatar
Tails developers committed
Full upgrades happen between major releases, eg. 0.7 and 0.8.

Tails developers's avatar
Tails developers committed
432 433
Full upgrades should look as much as possible as a new installs. So it should be
possible to do it from any platform supported by the USB installer.

Tails developers's avatar
Tails developers committed
435 436
As for installing, people should be able to provide the ISO to the installer or
ask the installer to download it.

Tails developers's avatar
Tails developers committed
438 439
From inside Tails it should also provide an opt-in option to download the full
ISO outside Tor.

Tails developers's avatar
Tails developers committed
441 442
Once the upgrade is finished the old ISO is removed automatically from the USB

Tails developers's avatar
Tails developers committed
444 445
Partial upgrade

Tails developers's avatar
Tails developers committed
Partial upgrades only happen between minor releases, eg. 0.7 and 0.7.1.

Tails developers's avatar
Tails developers committed
449 450
Partial upgrades should provide only what has changed between two releases
(deltas) and have a way to apply those changes to the previous version.

Tails developers's avatar
Tails developers committed
452 453
At boot-time the security warning telling that a new Tails version is available
should provide an automated way of doing the upgrade.

Tails developers's avatar
Tails developers committed
### Stacking `squashfs` images
456 457 458 459 460

Tails filesystem is already using `aufs` to provide a read-write filesystem on
top of the read-only `squashfs` image.

This system could probably be extended to support mounting multiple `squashfs`
Tails developers's avatar
Tails developers committed
461 462 463
filesystems on top of each others. Upgrades would be `squashfs` images with only
the files that have been modified since the previous releases. This handles file
464 465 466

Shipping upgrades could be as simple as shipping those extra `squashfs` images.

Tails developers's avatar
Tails developers committed
Actually there seems to be such an option in Debian live already, see in `man
live-boot` the part about `/live/filesystem.module`. There is need to do [[!taglink
Tails developers's avatar
Tails developers committed
todo/research]] on this.

Tails developers's avatar
Tails developers committed
471 472 473
Stacking squashfs images like this would still lack a way of upgrading the
kernel and the syslinux. This should also be handled by the automated upgrade

Tails developers's avatar
Tails developers committed
### A test
476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519

Here is a test. First the procedure to create the *delta* squashfs image, to be
done as `root`:

    mkdir /mnt/tails-0.7.1
    mkdir /mnt/tails-0.7.2
    mount -o loop tails-i386-0.7.1.iso /mnt/tails-0.7.1
    mount -o loop tails-i386-0.7.2.iso /mnt/tails-0.7.2
    mkdir /mnt/tails-0.7.1-root
    mkdir /mnt/tails-0.7.2-root
    mount -o loop /mnt/tails-0.7.1/live/filesystem.squashfs /mnt/tails-0.7.1-root
    mount -o loop /mnt/tails-0.7.2/live/filesystem.squashfs /mnt/tails-0.7.2-root

    mkdir /mnt/upgrade-0.7.1-to-0.7.2
    mount -t tmpfs tmpfs /mnt/upgrade-0.7.1-to-0.7.2

    mkdir /mnt/union
    mount -t aufs -o br=/mnt/upgrade-0.7.1-to-0.7.2=rw:/mnt/tails-0.7.1-root=ro none /mnt/union
    rsync -avP --delete-after /mnt/tails-0.7.2-root/ /mnt/union/

    mksquashfs /mnt/upgrade-0.7.1-to-0.7.2 upgrade-0.7.1-to-0.7.2.squashfs

Compressed size (using default gzip compression) is 82 MB.

Not bad, and the new kernel is included, which can probably be avoided.

Now, let's upgrade an USB stick:

    mkdir /media/disk/live
    cp   /mnt/tails-0.7.1/live/filesystem.squashfs \
         upgrade-0.7.1-to-0.7.2.squashfs \
         /mnt/tails/0.7.2/live/vmlinuz \
         /mnt/tails/0.7.2/live/initrd.img \

Then fiddle with GRUB or EXTLINUX.

On boot, the new squashfs gets properly integrated. But *whiteouts* are not
working. It looks like the `live-boot` mount options miss the `wh` attribute.
But wait, booting with `break=top` and modifying `/scripts/live` to replace
`roopt=rr` by `roopt=rr+wh` is enough to do the trick!

Initial test is pretty conclusive!

Tails developers's avatar
Tails developers committed
520 521 522 523 524
### Appending to `squashfs` image (discarded option)

`mksquashfs` can actually append new files to an existing `squashfs` image.

Initial images are created with files in a specific order to
[[improve boot time on cd]], but on a USB stick random access is a non-issue.
Tails developers's avatar
Tails developers committed
526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572

[[!taglink todo/test]] if `mksquashfs` can append an image that is currently
used without weakening a running system.

Upgrading the system would result in a series of files to be appended to the
current `squashfs` image.

This option had been discared because it is not possible to remove files.

### Deltas (discarded option)

A possible way to encode deltas for the two previous methods could be:

 * For each file that has been modified: a binary delta and a new set of
   metadata if they have changed.
 * A list of deleted files.

And `mksquashfs` would be used in the running live system after applying the
delta to create a `squashfs` image with the upgrade.

But there is probably things that have been left out of such an early draft.

### Binary diff (discarded option)

Plain `diff` does not work on binary files.

Binary diffs ([rsync], [xdelta], [bsdiff], [VCDIFF]) gives poor
result on live system images because `squashfs` images vary
strongly as a whole, even for tiny changes of the files inside.
That situation is unlikely to change ([[!debbug 602965]]) and is even worse
with `squashfs-lzma`. Quoting `xz` manpage:

     Compressed output may vary
       The exact compressed output produced from the same uncompressed input
       file may vary between XZ Utils  versions  even if  compression  options
       are identical.  This is because the encoder can be improved (faster or
       better compression) without affecting the file format.  The output can
       vary even between different builds of the same XZ Utils  version, if
       different build options are used.

       The  above means that implementing --rsyncable to create rsyncable .xz
       files is not going to happen without freezing a part of the encoder
       implementation, which can then be used with --rsyncable.