Experience report from trying out Obnam2 version 0.2.2
Hi!
I installed Obnam using the Debian package (I'm running Debian testing on amd64). My live data is my home directory, containing 74 gigabytes of data (121311 directories, 768438 files). The following notes are ordered chronologically because there's no other theme to them. Also note that I didn't look at the code prior to trying the program out; some of the problems below sound trivial when I do git grep
, but from a "user perspective", they weren't.
First, a note on the Debian package. Should it contain benchmarking binaries? They look like internal artifacts that slipped into the final archive.
Now on to actual backups. I back up onto an encrypted USB-HDD which I connect just for the occasion, so my laptop is both a client and a server in Obnam2's view. In this setup, I couldn't help but perceive the server as a nuisance: the configuration is spread out between /etc and my home directory, I have to think about self-signed SSL certificates, and arrange for the server to be stared and stopped as I reconnect the HDD. Most of this friction is a one-time cost, but under different conditions it would be enough to make me look at other software. Ideally, I should be able to set server_url: file:///media/usb-hdd/home-backups
and run obnam backup
, which would start the server in the background as if I created a server config with chunks: /media/usb-hdd/home-backups
, arrange for a temporary SSL certificate, do the backup, then tear down the server.
It was a bit of a challenge to figure out how to run the client. I ended up copying a config from the Subplot file, and tweaking it on a hunch. It'd be nice to have a barebones guide of "run Ansible, create ~/.obnam.yaml, run obnam --config ~/.obnam.yaml backup
. I can submit an MR to that effect, if you want it.
On the subject of the config: obnam
helpfully prints out a list of commands, but then running obnam config
or obnam backup
shows a very unhelpful error: "Error: No such file or directory (os error 2)". It's only now that I glance at the code and see that it expects a client config at ~/.config/obnam/obnam.yaml; it'd be nice if the error mentioned that. As mentioned earlier, I put mine into ~/.obnam.yaml, and passed --config
to the commands.
When I ran obnam --config .obnam.yaml backup
, the program printed out a big white bar across the whole terminal window, and put additional info under that bar. I thought it might be a progress bar, but it didn't change throughout the whole backup process. What is it?
obnam backup
prints out some running total of files, but it's two numbers, and the second one is always zero. What does it mean? I think the line should say something like "Files processed/inaccessible: 100500/3" (or whatever that second number means).
I was surprised by how important CACHEDIR.TAG and "exclusion rules" are to fast backups. With Obnam2, the initial backup of 74G took 1 hour and 17 minutes, and two incremental ones took 17 minutes each. Borg, which supports the tag and is configured with a couple exclusion rules, cuts that down to 23.5G and does an incremental backup in less than a minute.
Finally, the backups seem to be made, but I can't list generations or restore anything:
$ obnam --config .obnam.yaml show-generation
ERROR: Unknown generation: latest
Error: Unknown generation: latest
The "chunks location" look all right to me:
$ ls -la
total 58456
drwxr-x--- 18 minoru minoru 4096 Feb 8 20:30 ./
drwxrwxr-x 7 root minoru 4096 Feb 8 20:13 ../
drwxr-xr-x 18 root root 4096 Feb 8 20:30 100/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 101/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 102/
drwxr-xr-x 18 root root 4096 Feb 8 20:31 48/
drwxr-xr-x 18 root root 4096 Feb 8 20:31 49/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 50/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 51/
drwxr-xr-x 18 root root 4096 Feb 8 20:31 52/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 53/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 54/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 55/
drwxr-xr-x 18 root root 4096 Feb 8 20:31 56/
drwxr-xr-x 18 root root 4096 Feb 8 20:31 57/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 97/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 98/
drwxr-xr-x 18 root root 4096 Feb 8 20:30 99/
-rw-r--r-- 1 root root 55582720 Feb 8 22:23 meta.db
-rw-r--r-- 1 root root 32768 Feb 8 22:36 meta.db-shm
-rw-r--r-- 1 root root 4169472 Feb 8 22:36 meta.db-wal
I ran obnam backup
three times, but it seems like all chunks belong to a single generation:
$ sqlite3 meta.db 'select distinct generation from chunks'
0
The logs of Obnam server display None
where I think should be values:
Feb 08 22:38:06 firefly obnam-server[107362]: lookup: meta=ChunkMeta { sha256: "55e1052b0a59291991a7cf92da561ca66820f2005413b85b3f3670e075479912", generation: None, ended: None }
Feb 08 22:38:06 firefly obnam-server[107362]: lookup: meta=ChunkMeta { sha256: "15f70b38bb95f3002fec9aea54030f19bb4ecfbad64c67424b5e5fea09cd749e", generation: None, ended: None }
Feb 08 22:38:06 firefly obnam-server[107362]: lookup: meta=ChunkMeta { sha256: "cb2bd46b1918d6010015cfe5f3932f22e6b90972a5d328ddc6b96111fde398b4", generation: None, ended: None }
Feb 08 22:38:06 firefly obnam-server[107362]: lookup: meta=ChunkMeta { sha256: "49a1ac4cfdb1d675bea600213469e70ed550c5f35ef52232d07a0727b8a93fc5", generation: None, ended: None }
Feb 08 22:38:06 firefly obnam-server[107362]: lookup: meta=ChunkMeta { sha256: "d4a296b703289b120042b205b7a30bf13a445c40f00d57f0c276e2391b051362", generation: None, ended: None }
Feb 08 22:38:06 firefly obnam-server[107362]: lookup: meta=ChunkMeta { sha256: "8cca937b708b31704ecf9406b6e1041d6741dee535aaf603d0bf1ad1c996c0fb", generation: None, ended: None }
My configs:
# /etc/obnam/server.yaml
address: 127.0.0.1:8471
chunks: /media/mypassport-backup/obnam2
tls_key: /etc/obnam/server.key
tls_cert: /etc/obnam/server.pem
# ~/.obnam.yaml which I specify via --config
server_url: https://localhost:8471
verify_tls_cert: false
roots:
- /home/minoru
# I commented this line out after the initial backup.
# log: /tmp/obnam.log
Any ideas on what's going on? What other info can I provide to make this debuggable?