Feature: `--exclude-mountpoints`
Summary
When a filesystem is bindmounted under itself, duplicate files are backed up. --exclude-other-filesystems
suggests itself as a possible solution, but does not actually work since it just compares the filesystem identifier (this is as documented, so this is more a feature request than an actual bug).
Adding a new --exclude-mountpoints
option could help in this case, and also in the case described by #4.
Steps to reproduce
To reproduce, bindmount a filesystems inside itself and then back that up.
matthijs@grubby:/tmp$ mkdir backup
matthijs@grubby:/tmp$ mkdir backup/mount
matthijs@grubby:/tmp$ touch backup/file
matthijs@grubby:/tmp$ sudo mount -o bind backup backup/mount/
matthijs@grubby:/tmp$ tree source/
source/
├── file
└── mount
├── file
└── mount
Then backup:
matthijs@grubby:/tmp$ duplicity --version
duplicity 0.8.04
matthijs@grubby:/tmp$ snap run duplicity --version
duplicity 0.8.14.dev49
matthijs@grubby:/tmp$ snap run duplicity --exclude-other-filesystems source file:///tmp/backup
Warning, found signatures but no corresponding backup files
Synchronizing remote metadata to local cache...
Deleting local /home/matthijs/.cache/duplicity/ba8d32ccb88d13597b4784252744fc75/duplicity-full-signatures.20200615T083303Z.sigtar.gz (not authoritative at back
end).
Deleting local /home/matthijs/.cache/duplicity/ba8d32ccb88d13597b4784252744fc75/duplicity-full.20200615T083303Z.manifest (not authoritative at backend).
Last full backup date: none
GnuPG passphrase:
Retype passphrase to confirm:
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1592210483.18 (Mon Jun 15 10:41:23 2020)
EndTime 1592210483.19 (Mon Jun 15 10:41:23 2020)
ElapsedTime 0.01 (0.01 seconds)
SourceFiles 5
SourceFileSize 12288 (12.0 KB)
NewFiles 5
NewFileSize 12288 (12.0 KB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 5
RawDeltaSize 0 (0 bytes)
TotalDestinationSizeChange 261 (261 bytes)
Errors 0
-------------------------------------------------
What is the current bug behaviour?
The backup includes the bindmounted directory / files twice:
matthijs@grubby:/tmp$ snap run duplicity list file:///tmp/backup
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: Mon Jun 15 10:41:18 2020
Mon Jun 15 10:30:51 2020 .
Mon Jun 15 10:30:51 2020 file
Mon Jun 15 10:30:51 2020 mount
Mon Jun 15 10:30:51 2020 mount/file
Mon Jun 15 10:30:48 2020 mount/mount
What is the expected correct behaviour?
I would want to include everything only once, e.g.:
Mon Jun 15 10:30:51 2020 .
Mon Jun 15 10:30:51 2020 file
Mon Jun 15 10:30:51 2020 mount
(maybe not even include the mount
)
Possible fixes
Adding a --exclude-mountpoints
that would skip any directory that is a mountpoint would fix this. This could be a replacement for --exclude-other-filesystems
and even also fix #4 since this new exclude would apply only on the mountpoint and can be overridden with an explicit include to include more than one filesystem.
Detecting mountpoints is, however, not entirely trivial. A common approach is to compare the filesystem id of a directory with that of its parent, which works in most cases except for the case described above (bindmounting under itself), since then the filesystem id of both will be equal.
It seems the mountpoint
utility (src) from util-linux solves this by reading mountpoints from /proc/self/mountinfo
, falling back to the above approach when it is not available. I guess duplicity could adopt a similar approach, essentially putting the contents of mountinfo
into the exclude list might even be sufficient already.
I have:
-
searched https://gitlab.com/duplicity/duplicity/-/issues for similar issues. If you find a similar issue and the issue is still open, add a comment to the existing issue instead of opening a new one. If you find a Closed issue that seems like it is the same thing that you're experiencing, open a new issue and include a link to the original issue in the body of your new one. -
searched https://bugs.launchpad.net/duplicity for similar issues. If you find a similar issue, open a new issue on here and include a link to the original issue in the body of your new one. -
ideally, tested that this issue still occurs on the latest edge snap, if you can test without risking your data. Please include the snap version output: installed: x.xx.xx (xx)