[Suggestion needed] High memory usage when opening a LUKS2 partition

I was trying to open a 7.2TB encrypted ext4 partition on my Raspberry Pi (1G memory, 8G swap, using cryptsetup 2.0.2) but got:

Mar 27 00:26:25 Pi3BP kernel: Out of memory: Kill process 738 (cryptsetup) score 93 or sacrifice child
Mar 27 00:26:25 Pi3BP kernel: Killed process 738 (cryptsetup) total-vm:1056100kB, anon-rss:888844kB, file-rss:5704kB, shmem-rss:0kB
Mar 27 00:26:25 Pi3BP kernel: oom_reaper: reaped process 738 (cryptsetup), now anon-rss:888940kB, file-rss:5692kB, shmem-rss:0kB

The partition was created via

cryptsetup --cipher aes-xts-plain64 --key-size 256 --hash sha256 luksFormat --type luks2 /dev/sdb1

Having no idea why cryptsetup open was consuming such a large amount of memory, I spent hours googling the issue but got no clue at the end. But since most of the successful examples I found were quite old and apparently they were using LUKS1, I tried re-formatting the partition with --type luks1 and this time it just got mounted on Raspberry Pi without any error.

May I know:

1. Are there any workarounds to this issue?
2. What factors influence the memory usage?

This is pretty much the first time for me to try out disk encryption and I am still absolutely a n00b. It would be greatly appreciated if you could provide me some suggestions. Thanks!

It is not a bug, it is a feature :) Well, sort of:

The LUKS2 format uses by default Argon2i key derivation function that is so called memory-hard function - it requires certain amount of physical memory (to make dictionary attacks more costly).

I set default 1G required memory but it is trimmed by half of available physical memory - but I guess you format the device on different system with more memory available.

Solution is simple, either

1. switch to LUKS1 (as you did), or
2. use LUKS2, but switch to PBKDF2 (that is used in LUKS1),
   just add "--pbkdf pbkdf2" option to luksFormat (or to any command that creates keyslots), or
3. use LUKS2 but decrease number of memory for Argon2i function,
   for example to use up to 256kB, just add "--pbkdf-memory 256".

Please let me know if it helps.

I will add some info to FAQ, or o some blog but first wee need some real-world data. So thanks for reporting this, it will help others later :)

assigned to @mbroz

but I guess you format the device on different system with more memory available.

LOL that's exactly what I did! I should have patiently studied the output of luksDump before all the googling and filing this ticket — a Memory field straight there!

Anyway, I used --pbkdf-memory to limit it to 256MB and now it's working like a charm. Thanks a ton for your quick response!

closed

As an addition to above:

We recently ran into a similar situation, but with LUKS2 volumes opened automatically during system startup via /etc/crypttab. We are using key files instead of passphrases, to achieve fully automated boot.

Unfortunately we get an out-of-memory error when more than 1 or 2 LUKS2 volumes are to be opened during boot. In this case systemd (i.e. systemd-cryptse) is killed:

[ 10.440478] Out of memory: Kill process 806 (systemd-cryptse) score 42 or sacrifice child [ 10.440491] Killed process 806 (systemd-cryptse) total-vm:1157180kB, anon-rss:1081952kB, file-rss:3196kB, shmem-rss:0kB [[0;1;31mFAILED[0m] Failed to start Cryptography Setup for enc-disk2.

Systemd tries to open all the LUKS2 volumes configured in /etc/crypttab concurrently, resulting in the fact that the system runs out of memory because each of them uses a lot of memory for Argon2i. Even if there is enough memory for opening one LUKS2 volume, there is not enough for opening many of them concurrently.

Of course, above mentioned solutions, such as using PBKDF2 or limiting the memory for Argon2i also help here. I just wanted to mention this because when using /etc/crypttab with many LUKS2 volumes and keyfiles you will almost always run into this situation when you use the LUKS2 defaults.

When using interactive passphrase entry with /etc/crypttab (i.e. no keyfile specified), this should not happen, since it prompts for the passphrases for the LUKS2 volumes one after the other, so that only one volume is opened at a time.

Hi,

we'll add some API to identify the risk for users (systemd unlock service in this case) so they can act accordingly and serialize unlock actions. It'll provide max memory cost per specific existing keyslot or any existing keyslot in LUKS device.

I just ran into this problem. What baffles me is that I never asked for a luks2 format during luksFormat. The exact command was cryptsetup -v --cipher aes-xts-plain64 --key-size 512 --hash sha512 luksFormat. Now I can't open the device anymore due to low memory. Very odd.

Do I really have to format and rebuild the container? :(

Can't I add a new key slot with a key that doesn't require so much memory?

Can I add a new keyslot with a key that could be converted to luks1 format?

Can I backup the header to a more powerful machine and operate on it to do that?

No, you do not need to rebuild it.

You can downgrade/change keyslot options with luksConvertKey (wither with lowering memory or using --pbkdf pbkdf2 as in LUKS1).

Convert ro LUKS1 is possible only if you switch all slots back to pbkdf2 (with the command above).

Thanks Milan. :)
Here is what I tried:

> cryptsetup -v --pbkdf pbkdf2 luksConvertKey /dev/sdX
Enter passphrase for keyslot to be converted:
Key slot 1 created.
Command successful.

> cryptsetup -v luksDump /dev/sdX
Data segments:                                                                                                                                                                                      
  0: crypt                                                                                                                                                                                          
        offset: 16777216 [bytes]                                                                                                                                                                    
        length: (whole device)                                                                                                                                                                      
        cipher: aes-xts-plain64                                                                                                                                                                     
        sector: 512 [bytes]                                                                                                                                                                         
                                                                                                                                                                                                    
Keyslots:                                                                                                                                                                                           
  1: luks2                                                                                                                                                                                          
        Key:        512 bits                                                                                                                                                                        
        Priority:   normal                                                                                                                                                                          
        Cipher:     aes-xts-plain64                                                                                                                                                                 
        Cipher key: 512 bits                                                                                                                                                                        
        PBKDF:      pbkdf2
        Hash:       sha256
        Iterations: [...]
        Salt:       [...]
        AF stripes: 4000
        AF hash:    sha256
        Area offset:290816 [bytes]
        Area length:258048 [bytes]
        Digest ID:  0
Digests:
  0: pbkdf2
        Hash:       sha512
        Iterations: [...]
        Salt:       [...]
        Digest:     [...]

However, it seems I cannot convert the header to luks1:

> cryptsetup -v convert /dev/sdX --type luks1
[...]
Cannot convert to LUKS1 format - keyslot 1 is not LUKS1 compatible.
Command failed with code -1 (wrong or missing parameters).

I suspect the header size is too big for luks1 conversion? Or perhaps it's something else. Either way, it's probably not a big deal, as pbkdf2 is enough to solve my memory problem in this case.

Anyway, love your work guys. Thank you very much for your contributions! :)

LUKS1 cannot use different hash for digest and keyslot, you can perhaps use sha512 for both but it is not worth it - LUKS2 should work ok with PBKDF2 here.

mentioned in commit AnatomicJC/debian-bootstrap-install-scripts@1d4be643

Hi,

is there a way to use luksConvert on key slots used for clevis/tang?

When i run cryptsetup luksConvertKey --pbkdf-memory 262144 -S 1 /dev/sdxxx it asks for passphrase. I tried to supply passphrase from curl -s localhost:7406/adv | jose fmt -j- -g payload -y -o- | jose jwk use -i- -r -u verify -o- | jose jwk thp -i- , but it's not the right one, right?.

Or is there a way to specify --pbkdf-memory to command clevis luks bind? I can replace key slots, no problem.

We have 5 cryptodisks on machine (well 3 machines) with 2GB of RAM and unlocking is really pain.

Thanks

Yes, it should work provided you supply correct passphrase to the command.

Also, it depends what do you use to unlock your devices. We've recently provided libcryptsetup API with option to serialize unlock process for multiple LUKS2 devices when argon2 pbkdf is used. AFAICT systemd already adapted to it and there were no issues reported so far: https://github.com/systemd/systemd/commit/408c81f62454684dfbff1c95ce3210d06f256e58

For more info, see cryptsetup man page and look for --serialize-memory-hard-pbkdf option.

Perhaps RFE for clevis/tang?

We are on debian10 and use systemd 246.6-2 from backports, because there was some info about solving cryptsetup && OOM conditions in changelog. I have also upgraded to cryptsetup 2.3.4 from backports.

Well, systemd in that version activates with the serialization flag automatically. You should be good to go with systemd. If you activate luks devices via cryptsetup (our CLI) you just need to ask clevis/tang to add the --serialize-memory-hard-pbkdf option in.

Just for the record: It will help you only if you experience OOM due to multiple keyslots (devices) being unlocked in parallel. If your keyslot requires more memory than a system ram available, it will not help you.

Well, i think, it's activated via systemd (there are units generated by systemd-cryptsetup).

I will try to get rid of systemd target, which depends on all filesystems (it has Requires=multi-user.target mnt-btrfs-backup.mount mnt-btrfs-sklad.mount mnt-btrfs-sklad_local.mount mnt-btrfs-sql.mount mnt-btrfs-www.mount mariadb.service - we use it to make sure, all filesystems and services are up), but i guess it does not affect parallelism of units.

If i could shrink cryptsetup luksConvertKey --pbkdf-memory 262144 -S 1 /dev/sdxxx key in slot 1, it would be great, but i don't know the right passphrase. It's all still little blackboxy for me.

Ok, i have patched in --pbkdf-memory 262144 to /usr/bin/clevis-luks-bind. I will replace keys on all disks and test if it helps.

Hmm, just out of curiosity. Could you share systemd services used to unlock those devices? I really think systemd should avoid the OOM situation since v244 while unlocking in parallel. But maybe debian compiled systemd linked with older lbcryptsetup and the support for CRYPT_ACTIVATE_SERIALIZE_MEMORY_HARD_PBKDF flag was missing.

Replacing keys using patched clevis-luks-bind helped to get rid of OOM during boot.

And after changing mount units from (for example) What=LABEL=crypt-backup-fs to What=/dev/mapper/crypt_backup , everything seems to mount automatically.

Thanks for tips.

I guess debian backported systemd is compiled against cryptsetup from debian10, ie. version 2.1.0 .

Version in bullseye depends on libcryptsetup12 (>= 2:2.3), but version in buster-backports depends on libcryptsetup12 (>= 2:2.0.1).

How to check ? Using strings command on /lib/systemd/systemd-cryptsetup?