Skip to content

mm/memory-failure: fix deadlock when hugetlb_optimize_vmemmap is enabled

Aristeu Rozanski requested to merge arozansk/centos-stream-9:arozansk-RHEL-35091 into main-rt

JIRA: https://issues.redhat.com/browse/RHEL-35091
Tested: sanity, can't reproduce
CVE: CVE-2024-26987

commit 1983184c22dd84a4d95a71e5c6775c2638557dc7
Author: Miaohe Lin linmiaohe@huawei.com
Date: Sun Apr 7 16:54:56 2024 +0800

mm/memory-failure: fix deadlock when hugetlb_optimize_vmemmap is enabled  

When I did hard offline test with hugetlb pages, below deadlock occurs:  

======================================================  
WARNING: possible circular locking dependency detected  
6.8.0-11409-gf6cef5f8c37f #1 Not tainted  
------------------------------------------------------  
bash/46904 is trying to acquire lock:  
ffffffffabe68910 (cpu_hotplug_lock){++++}-{0:0}, at: static_key_slow_dec+0x16/0x60  

but task is already holding lock:  
ffffffffabf92ea8 (pcp_batch_high_lock){+.+.}-{3:3}, at: zone_pcp_disable+0x16/0x40  

which lock already depends on the new lock.  

the existing dependency chain (in reverse order) is:  

-> #1 (pcp_batch_high_lock){+.+.}-{3:3}:  
       __mutex_lock+0x6c/0x770  
       page_alloc_cpu_online+0x3c/0x70  
       cpuhp_invoke_callback+0x397/0x5f0  
       __cpuhp_invoke_callback_range+0x71/0xe0  
       _cpu_up+0xeb/0x210  
       cpu_up+0x91/0xe0  
       cpuhp_bringup_mask+0x49/0xb0  
       bringup_nonboot_cpus+0xb7/0xe0  
       smp_init+0x25/0xa0  
       kernel_init_freeable+0x15f/0x3e0  
       kernel_init+0x15/0x1b0  
       ret_from_fork+0x2f/0x50  
       ret_from_fork_asm+0x1a/0x30  

-> #0 (cpu_hotplug_lock){++++}-{0:0}:  
       __lock_acquire+0x1298/0x1cd0  
       lock_acquire+0xc0/0x2b0  
       cpus_read_lock+0x2a/0xc0  
       static_key_slow_dec+0x16/0x60  
       __hugetlb_vmemmap_restore_folio+0x1b9/0x200  
       dissolve_free_huge_page+0x211/0x260  
       __page_handle_poison+0x45/0xc0  
       memory_failure+0x65e/0xc70  
       hard_offline_page_store+0x55/0xa0  
       kernfs_fop_write_iter+0x12c/0x1d0  
       vfs_write+0x387/0x550  
       ksys_write+0x64/0xe0  
       do_syscall_64+0xca/0x1e0  
       entry_SYSCALL_64_after_hwframe+0x6d/0x75  

other info that might help us debug this:  

 Possible unsafe locking scenario:  

       CPU0                    CPU1  
       ----                    ----  
  lock(pcp_batch_high_lock);  
                               lock(cpu_hotplug_lock);  
                               lock(pcp_batch_high_lock);  
  rlock(cpu_hotplug_lock);  

 *** DEADLOCK ***  

5 locks held by bash/46904:  

stack backtrace:  
CPU: 10 PID: 46904 Comm: bash Kdump: loaded Not tainted 6.8.0-11409-gf6cef5f8c37f #1  
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014  
Call Trace:  
 <TASK>  
 dump_stack_lvl+0x68/0xa0  
 check_noncircular+0x129/0x140  
 __lock_acquire+0x1298/0x1cd0  
 lock_acquire+0xc0/0x2b0  
 cpus_read_lock+0x2a/0xc0  
 static_key_slow_dec+0x16/0x60  
 __hugetlb_vmemmap_restore_folio+0x1b9/0x200  
 dissolve_free_huge_page+0x211/0x260  
 __page_handle_poison+0x45/0xc0  
 memory_failure+0x65e/0xc70  
 hard_offline_page_store+0x55/0xa0  
 kernfs_fop_write_iter+0x12c/0x1d0  
 vfs_write+0x387/0x550  
 ksys_write+0x64/0xe0  
 do_syscall_64+0xca/0x1e0  
 entry_SYSCALL_64_after_hwframe+0x6d/0x75  
RIP: 0033:0x7fc862314887  
Code: 10 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24  
RSP: 002b:00007fff19311268 EFLAGS: 00000246 ORIG_RAX: 0000000000000001  
RAX: ffffffffffffffda RBX: 000000000000000c RCX: 00007fc862314887  
RDX: 000000000000000c RSI: 000056405645fe10 RDI: 0000000000000001  
RBP: 000056405645fe10 R08: 00007fc8623d1460 R09: 000000007fffffff  
R10: 0000000000000000 R11: 0000000000000246 R12: 000000000000000c  
R13: 00007fc86241b780 R14: 00007fc862417600 R15: 00007fc862416a00  

In short, below scene breaks the lock dependency chain:  

 memory_failure  
  __page_handle_poison  
   zone_pcp_disable -- lock(pcp_batch_high_lock)  
   dissolve_free_huge_page  
    __hugetlb_vmemmap_restore_folio  
     static_key_slow_dec  
      cpus_read_lock -- rlock(cpu_hotplug_lock)  

Fix this by calling drain_all_pages() instead.  

This issue won't occur until commit a6b40850c442 ("mm: hugetlb: replace  
hugetlb_free_vmemmap_enabled with a static_key").  As it introduced  
rlock(cpu_hotplug_lock) in dissolve_free_huge_page() code path while  
lock(pcp_batch_high_lock) is already in the __page_handle_poison().  

[linmiaohe@huawei.com: extend comment per Oscar]  
[akpm@linux-foundation.org: reflow block comment]  
Link: https://lkml.kernel.org/r/20240407085456.2798193-1-linmiaohe@huawei.com  
Fixes: a6b40850c442 ("mm: hugetlb: replace hugetlb_free_vmemmap_enabled with a static_key")  
Signed-off-by: Miaohe Lin <linmiaohe@huawei.com>  
Acked-by: Oscar Salvador <osalvador@suse.de>  
Reviewed-by: Jane Chu <jane.chu@oracle.com>  
Cc: Naoya Horiguchi <nao.horiguchi@gmail.com>  
Cc: <stable@vger.kernel.org>  
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>

Signed-off-by: Aristeu Rozanski arozansk@redhat.com

Merge request reports