Storage - swraid simple_reshape: BUG bio-152 (Not tainted): Objects remaining in bio-152 on __kmem_cache_shutdown()
Snippet of test failure
# https://s3.us-east-1.amazonaws.com/arr-cki-prod-datawarehouse-public/datawarehouse-public/2022/07/05/redhat:580450311/build_aarch64_redhat:580450311_aarch64/tests/5/results_0001/console.log/console.log
[ 6059.398582] Running test [R:12239809 T:76 - Storage - swraid simple_reshape]
[ 6093.883316] loop0: detected capacity change from 0 to 1024000
[ 6094.101045] loop1: detected capacity change from 0 to 1024000
[ 6094.203518] loop2: detected capacity change from 0 to 1024000
[ 6094.206969] I/O error, dev loop2, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 6094.232636] Buffer I/O error on dev loop2, logical block 0, async page read
[ 6094.320567] loop3: detected capacity change from 0 to 1024000
[ 6094.321166] I/O error, dev loop3, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 6094.454047] Buffer I/O error on dev loop3, logical block 0, async page read
[ 6094.542255] loop4: detected capacity change from 0 to 1024000
[ 6094.542878] I/O error, dev loop4, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 6094.545828] Buffer I/O error on dev loop4, logical block 0, async page read
[ 6094.638089] loop5: detected capacity change from 0 to 1024000
[ 6094.638790] I/O error, dev loop5, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 6094.642562] Buffer I/O error on dev loop5, logical block 0, async page read
[ 6094.766371] loop6: detected capacity change from 0 to 1024000
[ 6094.767200] I/O error, dev loop6, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 6094.769965] Buffer I/O error on dev loop6, logical block 0, async page read
[ 6094.870136] loop7: detected capacity change from 0 to 1024000
[ 6094.870918] I/O error, dev loop7, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 6094.873774] Buffer I/O error on dev loop7, logical block 0, async page read
[ 6094.986134] loop8: detected capacity change from 0 to 1024000
[ 6094.986816] I/O error, dev loop8, sector 0 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 6094.989697] Buffer I/O error on dev loop8, logical block 0, async page read
[ 6098.657460] md0: detected capacity change from 0 to 3059712
[ 6099.015709] =============================================================================
[ 6099.017781] BUG bio-152 (Not tainted): Objects remaining in bio-152 on __kmem_cache_shutdown()
[ 6099.019779] -----------------------------------------------------------------------------
[ 6099.019779]
[ 6099.021994] Slab 0x00000000656e5f6a objects=16 used=1 fp=0x00000000128cc6ea flags=0x2fffff00000200(slab|node=0|zone=2|lastcpupid=0xfffff)
[ 6099.024825] CPU: 2 PID: 293174 Comm: mdadm Not tainted 5.19.0-rc5 #1
[ 6099.026389] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[ 6099.028106] Call trace:
[ 6099.028733] dump_backtrace+0xc4/0x130
[ 6099.029785] show_stack+0x24/0x60
[ 6099.030637] dump_stack_lvl+0x64/0x80
[ 6099.031593] dump_stack+0x18/0x34
[ 6099.032401] slab_err+0xb0/0xc0
[ 6099.033187] list_slab_objects.constprop.0+0x4c/0x220
[ 6099.034459] __kmem_cache_shutdown+0x128/0x1f0
[ 6099.035570] kmem_cache_destroy+0x78/0x170
[ 6099.036586] bioset_exit+0x190/0x1e4
[ 6099.037488] acct_bioset_exit+0x20/0x30
[ 6099.038473] raid0_free+0x44/0x54 [raid0]
[ 6099.039492] level_store+0x2bc/0x704
[ 6099.040389] md_attr_store+0xac/0x150
[ 6099.041293] sysfs_kf_write+0x50/0x60
[ 6099.042213] kernfs_fop_write_iter+0x124/0x1f0
[ 6099.043321] new_sync_write+0xc4/0x130
[ 6099.044248] vfs_write+0x240/0x29c
[ 6099.045106] ksys_write+0x68/0xec
[ 6099.045928] __arm64_sys_write+0x28/0x34
[ 6099.046896] invoke_syscall+0x78/0x100
[ 6099.047827] el0_svc_common.constprop.0+0x4c/0xf4
[ 6099.049066] do_el0_svc+0x38/0x4c
[ 6099.049919] el0_svc+0x34/0x100
[ 6099.050709] el0t_64_sync_handler+0x11c/0x150
[ 6099.051787] el0t_64_sync+0x190/0x194
[ 6099.052697] Disabling lock debugging due to kernel taint
[ 6099.054022] Object 0x00000000aa1a6977 @offset=512
[ 6099.055304] ------------[ cut here ]------------
[ 6099.056457] kmem_cache_destroy bio-152: Slab cache still has objects when called from bioset_exit+0x190/0x1e4
[ 6099.056511] WARNING: CPU: 2 PID: 293174 at mm/slab_common.c:506 kmem_cache_destroy+0x160/0x170
[ 6099.061085] Modules linked in: raid10 raid0 nf_conntrack_netlink tun loop xt_addrtype xt_nat xt_mark xt_comment xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_compat nf_tables veth bridge stp llc nfnetlink overlay rfkill sunrpc vfat fat virtio_net net_failover failover scsi_dh_rdac scsi_dh_emc scsi_dh_alua dm_multipath i2c_dev fuse drm zram xfs crct10dif_ce virtio_blk ghash_ce virtio_console virtio_mmio ipmi_devintf ipmi_msghandler qemu_fw_cfg [last unloaded: dm_persistent_data]
[ 6099.072617] CPU: 2 PID: 293174 Comm: mdadm Tainted: G B 5.19.0-rc5 #1
[ 6099.074557] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[ 6099.076294] pstate: 00400005 (nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 6099.078057] pc : kmem_cache_destroy+0x160/0x170
[ 6099.079212] lr : kmem_cache_destroy+0x160/0x170
[ 6099.080367] sp : ffff80000c723950
[ 6099.081220] x29: ffff80000c723950 x28: ffffcc524687d078 x27: ffffcc524688c308
[ 6099.083006] x26: ffff53d08f148298 x25: ffff53d08f1480d8 x24: 0000000000000000
[ 6099.084773] x23: ffffcc52b4ef10b0 x22: ffffcc52b4afc580 x21: ffffcc52b2c23cc0
[ 6099.129394] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000
[ 6099.158818] x20: ffffcc52b4a9b9f0
[ 6099.161453] Mem abort info:
[ 6099.161453] x19: ffff53d007d4c400 x18: ffffffffffffffff
[ 6099.162349] ESR = 0x0000000096000044
[ 6099.163094]
[ 6099.164434] EC = 0x25: DABT (current EL), IL = 32 bits
[ 6099.165392] x17: 2061653663633832
[ 6099.165794] SET = 0, FnV = 0
[ 6099.167173] x16: 3130303030303030
[ 6099.168010] EA = 0, S1PTW = 0
[ 6099.168766] x15: 00000000fffffffd
[ 6099.169646] FSC = 0x04: level 0 translation fault
[ 6099.170436]
[ 6099.171317] Data abort info:
[ 6099.172539] x14: ffff80008c723420
[ 6099.172930] ISV = 0, ISS = 0x00000044
[ 6099.173686] x13: ffff80000c72342a
[ 6099.174598] CM = 0, WnR = 1
[ 6099.175564] x12: 6c6c6163206e6568
[ 6099.176433] user pgtable: 4k pages, 48-bit VAs, pgdp=000000010c24f000
[ 6099.177196]
[ 6099.178101] [0000000000000000] pgd=0000000000000000
[ 6099.179716] x11: 00000000ffffdfff
[ 6099.180096] , p4d=0000000000000000
[ 6099.181320] x10: ffffcc52b4a20258
[ 6099.182178]
[ 6099.183025] x9 : ffffcc52b2664db0
[ 6099.183878] Internal error: Oops: 96000044 [#1] SMP
[ 6099.184248]
[ 6099.185131] Modules linked in:
[ 6099.186356] x8 : 000000000002ffe8
[ 6099.186760] raid10
[ 6099.187546] x7 : c0000000ffffdfff
[ 6099.188410] raid0
[ 6099.188976] x6 : 00000000000affa8
[ 6099.189892] nf_conntrack_netlink
[ 6099.190415]
[ 6099.191297] tun
[ 6099.192120] x5 : 0000000000001fff
[ 6099.192500] loop
[ 6099.192968] x4 : 0000000000000001
[ 6099.193810] xt_addrtype
[ 6099.194298] x3 : ffffcc52b43a5008
[ 6099.195164] xt_nat
[ 6099.195801]
[ 6099.196711] xt_mark
[ 6099.197281] x2 : 0000000000000001
[ 6099.197693] xt_comment
[ 6099.198304] x1 : ffff53d0755ec400
[ 6099.199150] xt_conntrack
[ 6099.199807] x0 : 0000000000000061
[ 6099.200719] nft_chain_nat
[ 6099.201404]
[ 6099.202252] xt_MASQUERADE
[ 6099.202939] Call trace:
[ 6099.203329] nf_nat
[ 6099.204026] kmem_cache_destroy+0x160/0x170
[ 6099.204644] nf_conntrack
[ 6099.205181] bioset_exit+0x190/0x1e4
[ 6099.206251] nf_defrag_ipv6
[ 6099.206900] acct_bioset_exit+0x20/0x30
[ 6099.207810] nf_defrag_ipv4
[ 6099.208522] raid0_free+0x44/0x54 [raid0]
[ 6099.209541] nft_compat
[ 6099.210272] level_store+0x2bc/0x704
[ 6099.211295] nf_tables
[ 6099.211916] md_attr_store+0xac/0x150
[ 6099.212830] veth
[ 6099.213430] sysfs_kf_write+0x50/0x60
[ 6099.214387] bridge
[ 6099.214896] kernfs_fop_write_iter+0x124/0x1f0
[ 6099.215859] stp
[ 6099.216390] new_sync_write+0xc4/0x130
[ 6099.217529] llc
[ 6099.217994] vfs_write+0x240/0x29c
[ 6099.218990] nfnetlink
[ 6099.219511] ksys_write+0x68/0xec
[ 6099.220404] overlay
[ 6099.221028] __arm64_sys_write+0x28/0x34
[ 6099.221909] rfkill
[ 6099.222476] invoke_syscall+0x78/0x100
[ 6099.223498] sunrpc
[ 6099.224037] el0_svc_common.constprop.0+0x4c/0xf4
[ 6099.225019] vfat
[ 6099.225565] do_el0_svc+0x38/0x4c
[ 6099.226782] fat
[ 6099.227286] el0_svc+0x34/0x100
[ 6099.228171] virtio_net
[ 6099.228673] el0t_64_sync_handler+0x11c/0x150
[ 6099.229482] net_failover
[ 6099.230113] el0t_64_sync+0x190/0x194
[ 6099.231228] failover
[ 6099.231883] ---[ end trace 0000000000000000 ]---
[ 6099.232800] scsi_dh_rdac
[ 6099.233659] md/raid10:md0: active with 3 out of 6 devices
[ 6099.234917] scsi_dh_emc scsi_dh_alua dm_multipath i2c_dev fuse drm zram xfs crct10dif_ce virtio_blk ghash_ce virtio_console virtio_mmio ipmi_devintf ipmi_msghandler qemu_fw_cfg [last unloaded: dm_persistent_data]
[ 6099.316128] CPU: 1 PID: 21 Comm: ksoftirqd/1 Tainted: G B W 5.19.0-rc5 #1
[ 6099.318162] Hardware name: QEMU KVM Virtual Machine, BIOS 0.0.0 02/06/2015
[ 6099.320167] pstate: 804000c5 (Nzcv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 6099.353110] pc : mempool_free+0x88/0xbc
[ 6099.354099] lr : mempool_free+0x68/0xbc
[ 6099.355063] sp : ffff800008123bb0
[ 6099.355927] x29: ffff800008123bb0 x28: ffffcc52b492ec48 x27: ffffcc52b39f6000
[ 6099.357685] x26: ffffcc52b43a7a90 x25: 0000000000000000 x24: 000000000000000a
[ 6099.359488] x23: 0000000000000000 x22: 0000000000000001 x21: 00000000ffffffff
[ 6099.361348] x20: ffff53d02093c200 x19: ffff53d08f148708 x18: 00000000002c0000
[ 6099.363202] x17: 0b400000c0fd0500 x16: ffffcc52b2c378e4 x15: 1214723fc9c25693
[ 6099.365052] x14: 2d3211055482449e x13: 1214723fc9c25693 x12: 2d3211055482449e
[ 6099.367322] x11: ffffcc52b492f6d8 x10: 0000000000001d90 x9 : ffffcc52b34f8178
[ 6099.369206] x8 : ffff53d0003b3ff0 x7 : 0000000000000001 x6 : 000006b64153e015
[ 6099.371039] x5 : 03ffffffffffffff x4 : 0000000000000101 x3 : 0000000000000000
[ 6099.372908] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff53d08f148708
[ 6099.374766] Call trace:
[ 6099.375387] mempool_free+0x88/0xbc
[ 6099.376301] bio_free+0x4c/0x6c
[ 6099.377166] bio_put+0xbc/0x150
[ 6099.377967] md_end_io_acct+0x48/0x60
[ 6099.378922] bio_endio+0x138/0x1f0
[ 6099.379814] blk_update_request+0x1e0/0x464
[ 6099.380896] blk_mq_end_request+0x30/0x16c
[ 6099.381972] lo_complete_rq+0x7c/0xc0 [loop]
[ 6099.383088] blk_complete_reqs+0x5c/0x74
[ 6099.384134] blk_done_softirq+0x3c/0x50
[ 6099.385119] __do_softirq+0x170/0x418
[ 6099.386057] run_ksoftirqd+0x64/0x80
[ 6099.387020] smpboot_thread_fn+0x1e8/0x220
[ 6099.388123] kthread+0xf0/0x100
[ 6099.388935] ret_from_fork+0x10/0x20
[ 6099.389856] Code: f9400a63 11000440 b9000a60 aa1303e0 (f822d874)
[ 6099.391520] ---[ end trace 0000000000000000 ]---
[ 6099.392694] Kernel panic - not syncing: Oops: Fatal exception in interrupt
[ 6099.394408] SMP: stopping secondary CPUs
[ 6099.395940] Kernel Offset: 0x4c52aa530000 from 0xffff800008000000
[ 6099.397509] PHYS_OFFSET: 0xffffac3100000000
[ 6099.398549] CPU features: 0x0020,00045021,19001080
[ 6099.399758] Memory Limit: none Test logs on DataWarehouse
DataWarehouse issue
-
DW issue : https://datawarehouse.cki-project.org/issue/1396
-
Regex: https://datawarehouse.cki-project.org/issue/-/regex/1362- Text Match :
BUG bio-152 \(Not tainted\): Objects remaining in bio-152 on __kmem_cache_shutdown\(\) - (Log) File Name Match :
console.log - Test Name Match :
Storage - swraid simple_reshape - Tree Name Match :
arm-next
- Text Match :
-
Additional details
N/A
In case opening the links above result in a 404 page on DataWarehouse, please make sure you are correctly logged into DataWarehouse via Red Hat SSO.