nwf

Apologies for the very long delay; life happened. I grabbed the Ubuntu Noble 25.8.1 image, ran through initial setup, and re-ran the `apt` commands in my initial post. This got me Linux 6.12.56-current-rockchip, since time has advanced. Alas, here too `modprobe zfs` crashes in much the same way: [ 430.862366] spl: loading out-of-tree module taints kernel. [ 430.910755] zfs: module license 'CDDL' taints kernel. [ 430.910773] Disabling lock debugging due to kernel taint [ 430.910832] zfs: module license taints kernel. [ 430.911221] 8<--- cut here --- [ 430.914682] Unable to handle kernel paging request at virtual address 5e56ce80 when read [ 430.923821] [5e56ce80] *pgd=00000000 [ 430.927842] Internal error: Oops: 5 [#1] SMP ARM [ 430.933004] Modules linked in: spl(O) squashfs algif_hash algif_skcipher af_alg sunrpc snd_soc_hdmi_codec snd_soc_simple_card snd_soc_simple_card_utils snd_usb_audio snd_soc_rockchip_i2s snd_hwdep hantro_vpu snd_usbmidi_lib snd_soc_core rockchip_vdec(C) snd_ump snd_rawmidi rockchip_iep v4l2_vp9 snd_pcm_dmaengine rockchip_rga snd_seq_device v4l2_jpeg panfrost r8723bs(C) snd_pcm v4l2_h264 videobuf2_dma_contig rk_crypto dw_hdmi_i2s_audio gpu_sched dw_hdmi_cec v4l2_mem2mem snd_timer crypto_engine dw_wdt rk3288_gpiomem snd soundcore rockchip_thermal cpufreq_dt zram zsmalloc binfmt_misc sch_fq_codel fuse gpio_keys syscon_reboot_mode reboot_mode realtek [ 430.997122] CPU: 0 UID: 0 PID: 3102 Comm: modprobe Tainted: P C O 6.12.56-current-rockchip #1 [ 431.008012] Tainted: [P]=PROPRIETARY_MODULE, [C]=CRAP, [O]=OOT_MODULE [ 431.015204] Hardware name: Rockchip (Device Tree) [ 431.020456] PC is at resolve_symbol+0x134/0x36c [ 431.025522] LR is at cmp_name+0x14/0x18 [ 431.029805] pc : [<b01ac008>] lr : [<b01ab0ec>] psr: 30070013 [ 431.036796] sp : f27c1d78 ip : f27c1cb8 fp : f27c1dcc [ 431.042622] r10: f27c1ed8 r9 : af317102 r8 : af3152b8 [ 431.048447] r7 : af2859c0 r6 : af285c48 r5 : f27c1e50 r4 : af592300 [ 431.055737] r3 : 00000000 r2 : 5e56ce70 r1 : af317102 r0 : 00000001 [ 431.063028] Flags: nzCV IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 431.071000] Control: 10c5387d Table: 06f2806a DAC: 00000051 [ 431.077407] Register r0 information: non-paged memory [ 431.083050] Register r1 information: 3-page vmalloc region starting at 0xaf315000 allocated at load_module+0x760/0x1b8c [ 431.095099] Register r2 information: non-paged memory [ 431.100740] Register r3 information: NULL pointer [ 431.105992] Register r4 information: 201-page vmalloc region starting at 0xaf57a000 allocated at load_module+0x760/0x1b8c [ 431.118242] Register r5 information: 2-page vmalloc region starting at 0xf27c0000 allocated at kernel_clone+0x9c/0x348 [ 431.130194] Register r6 information: 5-page vmalloc region starting at 0xaf284000 allocated at load_module+0x760/0x1b8c [ 431.142250] Register r7 information: 5-page vmalloc region starting at 0xaf284000 allocated at load_module+0x760/0x1b8c [ 431.154306] Register r8 information: 3-page vmalloc region starting at 0xaf315000 allocated at load_module+0x760/0x1b8c [ 431.166361] Register r9 information: 3-page vmalloc region starting at 0xaf315000 allocated at load_module+0x760/0x1b8c [ 431.178416] Register r10 information: 2-page vmalloc region starting at 0xf27c0000 allocated at kernel_clone+0x9c/0x348 [ 431.190470] Register r11 information: 2-page vmalloc region starting at 0xf27c0000 allocated at kernel_clone+0x9c/0x348 [ 431.202523] Register r12 information: 2-page vmalloc region starting at 0xf27c0000 allocated at kernel_clone+0x9c/0x348 [ 431.214578] Process modprobe (pid: 3102, stack limit = 0xa4dfd6e7) [ 431.221483] Stack: (0xf27c1d78 to 0xf27c2000) [ 431.226347] 1d60: b01ab490 b5f1dbc0 [ 431.235485] 1d80: 00000018 f2db7171 00000100 af2859c0 00000000 af3152b8 00000000 ac1f1f5c [ 431.244622] 1da0: b529a040 af592300 00005a8b 0006aa80 af59230c af7428b4 f2db7171 f27c1ed8 [ 431.253760] 1dc0: f27c1ebc f27c1dd0 b01ad948 b01abee0 f2e5c524 00000000 00000003 00000000 [ 431.262896] 1de0: 004ea3f8 00000000 00000001 b132d7f4 b132d80c b132d79c 00000000 00000000 [ 431.272033] 1e00: 00000028 b132d790 00000000 af592300 af5923fc 00000001 f2e5c4ac b1bd8b84 [ 431.281171] 1e20: 00000000 b1c8a214 b5eb76c0 f2e5c524 00000000 00455524 f27c1e64 f27c1e48 [ 431.290308] 1e40: 7fffffff 00000000 00455524 b5eb76c0 00000000 00455524 f27c1ebc f27c1e68 [ 431.299444] 1e60: b037a83c b06c390c 00000002 b02a4d7c f2a07000 00455524 00000001 00000000 [ 431.308581] 1e80: 00455524 00000000 00004004 ac1f1f5c f27c1ed4 00000000 b5eb76c0 004ea3f8 [ 431.317718] 1ea0: 00000000 b1c8a624 b663c708 b5eb76c0 f27c1f44 f27c1ec0 b01ae768 b01acb4c [ 431.326855] 1ec0: f27c1ed4 7fffffff 00000000 00000002 b0332b30 f2a07000 f2d154c1 f2c759c0 [ 431.335991] 1ee0: f2a07000 00455524 f2e5be6c f2e5bcec f2d8970c 00082824 000a81a4 000a639c [ 431.345128] 1f00: 000c6142 00000000 000a638c 00000028 00000029 00000016 00000000 0000001e [ 431.354266] 1f20: 00000000 ac1f1f5c 00000004 00000056 b1c8a214 00000001 f27c1fa4 f27c1f48 [ 431.363402] 1f40: b01aeb2c b01ae6d8 004ea3f8 b5eb76c0 00000000 b663c708 00000000 b1c8a37c [ 431.372539] 1f60: 00000000 00000000 f27c1f68 f27c1f68 fffffffc ac1f1f5c 00000006 004ec5bc [ 431.381676] 1f80: 0210c4ec c8b8ca00 0000017b b01002c8 b529a040 0000017b 00000000 f27c1fa8 [ 431.390812] 1fa0: b0100060 b01ae9a8 004ec5bc 0210c4ec 00000004 004ea3f8 00000000 004eba84 [ 431.399950] 1fc0: 004ec5bc 0210c4ec c8b8ca00 0000017b 0210c550 a6f40cd8 0210b190 ae8cb2b4 [ 431.409086] 1fe0: ae8cb170 ae8cb160 004e2707 a6b97ca2 40070030 00000004 00000000 00000000 [ 431.418222] Call trace: [ 431.421046] resolve_symbol from load_module+0xe08/0x1b8c [ 431.427084] r10:f27c1ed8 r9:f2db7171 r8:af7428b4 r7:af59230c r6:0006aa80 r5:00005a8b [ 431.435831] r4:af592300 [ 431.438654] load_module from init_module_from_file+0x9c/0xdc [ 431.445080] r10:b5eb76c0 r9:b663c708 r8:b1c8a624 r7:00000000 r6:004ea3f8 r5:b5eb76c0 [ 431.453827] r4:00000000 [ 431.456649] init_module_from_file from sys_finit_module+0x190/0x2cc [ 431.463754] r6:00000001 r5:b1c8a214 r4:00000056 [ 431.468907] sys_finit_module from ret_fast_syscall+0x0/0x54 [ 431.475232] Exception stack(0xf27c1fa8 to 0xf27c1ff0) [ 431.480864] 1fa0: 004ec5bc 0210c4ec 00000004 004ea3f8 00000000 004eba84 [ 431.490002] 1fc0: 004ec5bc 0210c4ec c8b8ca00 0000017b 0210c550 a6f40cd8 0210b190 ae8cb2b4 [ 431.499138] 1fe0: ae8cb170 ae8cb160 004e2707 a6b97ca2 [ 431.504772] r10:0000017b r9:b529a040 r8:b01002c8 r7:0000017b r6:c8b8ca00 r5:0210c4ec [ 431.513519] r4:004ec5bc [ 431.516343] Code: ea000036 e5922000 e1560002 0a000033 (e5923010) [ 431.523419] ---[ end trace 0000000000000000 ]---

A crash in the Linux kernel's module loader is not really ZFS's fault, though, right? There's nothing about `resolve_symbol` that's part of ZFS.

I grabbed the "Armbian 25.8.1 Bookworm Minimal / IOT" image from https://www.armbian.com/tinkerboard/, landed it on a microSD card, booted it, and ran apt update apt upgrade apt install linux-headers-current-rockchip apt install zfs-dkms zfsutils-linux This gets me linux-image-current-rockchip 25.8.1 (kernel 6.12.44-current-rockchip) and a module built from zfs-dkms 2.3.2-2~bpo12+2. So far so good. And then I ran modprobe zfs.ko and was rewarded with the following panic in the kernel's module loader: [ 1128.728803] spl: loading out-of-tree module taints kernel. [ 1128.859152] zfs: module license 'CDDL' taints kernel. [ 1128.859178] Disabling lock debugging due to kernel taint [ 1128.859401] zfs: module license taints kernel. [ 1128.860612] 8<--- cut here --- [ 1128.860624] Unable to handle kernel paging request at virtual address 5e4f51d0 when read [ 1128.860641] [5e4f51d0] *pgd=00000000 [ 1128.860663] Internal error: Oops: 5 [#1] SMP ARM [ 1128.865857] Modules linked in: spl(O) zram zsmalloc snd_soc_hdmi_codec snd_usb_audio snd_soc_simple_card binfmt_misc snd_soc_rockchip_i2s snd_soc_simple_card_utils snd_hwdep hantro_vpu snd_usbmidi_lib rockchip_vdec(C) snd_ump snd_soc_core r8723bs(C) rockchip_rga rockchip_iep snd_rawmidi v4l2_vp9 panfrost snd_pcm_dmaengine v4l2_jpeg rk_crypto snd_seq_device v4l2_h264 dw_hdmi_i2s_audio snd_pcm videobuf2_dma_contig gpu_sched dw_hdmi_cec v4l2_mem2mem snd_timer syscon_reboot_mode crypto_engine reboot_mode snd dw_wdt rk3288_gpiomem soundcore rockchip_thermal cpufreq_dt fuse gpio_keys realtek [ 1128.924379] CPU: 3 UID: 0 PID: 20465 Comm: modprobe Tainted: P C O 6.12.44-current-rockchip #1 [ 1128.935373] Tainted: [P]=PROPRIETARY_MODULE, [C]=CRAP, [O]=OOT_MODULE [ 1128.942570] Hardware name: Rockchip (Device Tree) [ 1128.947822] PC is at resolve_symbol+0xf8/0x368 [ 1128.952795] LR is at cmp_name+0x14/0x18 [ 1128.957081] pc : [<b01ac790>] lr : [<b01ab960>] psr: 90010013 [ 1128.964083] sp : f1f1dd68 ip : f1f1dca8 fp : f1f1ddbc [ 1128.969919] r10: f1f1ded0 r9 : af1f81dc r8 : af1f62ac [ 1128.975755] r7 : af2489c0 r6 : af248c48 r5 : f1f1de48 r4 : af52c300 [ 1128.983049] r3 : 00000000 r2 : 5e4f51c0 r1 : 00000000 r0 : 00000001 [ 1128.990342] Flags: NzcV IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 1128.998318] Control: 10c5387d Table: 0364c06a DAC: 00000051 [ 1129.004737] Register r0 information: non-paged memory [ 1129.010383] Register r1 information: NULL pointer [ 1129.015639] Register r2 information: non-paged memory [ 1129.021283] Register r3 information: NULL pointer [ 1129.026538] Register r4 information: 201-page vmalloc region starting at 0xaf514000 allocated at load_module+0x744/0x1b34 [ 1129.038795] Register r5 information: 2-page vmalloc region starting at 0xf1f1c000 allocated at kernel_clone+0xac/0x340 [ 1129.050759] Register r6 information: 5-page vmalloc region starting at 0xaf247000 allocated at load_module+0x744/0x1b34 [ 1129.062820] Register r7 information: 5-page vmalloc region starting at 0xaf247000 allocated at load_module+0x744/0x1b34 [ 1129.074881] Register r8 information: 3-page vmalloc region starting at 0xaf1f6000 allocated at load_module+0x744/0x1b34 [ 1129.086942] Register r9 information: 3-page vmalloc region starting at 0xaf1f6000 allocated at load_module+0x744/0x1b34 [ 1129.099003] Register r10 information: 2-page vmalloc region starting at 0xf1f1c000 allocated at kernel_clone+0xac/0x340 [ 1129.111062] Register r11 information: 2-page vmalloc region starting at 0xf1f1c000 allocated at kernel_clone+0xac/0x340 [ 1129.123120] Register r12 information: 2-page vmalloc region starting at 0xf1f1c000 allocated at kernel_clone+0xac/0x340 [ 1129.135179] Process modprobe (pid: 20465, stack limit = 0xf191b45c) [ 1129.142183] Stack: (0xf1f1dd68 to 0xf1f1e000) [ 1129.147050] dd60: b01abd14 af52c3fc 00000000 f26174f8 00000100 af2489c0 [ 1129.156193] dd80: 00000000 af1f62ac 00000000 e7df2e2d b4795600 00005a2b f26174f8 f26bcb48 [ 1129.165334] dda0: af6dc2b4 00000000 0006a3d0 f1f1ded0 f1f1deb4 f1f1ddc0 b01ae384 b01ac6a4 [ 1129.174476] ddc0: b02f81d4 b02f2340 f26bcbc0 00000000 00000003 004d6d38 00000000 00000001 [ 1129.183618] dde0: 00000000 b4795600 f26bc508 00000000 af52c3fc 0000001e 00000000 b132bdb0 [ 1129.192759] de00: 00000000 af52c300 af52c300 b1bd8b84 af52c30c b1c8a1a4 00000000 00000001 [ 1129.201901] de20: b5b7e600 f26bcbc0 00459bc0 00459bc0 f1f1de5c f1f1de40 00459bc0 00000000 [ 1129.211041] de40: 7fffffff b5b7e600 00000000 00000000 00000000 00000000 00000000 00000000 [ 1129.220182] de60: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 1129.229324] de80: 00000000 e7df2e2d 00000002 00000000 b5b7e600 004d6d38 b5b7e600 b1c8a5b4 [ 1129.238465] dea0: bbba0708 b5b7e600 f1f1df3c f1f1deb8 b01aeedc b01ad31c f1f1decc 7fffffff [ 1129.247607] dec0: 00000000 00000002 00000000 f2263000 f2576523 f24d6640 f2263000 00459bc0 [ 1129.256749] dee0: f26bc508 f26bc388 f25e9c54 0008272c 000a7eac 000a5a90 000c5d78 00000000 [ 1129.265890] df00: 000a5a80 00000028 00000029 00000016 00000000 0000001e 00000000 e7df2e2d [ 1129.275031] df20: 00000001 000000f6 b1c8a1a4 00000001 f1f1dfa4 f1f1df40 b01af2ac b01aee50 [ 1129.284172] df40: 00000000 00000000 004d6d38 b1c8a58c bbba0708 00000000 b1c8a58c 00000000 [ 1129.293314] df60: 00000000 f1f1df64 f1f1df64 fffffffc 00000000 e7df2e2d 000000c0 004d8ad8 [ 1129.302456] df80: 00fa74bc 22b57200 0000017b b01002c8 b4795600 0000017b 00000000 f1f1dfa8 [ 1129.311598] dfa0: b0100060 b01af108 004d8ad8 00fa74bc 00000004 004d6d38 00000000 004d86bc [ 1129.320738] dfc0: 004d8ad8 00fa74bc 22b57200 0000017b 00fa7520 00000000 00000000 004d50bc [ 1129.329879] dfe0: aeb5c0e8 aeb5c0d8 004cf149 a6bc3002 40070030 00000004 00000000 00000000 [ 1129.339018] Call trace: [ 1129.341845] resolve_symbol from load_module+0x1074/0x1b34 [ 1129.347983] r10:f1f1ded0 r9:0006a3d0 r8:00000000 r7:af6dc2b4 r6:f26bcb48 r5:f26174f8 [ 1129.356735] r4:00005a2b [ 1129.359559] load_module from init_module_from_file+0x98/0xd4 [ 1129.365987] r10:b5b7e600 r9:bbba0708 r8:b1c8a5b4 r7:b5b7e600 r6:004d6d38 r5:b5b7e600 [ 1129.374738] r4:00000000 [ 1129.377562] init_module_from_file from sys_finit_module+0x1b0/0x310 [ 1129.384670] r6:00000001 r5:b1c8a1a4 r4:000000f6 [ 1129.389826] sys_finit_module from ret_fast_syscall+0x0/0x54 [ 1129.396154] Exception stack(0xf1f1dfa8 to 0xf1f1dff0) [ 1129.401798] dfa0: 004d8ad8 00fa74bc 00000004 004d6d38 00000000 004d86bc [ 1129.410939] dfc0: 004d8ad8 00fa74bc 22b57200 0000017b 00fa7520 00000000 00000000 004d50bc [ 1129.420079] dfe0: aeb5c0e8 aeb5c0d8 004cf149 a6bc3002 [ 1129.425724] r10:0000017b r9:b4795600 r8:b01002c8 r7:0000017b r6:22b57200 r5:00fa74bc [ 1129.434475] r4:004d8ad8 [ 1129.437302] Code: ea00004a e5922000 e1520006 0a000047 (e5923010) [ 1129.444162] ---[ end trace 0000000000000000 ]--- I have been unable to find debug symbols or a way to get an objdump of the loaded kernel (probably I am just missing something). Please advise what more I can tell you to be useful. I am unable to run armbianmonitor; after the above, lsmod hangs, and upon boot, the machine attempts to load zfs.ko which again crashes as above and userspace is pretty direly stuck. FWIW, I am happily running ZFS on a Tinkerboard with an older system, specifically linux-image-current-rockchip 24.11.1 (which is kernel 6.6.63-current-rockchip) and zfs-dkms 2.2.2-0ubuntu9.1.