September 8Sep 8 Hello, after upgrading my xu4 to trixie I have begun seeing the oom killer wrecking havoc! I had 1Gb of swap and have upgraded it to have 6Gb of swap but it does not seem to help. I have set vm.panic_on_oom = 1 as without this it just keeps killing tasks and then says deadlocked on memory here are 2 seperate boots of the os any help would be appreciated [ 304.195611] dnsmasq invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=0, oom_score_adj=0 [ 304.202725] CPU: 2 PID: 1322 Comm: dnsmasq Not tainted 6.6.141-current-odroidxu4 #1 [ 304.210252] Hardware name: Samsung Exynos (Flattened Device Tree) [ 304.216332] unwind_backtrace from show_stack+0x10/0x14 [ 304.221515] show_stack from dump_stack_lvl+0x40/0x4c [ 304.226541] dump_stack_lvl from dump_header+0x4c/0x208 [ 304.231740] dump_header from out_of_memory+0x2a8/0x344 [ 304.236940] out_of_memory from __alloc_pages+0xcd4/0x122c [ 304.242398] __alloc_pages from __get_free_pages+0x14/0x24 [ 304.247857] __get_free_pages from __pollwait+0xa8/0xf8 [ 304.253056] __pollwait from datagram_poll+0x28/0xfc [ 304.257996] datagram_poll from udp_poll+0x10/0x88 [ 304.262762] udp_poll from sock_poll+0x50/0xd0 [ 304.267181] sock_poll from do_sys_poll+0x350/0x5e8 [ 304.272033] do_sys_poll from sys_poll+0x40/0x110 [ 304.276711] sys_poll from ret_fast_syscall+0x0/0x54 [ 304.281648] Exception stack(0xf2795fa8 to 0xf2795ff0) [ 304.286680] 5fa0: 00000000 00000000 00551a40 00000018 ffffffff 00000000 [ 304.294829] 5fc0: 00000000 00000000 00000000 000000a8 00000019 ffffffff 00545430 bea589e8 [ 304.302972] 5fe0: 000000a8 bea58958 b6bdeb49 b6b9f736 [ 304.308091] Mem-Info: [ 304.310257] active_anon:1897 inactive_anon:70334 isolated_anon:0 [ 304.310257] active_file:11947 inactive_file:40757 isolated_file:0 [ 304.310257] unevictable:4754 dirty:3 writeback:0 [ 304.310257] slab_reclaimable:5012 slab_unreclaimable:178269 [ 304.310257] mapped:27742 shmem:4819 pagetables:1382 [ 304.310257] sec_pagetables:0 bounce:0 [ 304.310257] kernel_misc_reclaimable:0 [ 304.310257] free:183203 free_pcp:3 free_cma:31735 [ 304.350026] Node 0 active_anon:7588kB inactive_anon:281336kB active_file:47552kB inactive_file:163464kB unevictable:19016kB isolated(anon):0kB isolated(file):0kB mapped:110968kB dirty:12kB writeback:0kB shmem:19276kB writeback_tmp:0kB kernel_stack:2936kB pagetables:5528kB sec_pagetables:0kB all_unreclaimable? yes [ 304.377596] Normal free:3908kB boost:0kB min:3416kB low:4268kB high:5120kB reserved_highatomic:8192KB active_anon:0kB inactive_anon:0kB active_file:1620kB inactive_file:0kB unevictable:0kB writepending:0kB present:786432kB managed:748292kB mlocked:0kB bounce:0kB free_pcp:12kB local_pcp:0kB free_cma:0kB [ 304.404303] lowmem_reserve[]: 0 10064 10064 [ 304.408439] Normal: 260*4kB (UME) 187*8kB (UME) 58*16kB (UME) 7*32kB (UE) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB = 3688kB [ 304.421806] 59668 total pagecache pages [ 304.425500] 0 pages in swap cache [ 304.428884] Free swap = 6011596kB [ 304.432169] Total swap = 6011596kB [ 304.435552] 518656 pages RAM [ 304.438528] 322048 pages HighMem/MovableOnly [ 304.442657] 9535 pages reserved [ 304.445778] 32768 pages cma reserved [ 304.449473] Unreclaimable slab info: [ 304.452883] Name Used Total [ 304.458357] nf_conntrack 128KB 128KB [ 304.463465] rpc_buffers 256KB 256KB [ 304.468815] RAWv6 159KB 159KB [ 304.473856] UDPv6 256KB 256KB [ 304.479150] tw_sock_TCPv6 39KB 39KB [ 304.484252] request_sock_TCPv6 47KB 47KB [ 304.489632] TCPv6 248KB 248KB [ 304.494742] btrfs_delayed_ref_head 59KB 59KB [ 304.500498] btrfs_ordered_extent 39KB 39KB [ 304.505975] btrfs_extent_map 179KB 179KB [ 304.511028] bio-240 8KB 8KB [ 304.516319] bio-288 78KB 78KB [ 304.521419] btrfs_path 31KB 31KB [ 304.526676] bio-304 7KB 7KB [ 304.531817] bio-336 7KB 7KB [ 304.537068] scsi_sense_cache 16KB 16KB [ 304.542219] bio-96 32KB 32KB [ 304.547473] bio-208 8KB 8KB [ 304.552613] mqueue_inode_cache 31KB 31KB [ 304.557951] fuse_request 3KB 3KB [ 304.563100] ecryptfs_headers 1760KB 1760KB [ 304.568351] ecryptfs_file_cache 8KB 8KB [ 304.573679] audit_tree_mark 43KB 43KB [ 304.578913] posix_timers_cache 15KB 15KB [ 304.584156] ip4-frags 172KB 179KB [ 304.589416] RAW 15KB 15KB [ 304.594557] UDP 252KB 252KB [ 304.599801] tw_sock_TCP 23KB 23KB [ 304.604955] request_sock_TCP 31KB 31KB [ 304.610197] TCP 246KB 246KB [ 304.615351] bio-192 8KB 8KB [ 304.620607] ep_head 32KB 32KB [ 304.625750] sgpool-128 30KB 30KB [ 304.630989] sgpool-64 218KB 218KB [ 304.636223] sgpool-32 109KB 109KB [ 304.641351] sgpool-16 62KB 62KB [ 304.646672] request_queue 111KB 111KB [ 304.651754] bio-144 145KB 145KB [ 304.657027] biovec-max 480KB 480KB [ 304.662148] biovec-128 63KB 63KB [ 304.667417] biovec-64 126KB 126KB [ 304.672547] user_namespace 63KB 63KB [ 304.677842] skbuff_small_head 312KB 375KB [ 304.682946] skbuff_fclone_cache 63KB 63KB [ 304.688370] skbuff_head_cache 208KB 228KB [ 304.693514] exynos-iommu-lv2table 32KB 32KB [ 304.699140] file_lock_cache 40KB 40KB [ 304.704266] file_lock_ctx 32KB 32KB [ 304.709547] fsnotify_mark_connector 31KB 31KB [ 304.715184] taskstats 109KB 109KB [ 304.720452] proc_dir_entry 236KB 236KB [ 304.725576] pde_opener 31KB 31KB [ 304.730837] seq_file 31KB 31KB [ 304.736035] sigqueue 39KB 39KB [ 304.741180] shmem_inode_cache 2004KB 2030KB [ 304.746442] kernfs_iattrs_cache 3146KB 3146KB [ 304.751753] kernfs_node_cache 6333KB 6343KB [ 304.757073] filp 864KB 1019KB [ 304.762151] names_cache 256KB 256KB [ 304.767419] net_namespace 125KB 125KB [ 304.772549] lsm_file_cache 67KB 72KB [ 304.777824] key_jar 70KB 70KB [ 304.782940] uts_namespace 63KB 63KB [ 304.788213] nsproxy 31KB 31KB [ 304.793346] vm_area_struct 1081KB 1173KB [ 304.798581] files_cache 72KB 72KB [ 304.803737] signal_cache 598KB 598KB [ 304.808983] sighand_cache 661KB 661KB [ 304.814138] task_struct 1235KB 1320KB [ 304.819399] cred_jar 180KB 216KB [ 304.824542] anon_vma_chain 468KB 528KB [ 304.829773] anon_vma 426KB 463KB [ 304.834931] pid 68KB 68KB [ 304.840203] trace_event_file 151KB 151KB [ 304.845344] maple_node 464KB 808KB [ 304.850612] task_group 128KB 128KB [ 304.855731] mm_struct 158KB 158KB [ 304.861015] vmap_area 366KB 370KB [ 304.866166] kmalloc-cg-4k 448KB 448KB [ 304.871325] kmalloc-cg-2k 384KB 384KB [ 304.876563] kmalloc-cg-1k 704KB 768KB [ 304.881725] kmalloc-cg-512 374KB 432KB [ 304.886969] kmalloc-cg-256 72KB 72KB [ 304.892123] kmalloc-cg-192 51KB 51KB [ 304.897382] kmalloc-cg-128 60KB 60KB [ 304.902525] kmalloc-cg-64 170KB 180KB [ 304.907768] kmalloc-8k 7424KB 7424KB [ 304.912923] kmalloc-4k 928KB 928KB [ 304.918165] kmalloc-2k 1268KB 1344KB [ 304.923325] kmalloc-1k 2926KB 3008KB [ 304.928567] kmalloc-512 2050KB 2080KB [ 304.933723] kmalloc-256 1158KB 1168KB [ 304.938971] kmalloc-192 478KB 492KB [ 304.944136] kmalloc-128 1194KB 1260KB [ 304.949372] kmalloc-64 653364KB 653364KB [ 304.954516] kmem_cache_node 28KB 28KB [ 304.959773] kmem_cache 47KB 47KB [ 304.964916] Tasks state (memory values in pages): [ 304.969653] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [ 304.978284] [ 290] 0 290 6504 2244 24576 0 -250 systemd-journal [ 304.987251] [ 345] 0 345 7622 2112 26624 0 -1000 systemd-udevd [ 304.996090] [ 621] 109 621 1262 576 10240 0 0 rpcbind [ 305.004345] [ 743] 0 743 904 288 10240 0 0 blkmapd [ 305.012710] [ 758] 0 758 882 544 8192 0 0 nfsdcld [ 305.021023] [ 773] 121 773 4490 1888 22528 0 0 systemd-network [ 305.030040] [ 855] 108 855 1839 800 14336 0 0 avahi-daemon [ 305.038786] [ 871] 107 871 2123 1112 14336 0 -900 dbus-daemon [ 305.047449] [ 886] 0 886 840 576 8192 0 0 fsidd [ 305.055556] [ 908] 0 908 13970 1141 36864 0 0 rsyslogd [ 305.064019] [ 911] 0 911 3738 1664 20480 0 0 systemd-logind [ 305.072933] [ 913] 0 913 2821 342 16384 0 0 saslauthd [ 305.081425] [ 914] 0 914 2821 279 16384 0 0 saslauthd [ 305.089915] [ 916] 0 916 2821 247 16384 0 0 saslauthd [ 305.098397] [ 917] 0 917 2821 247 16384 0 0 saslauthd [ 305.106909] [ 918] 0 918 2821 247 16384 0 0 saslauthd [ 305.115348] [ 920] 0 920 19395 2560 47104 0 0 udisksd [ 305.123721] [ 957] 0 957 6717 352 18432 0 -1000 lxcfs [ 305.131860] [ 970] 108 970 1767 384 14336 0 0 avahi-daemon [ 305.140671] [ 1030] 0 1030 65798 8690 180224 0 0 fail2ban-server [ 305.149628] [ 1037] 0 1037 1172 416 10240 0 0 lxc-monitord [ 305.158389] [ 1050] 124 1050 22565 1024 40960 0 0 memcached [ 305.166886] [ 1066] 995 1066 11986 1408 32768 0 0 polkitd [ 305.175142] [ 1114] 0 1114 174702 8288 94208 0 -999 containerd [ 305.183780] [ 1116] 0 1116 1733 640 12288 0 0 upsmon [ 305.192014] [ 1140] 0 1140 2278 1504 14336 0 -1000 sshd [ 305.200056] [ 1147] 113 1147 2410 960 16384 0 0 upsmon [ 305.208300] [ 1255] 0 1255 1053 480 10240 0 0 rpc.idmapd [ 305.216896] [ 1282] 0 1282 35552 31715 151552 0 0 spamd [ 305.224966] [ 1287] 0 1287 1531 288 12288 0 0 rpc.mountd [ 305.233616] [ 1322] 100 1322 3657 389 18432 0 0 dnsmasq [ 305.241923] [ 1362] 132 1362 2994 1568 18432 0 0 mosquitto [ 305.250414] [ 1379] 100 1379 3657 388 18432 0 0 dnsmasq [ 305.258749] [ 1396] 103 1396 964 321 10240 0 0 rpc.statd [ 305.267220] [ 1422] 116 1422 5139 4580 28672 0 0 ntpd [ 305.275223] [ 1434] 0 1434 19324 3328 59392 0 0 nmbd [ 305.283345] [ 1763] 0 1763 2426 704 16384 0 0 cron [ 305.291378] [ 1832] 0 1832 20931 3936 63488 0 0 winbindd [ 305.299784] [ 1834] 0 1834 1269 512 12288 0 0 atd [ 305.307758] [ 1931] 0 1931 47640 6003 73728 0 0 apache2 [ 305.316081] [ 1956] 0 1956 21539 3594 65536 0 0 wb[ODROID] [ 305.324617] [ 1961] 33 1961 710 289 8192 0 0 htcacheclean [ 305.333415] [ 1969] 0 1969 1370 384 12288 0 0 agetty [ 305.341649] [ 1973] 0 1973 3951 1088 22528 0 0 login [ 305.349791] [ 1986] 33 1986 48082 4379 69632 0 0 apache2 [ 305.358116] [ 1988] 33 1988 48083 4379 69632 0 0 apache2 [ 305.366428] [ 1989] 33 1989 48079 4251 69632 0 0 apache2 [ 305.374702] [ 1991] 33 1991 48092 4475 69632 0 0 apache2 [ 305.383071] [ 1992] 33 1992 48082 4379 69632 0 0 apache2 [ 305.391397] [ 2065] 102 2065 8415 3959 43008 0 0 exim4 [ 305.399558] [ 2103] 110 2103 3610 1708 18432 0 0 fetchmail [ 305.408046] [ 2110] 0 2110 10755 8126 51200 0 0 python3 [ 305.416377] [ 2134] 0 2134 415 352 6144 0 -1000 watchdog [ 305.424705] [ 2248] 0 2248 13064 6624 51200 0 0 python3 [ 305.433077] [ 2257] 0 2257 10755 6785 51200 0 0 python3 [ 305.441398] [ 2491] 0 2491 35552 29712 151552 0 0 spamd child [ 305.450056] [ 2492] 0 2492 35552 29712 151552 0 0 spamd child [ 305.458762] [ 2613] 0 2613 3790 2176 20480 0 100 systemd [ 305.467036] [ 2616] 0 2616 4926 721 24576 0 100 (sd-pam) [ 305.475396] [ 2649] 0 2649 2656 1088 16384 0 0 bash [ 305.483514] [ 2767] 33 2767 48077 4443 69632 0 0 apache2 [ 305.491835] [ 2888] 0 2888 20931 1664 59392 0 0 wb-idmap [ 305.500254] [ 2896] 61789 2896 14074 1088 34816 0 0 cockpit-tls [ 305.508883] [ 2906] 63714 2906 11447 1309 28672 0 0 cockpit-ws [ 305.517460] [ 2915] 0 2915 2736 992 18432 0 0 cockpit-session [ 305.526480] [ 2921] 1000 2921 1487 769 10240 0 0 ssh-agent [ 305.534926] [ 2926] 1000 2926 3827 2240 22528 0 100 systemd [ 305.543291] [ 2928] 1000 2928 5028 755 24576 0 100 (sd-pam) [ 305.551705] [ 2948] 1000 2948 1305 576 10240 0 200 mpris-proxy [ 305.560357] [ 2950] 1000 2950 1979 992 14336 0 200 dbus-daemon [ 305.569021] [ 2951] 1000 2951 16036 5811 53248 0 0 cockpit-bridge [ 305.577953] [ 2953] 1000 2953 5086 1472 26624 0 0 sudo [ 305.586020] [ 2967] 0 2967 8712 5425 40960 0 0 cockpit-bridge [ 305.594891] [ 2974] 0 2974 2995 1280 18432 0 0 systemd-timedat [ 305.603990] [ 2986] 0 2986 22409 5029 67584 0 0 packagekitd [ 305.612624] [ 3002] 0 3002 1615 448 12288 0 0 dmesg [ 305.620765] Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled [ 305.628949] CPU: 7 PID: 1322 Comm: dnsmasq Not tainted 6.6.141-current-odroidxu4 #1 [ 305.636569] Hardware name: Samsung Exynos (Flattened Device Tree) [ 305.642638] unwind_backtrace from show_stack+0x10/0x14 [ 305.647836] show_stack from dump_stack_lvl+0x40/0x4c [ 305.652862] dump_stack_lvl from panic+0x118/0x32c [ 305.657628] panic from out_of_memory+0x2d0/0x344 [ 305.662308] out_of_memory from __alloc_pages+0xcd4/0x122c [ 305.667769] __alloc_pages from __get_free_pages+0x14/0x24 [ 305.673221] __get_free_pages from __pollwait+0xa8/0xf8 [ 305.678425] __pollwait from datagram_poll+0x28/0xfc [ 305.683364] datagram_poll from udp_poll+0x10/0x88 [ 305.688134] udp_poll from sock_poll+0x50/0xd0 [ 305.692548] sock_poll from do_sys_poll+0x350/0x5e8 [ 305.697401] do_sys_poll from sys_poll+0x40/0x110 [ 305.702079] sys_poll from ret_fast_syscall+0x0/0x54 [ 305.707017] Exception stack(0xf2795fa8 to 0xf2795ff0) [ 305.712045] 5fa0: 00000000 00000000 00551a40 00000018 ffffffff 00000000 [ 305.720196] 5fc0: 00000000 00000000 00000000 000000a8 00000019 ffffffff 00545430 bea589e8 [ 305.728336] 5fe0: 000000a8 bea58958 b6bdeb49 b6b9f736 [ 305.733371] CPU1: stopping [ 305.733393] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.6.141-current-odroidxu4 #1 [ 305.733407] Hardware name: Samsung Exynos (Flattened Device Tree) [ 305.733420] unwind_backtrace from show_stack+0x10/0x14 [ 305.733455] show_stack from dump_stack_lvl+0x40/0x4c [ 305.733475] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 305.733495] do_handle_IPI from ipi_handler+0x18/0x20 [ 305.733515] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 305.733539] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 305.733566] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 305.733590] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 305.733608] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 305.733634] call_with_stack from __irq_svc+0x98/0xc8 [ 305.733653] Exception stack(0xf08b9f50 to 0xf08b9f98) [ 305.733666] 9f40: 000846d4 00000001 00000001 00000000 [ 305.733678] 9f60: c1963200 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 305.733690] 9f80: c1355390 f08b9fa0 c0c875e4 c0c884fc 60070013 ffffffff [ 305.733698] __irq_svc from default_idle_call+0x50/0x110 [ 305.733718] default_idle_call from do_idle+0x28c/0x2f0 [ 305.733743] do_idle from cpu_startup_entry+0x28/0x2c [ 305.733765] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 305.733787] secondary_start_kernel from 0x40101340 [ 305.733835] CPU0: stopping [ 305.733844] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.6.141-current-odroidxu4 #1 [ 305.733855] Hardware name: Samsung Exynos (Flattened Device Tree) [ 305.733863] unwind_backtrace from show_stack+0x10/0x14 [ 305.733887] show_stack from dump_stack_lvl+0x40/0x4c [ 305.733904] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 305.733923] do_handle_IPI from ipi_handler+0x18/0x20 [ 305.733942] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 305.733962] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 305.733985] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 305.734007] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 305.734025] generic_handle_arch_irq from __irq_svc+0x88/0xc8 [ 305.734042] Exception stack(0xc1201f20 to 0xc1201f68) [ 305.734057] 1f20: 0042009c 00000001 00000000 00000000 c1207900 c13379c4 c1203e54 c1203eb0 [ 305.734070] 1f40: 00000000 00000000 10c5387d 00000000 c1355390 c1201f70 c0c875e4 c0c884fc [ 305.734078] 1f60: 600d0013 ffffffff [ 305.734084] __irq_svc from default_idle_call+0x50/0x110 [ 305.734103] default_idle_call from do_idle+0x28c/0x2f0 [ 305.734124] do_idle from cpu_startup_entry+0x28/0x2c [ 305.734145] cpu_startup_entry from rest_init+0xcc/0xd0 [ 305.734166] rest_init from arch_post_acpi_subsys_init+0x0/0x8 [ 305.734193] CPU3: stopping [ 305.734202] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.6.141-current-odroidxu4 #1 [ 305.734214] Hardware name: Samsung Exynos (Flattened Device Tree) [ 305.734222] unwind_backtrace from show_stack+0x10/0x14 [ 305.734245] show_stack from dump_stack_lvl+0x40/0x4c [ 305.734262] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 305.734281] do_handle_IPI from ipi_handler+0x18/0x20 [ 305.734300] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 305.734320] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 305.734343] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 305.734366] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 305.734384] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 305.734405] call_with_stack from __irq_svc+0x98/0xc8 [ 305.734424] Exception stack(0xf08c1f50 to 0xf08c1f98) [ 305.734435] 1f40: 0007e27c 00000001 00000003 00000000 [ 305.734448] 1f60: c1964600 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 305.734459] 1f80: c1355390 f08c1fa0 c0c875e4 c0c884fc 60030013 ffffffff [ 305.734467] __irq_svc from default_idle_call+0x50/0x110 [ 305.734486] default_idle_call from do_idle+0x28c/0x2f0 [ 305.734506] do_idle from cpu_startup_entry+0x28/0x2c [ 305.734528] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 305.734551] secondary_start_kernel from 0x40101340 [ 305.734574] CPU6: stopping [ 305.734581] CPU: 6 PID: 437 Comm: btrfs-cleaner Not tainted 6.6.141-current-odroidxu4 #1 [ 305.734586] Hardware name: Samsung Exynos (Flattened Device Tree) [ 305.734591] unwind_backtrace from show_stack+0x10/0x14 [ 305.734602] show_stack from dump_stack_lvl+0x40/0x4c [ 305.734609] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 305.734616] do_handle_IPI from ipi_handler+0x18/0x20 [ 305.734623] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 305.734632] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 305.734641] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 305.734650] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 305.734656] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 305.734665] call_with_stack from __irq_svc+0x98/0xc8 [ 305.734672] Exception stack(0xf1341550 to 0xf1341598) [ 305.734677] 1540: c5437c4c 00000000 0000385e 0000385e [ 305.734682] 1560: c5437c38 c5437c4c f13416f0 00000001 f134184c eef94d28 f1341738 c1f17000 [ 305.734685] 1580: 00000000 f13415a0 bf1c71f4 bf1c71f4 600d0013 ffffffff [ 305.734688] __irq_svc from try_release_extent_buffer+0x64/0x310 [btrfs] [ 305.734807] try_release_extent_buffer [btrfs] from btree_release_folio+0x2c/0x48 [btrfs] [ 305.734970] btree_release_folio [btrfs] from shrink_folio_list+0x674/0xac0 [ 305.735056] shrink_folio_list from shrink_lruvec+0x638/0xc04 [ 305.735063] shrink_lruvec from shrink_node+0x374/0x808 [ 305.735069] shrink_node from do_try_to_free_pages+0x150/0x588 [ 305.735076] do_try_to_free_pages from try_to_free_pages+0x130/0x2d4 [ 305.735082] try_to_free_pages from __alloc_pages+0x4d8/0x122c [ 305.735090] __alloc_pages from __filemap_get_folio+0xc8/0x2dc [ 305.735106] __filemap_get_folio from pagecache_get_page+0xc/0x3c [ 305.735117] pagecache_get_page from alloc_extent_buffer+0x220/0x620 [btrfs] [ 305.735204] alloc_extent_buffer [btrfs] from read_tree_block+0x20/0x8c [btrfs] [ 305.735365] read_tree_block [btrfs] from read_block_for_search+0x234/0x334 [btrfs] [ 305.735526] read_block_for_search [btrfs] from btrfs_search_slot+0x2f4/0xc08 [btrfs] [ 305.735685] btrfs_search_slot [btrfs] from find_parent_nodes+0x104/0x2034 [btrfs] [ 305.735845] find_parent_nodes [btrfs] from btrfs_find_all_roots_safe+0x88/0x130 [btrfs] [ 305.736005] btrfs_find_all_roots_safe [btrfs] from btrfs_qgroup_trace_extent_post+0x78/0xb4 [btrfs] [ 305.736166] btrfs_qgroup_trace_extent_post [btrfs] from btrfs_qgroup_trace_leaf_items+0x118/0x14c [btrfs] [ 305.736326] btrfs_qgroup_trace_leaf_items [btrfs] from btrfs_qgroup_trace_subtree+0x1c4/0x314 [btrfs] [ 305.736486] btrfs_qgroup_trace_subtree [btrfs] from do_walk_down+0x648/0x6cc [btrfs] [ 305.736646] do_walk_down [btrfs] from walk_down_tree+0xc8/0x110 [btrfs] [ 305.736805] walk_down_tree [btrfs] from btrfs_drop_snapshot+0x1bc/0xa48 [btrfs] [ 305.736963] btrfs_drop_snapshot [btrfs] from btrfs_clean_one_deleted_snapshot+0x9c/0xe0 [btrfs] [ 305.737122] btrfs_clean_one_deleted_snapshot [btrfs] from cleaner_kthread+0x7c/0x15c [btrfs] [ 305.737281] cleaner_kthread [btrfs] from kthread+0x110/0x12c [ 305.737368] kthread from ret_from_fork+0x14/0x28 [ 305.737374] Exception stack(0xf1341fb0 to 0xf1341ff8) [ 305.737378] 1fa0: 00000000 00000000 00000000 00000000 [ 305.737382] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 305.737386] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 305.737390] CPU2: stopping [ 305.737400] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 6.6.141-current-odroidxu4 #1 [ 305.737412] Hardware name: Samsung Exynos (Flattened Device Tree) [ 305.737420] unwind_backtrace from show_stack+0x10/0x14 [ 305.737444] show_stack from dump_stack_lvl+0x40/0x4c [ 305.737461] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 305.737479] do_handle_IPI from ipi_handler+0x18/0x20 [ 305.737499] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 305.737518] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 305.737541] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 305.737564] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 305.737582] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 305.737603] call_with_stack from __irq_svc+0x98/0xc8 [ 305.737622] Exception stack(0xf08bdf50 to 0xf08bdf98) [ 305.737633] df40: 00083ad4 00000001 00000002 00000000 [ 305.737646] df60: c1963c00 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 305.737657] df80: c1355390 f08bdfa0 c0c875e4 c0c884fc 600d0013 ffffffff [ 305.737665] __irq_svc from default_idle_call+0x50/0x110 [ 305.737684] default_idle_call from do_idle+0x28c/0x2f0 [ 305.737704] do_idle from cpu_startup_entry+0x28/0x2c [ 305.737726] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 305.737749] secondary_start_kernel from 0x40101340 [ 305.737772] CPU4: stopping [ 305.737777] CPU: 4 PID: 0 Comm: swapper/4 Not tainted 6.6.141-current-odroidxu4 #1 [ 305.737782] Hardware name: Samsung Exynos (Flattened Device Tree) [ 305.737786] unwind_backtrace from show_stack+0x10/0x14 [ 305.737795] show_stack from dump_stack_lvl+0x40/0x4c [ 305.737802] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 305.737809] do_handle_IPI from ipi_handler+0x18/0x20 [ 305.737816] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 305.737824] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 305.737832] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 305.737841] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 305.737847] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 305.737855] call_with_stack from __irq_svc+0x98/0xc8 [ 305.737862] Exception stack(0xf08c5f50 to 0xf08c5f98) [ 305.737867] 5f40: 000ee534 00000001 00000004 00000000 [ 305.737872] 5f60: c1965000 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 305.737876] 5f80: c1355390 f08c5fa0 c0c875e4 c0c884fc 60000013 ffffffff [ 305.737878] __irq_svc from default_idle_call+0x50/0x110 [ 305.737886] default_idle_call from do_idle+0x28c/0x2f0 [ 305.737895] do_idle from cpu_startup_entry+0x28/0x2c [ 305.737904] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 305.737912] secondary_start_kernel from 0x40101340 [ 305.737935] CPU5: stopping [ 305.737938] CPU: 5 PID: 0 Comm: swapper/5 Not tainted 6.6.141-current-odroidxu4 #1 [ 305.737944] Hardware name: Samsung Exynos (Flattened Device Tree) [ 305.737947] unwind_backtrace from show_stack+0x10/0x14 [ 305.737957] show_stack from dump_stack_lvl+0x40/0x4c [ 305.737963] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 305.737970] do_handle_IPI from ipi_handler+0x18/0x20 [ 305.737977] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 305.737985] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 305.737993] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 305.738002] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 305.738009] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 305.738016] call_with_stack from __irq_svc+0x98/0xc8 [ 305.738024] Exception stack(0xf08c9f50 to 0xf08c9f98) [ 305.738028] 9f40: 000e6c5c 00000001 00000005 00000000 [ 305.738033] 9f60: c1965a00 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 305.738036] 9f80: c1355390 f08c9fa0 c0c875e4 c0c884fc 600d0013 ffffffff [ 305.738039] __irq_svc from default_idle_call+0x50/0x110 [ 305.738046] default_idle_call from do_idle+0x28c/0x2f0 [ 305.738055] do_idle from cpu_startup_entry+0x28/0x2c [ 305.738064] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 305.738072] secondary_start_kernel from 0x40101340 [ 306.802472] ---[ end Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled ]--- [ 279.613717] f2b/f.ADSL-sysl invoked oom-killer: gfp_mask=0xcc0(GFP_KERNEL), order=2, oom_score_adj=0 [ 279.621431] CPU: 4 PID: 5055 Comm: f2b/f.ADSL-sysl Not tainted 6.6.141-current-odroidxu4 #1 [ 279.629697] Hardware name: Samsung Exynos (Flattened Device Tree) [ 279.635770] unwind_backtrace from show_stack+0x10/0x14 [ 279.640972] show_stack from dump_stack_lvl+0x40/0x4c [ 279.646004] dump_stack_lvl from dump_header+0x4c/0x208 [ 279.651194] dump_header from out_of_memory+0x2a8/0x344 [ 279.656399] out_of_memory from __alloc_pages+0xcd4/0x122c [ 279.661856] __alloc_pages from __get_free_pages+0x14/0x24 [ 279.667314] __get_free_pages from pgd_alloc+0x14/0x138 [ 279.672512] pgd_alloc from mm_init+0xfc/0x274 [ 279.676930] mm_init from alloc_bprm+0x88/0x2b0 [ 279.681439] alloc_bprm from do_execveat_common+0x6c/0x1d8 [ 279.686896] do_execveat_common from sys_execve+0x30/0x38 [ 279.692267] sys_execve from ret_fast_syscall+0x0/0x54 [ 279.697378] Exception stack(0xf3115fa8 to 0xf3115ff0) [ 279.702409] 5fa0: 00000001 00000000 b1200a50 b0f86f00 beb4dd28 004a9640 [ 279.710552] 5fc0: 00000001 00000000 00000000 0000000b 00000000 b0f86f00 b1200a30 b1200af0 [ 279.718697] 5fe0: b6dfc2e9 aabfe464 0020585d b6dfc308 [ 279.723875] Mem-Info: [ 279.725983] active_anon:6164 inactive_anon:88963 isolated_anon:0 [ 279.725983] active_file:24159 inactive_file:59714 isolated_file:0 [ 279.725983] unevictable:4754 dirty:0 writeback:0 [ 279.725983] slab_reclaimable:6042 slab_unreclaimable:176972 [ 279.725983] mapped:29134 shmem:2208 pagetables:1687 [ 279.725983] sec_pagetables:0 bounce:0 [ 279.725983] kernel_misc_reclaimable:0 [ 279.725983] free:127704 free_pcp:185 free_cma:26532 [ 279.765847] Node 0 active_anon:24656kB inactive_anon:355852kB active_file:96336kB inactive_file:238908kB unevictable:19016kB isolated(anon):0kB isolated(file):0kB mapped:116536kB dirty:0kB writeback:0kB shmem:8832kB writeback_tmp:0kB kernel_stack:3200kB pagetables:6748kB sec_pagetables:0kB all_unreclaimable? no [ 279.793274] Normal free:3648kB boost:0kB min:3416kB low:4268kB high:5120kB reserved_highatomic:0KB active_anon:4kB inactive_anon:0kB active_file:1236kB inactive_file:312kB unevictable:0kB writepending:0kB present:786432kB managed:748292kB mlocked:0kB bounce:0kB free_pcp:164kB local_pcp:0kB free_cma:0kB [ 279.819959] lowmem_reserve[]: 0 10064 10064 [ 279.824059] Normal: 350*4kB (UME) 166*8kB (UME) 39*16kB (UME) 1*32kB (U) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB = 3384kB [ 279.837362] 88139 total pagecache pages [ 279.841129] 14 pages in swap cache [ 279.844507] Free swap = 6006988kB [ 279.847927] Total swap = 6011596kB [ 279.851268] 518656 pages RAM [ 279.854125] 322048 pages HighMem/MovableOnly [ 279.858418] 9535 pages reserved [ 279.861492] 32768 pages cma reserved [ 279.865046] Tasks state (memory values in pages): [ 279.869806] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name [ 279.878471] [ 290] 0 290 5804 1410 24576 0 -250 systemd-journal [ 279.887405] [ 344] 0 344 7617 1841 24576 0 -1000 systemd-udevd [ 279.896248] [ 661] 109 661 1262 576 12288 0 0 rpcbind [ 279.904482] [ 683] 0 683 904 288 10240 0 0 blkmapd [ 279.912871] [ 685] 0 685 882 512 8192 0 0 nfsdcld [ 279.921175] [ 745] 121 745 4490 1856 24576 0 0 systemd-network [ 279.930190] [ 819] 108 819 1801 800 14336 0 0 avahi-daemon [ 279.938933] [ 840] 107 840 2130 1054 14336 0 -900 dbus-daemon [ 279.947607] [ 867] 0 867 840 576 10240 0 0 fsidd [ 279.955696] [ 911] 0 911 13970 928 32768 0 0 rsyslogd [ 279.964160] [ 928] 0 928 3738 1696 20480 0 0 systemd-logind [ 279.973082] [ 932] 0 932 2821 359 14336 0 0 saslauthd [ 279.981646] [ 935] 0 935 2821 296 14336 0 0 saslauthd [ 279.990189] [ 936] 0 936 2821 264 14336 0 0 saslauthd [ 279.998636] [ 937] 0 937 2821 264 14336 0 0 saslauthd [ 280.007156] [ 938] 0 938 2821 264 14336 0 0 saslauthd [ 280.015498] [ 939] 0 939 19396 2464 47104 0 0 udisksd [ 280.023942] [ 978] 0 978 6717 352 18432 0 -1000 lxcfs [ 280.032095] [ 993] 108 993 1767 384 14336 0 0 avahi-daemon [ 280.040824] [ 1006] 995 1006 11986 1440 32768 0 0 polkitd [ 280.049235] [ 1037] 0 1037 58269 7051 163840 0 0 fail2ban-server [ 280.058147] [ 1057] 0 1057 1172 416 10240 0 0 lxc-monitord [ 280.066966] [ 1078] 124 1078 22565 960 43008 0 0 memcached [ 280.075291] [ 1142] 0 1142 176942 8185 100352 0 -999 containerd [ 280.084044] [ 1144] 0 1144 1733 608 14336 0 0 upsmon [ 280.092243] [ 1154] 0 1154 2278 1504 14336 0 -1000 sshd [ 280.100190] [ 1175] 113 1175 2410 1088 16384 0 0 upsmon [ 280.108456] [ 1388] 0 1388 1053 480 10240 0 0 rpc.idmapd [ 280.117011] [ 1418] 0 1418 35543 31476 151552 0 0 spamd [ 280.125098] [ 1423] 0 1423 1531 288 12288 0 0 rpc.mountd [ 280.133731] [ 1454] 105 1454 135555 28997 221184 0 -1000 mariadbd [ 280.142135] [ 1477] 100 1477 3657 389 18432 0 0 dnsmasq [ 280.150447] [ 1507] 132 1507 2967 1568 18432 0 0 mosquitto [ 280.158955] [ 1526] 0 1526 19324 3296 57344 0 0 nmbd [ 280.167004] [ 1541] 100 1541 3657 388 18432 0 0 dnsmasq [ 280.175272] [ 1570] 103 1570 964 321 10240 0 0 rpc.statd [ 280.183816] [ 1584] 116 1584 5139 4578 30720 0 0 ntpd [ 280.191876] [ 1671] 0 1671 2426 704 16384 0 0 cron [ 280.199930] [ 1809] 0 1809 1269 512 10240 0 0 atd [ 280.207910] [ 1997] 0 1997 20931 3872 67584 0 0 winbindd [ 280.216326] [ 2040] 0 2040 47654 5944 73728 96 0 apache2 [ 280.224584] [ 2094] 0 2094 1370 416 12288 0 0 agetty [ 280.232993] [ 2100] 0 2100 3951 1024 24576 0 0 login [ 280.241025] [ 2119] 33 2119 710 257 8192 0 0 htcacheclean [ 280.249786] [ 2144] 0 2144 21539 3530 67584 0 0 wb[ODROID] [ 280.258373] [ 2260] 102 2260 8415 3872 38912 0 0 exim4 [ 280.266518] [ 2304] 110 2304 3610 1676 18432 0 0 fetchmail [ 280.274929] [ 2375] 0 2375 10755 7998 51200 0 0 python3 [ 280.283323] [ 2390] 0 2390 415 352 8192 0 -1000 watchdog [ 280.291731] [ 2588] 0 2588 13064 6496 51200 128 0 python3 [ 280.300073] [ 2590] 0 2590 10755 6625 49152 128 0 python3 [ 280.308401] [ 2685] 0 2685 35543 29400 147456 32 0 spamd child [ 280.317029] [ 2687] 0 2687 35543 29464 147456 0 0 spamd child [ 280.325620] [ 2912] 0 2912 3790 2240 20480 0 100 systemd [ 280.333993] [ 2918] 0 2918 4926 743 24576 32 100 (sd-pam) [ 280.342393] [ 2990] 0 2990 2656 1088 16384 0 0 bash [ 280.350448] [ 3088] 0 3088 20931 1631 61440 0 0 wb-idmap [ 280.358852] [ 3102] 61789 3102 14586 1152 40960 0 0 cockpit-tls [ 280.367522] [ 3128] 63714 3128 9111 1142 24576 0 0 cockpit-ws [ 280.376105] [ 3169] 0 3169 2736 992 18432 0 0 cockpit-session [ 280.385062] [ 3175] 1000 3175 1487 769 10240 0 0 ssh-agent [ 280.393651] [ 3180] 1000 3180 3827 2272 20480 0 100 systemd [ 280.401926] [ 3182] 1000 3182 5028 689 28672 64 100 (sd-pam) [ 280.410324] [ 3202] 1000 3202 1305 576 10240 0 200 mpris-proxy [ 280.418987] [ 3204] 1000 3204 1979 960 14336 0 200 dbus-daemon [ 280.427649] [ 3205] 1000 3205 18073 5453 55296 0 0 cockpit-bridge [ 280.436583] [ 3208] 1000 3208 5086 1472 28672 0 0 sudo [ 280.444593] [ 3215] 0 3215 8655 5263 40960 0 0 cockpit-bridge [ 280.453562] [ 3233] 0 3233 2995 1280 18432 0 0 systemd-timedat [ 280.462676] [ 3241] 0 3241 22428 5185 67584 0 0 packagekitd [ 280.471488] [ 3275] 0 3275 17523 2243 71680 0 0 journalctl [ 280.479932] [ 3318] 33 3318 48092 4351 65536 102 0 apache2 [ 280.488222] [ 3319] 33 3319 48092 4351 65536 102 0 apache2 [ 280.496550] [ 3320] 33 3320 48092 4415 65536 102 0 apache2 [ 280.504750] [ 3321] 33 3321 48092 4319 65536 102 0 apache2 [ 280.513206] [ 3322] 33 3322 48092 4383 65536 102 0 apache2 [ 280.521512] [ 3350] 0 3350 1552 320 10240 0 0 sync [ 280.529561] [ 4572] 33 4572 48078 3621 65536 153 0 apache2 [ 280.537858] [ 4580] 0 4580 21222 2851 63488 0 0 samba-dcerpcd [ 280.546792] [ 4600] 0 4600 22362 4288 73728 0 0 rpcd_lsad [ 280.555100] [ 4608] 0 4608 22362 4224 75776 0 0 rpcd_lsad [ 280.563734] [ 5054] 0 5054 17521 2720 73728 0 0 journalctl [ 280.572303] [ 5055] 0 5055 58269 3624 86016 0 0 f2b/f.ADSL-sysl [ 280.581480] Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled [ 280.589396] CPU: 4 PID: 5055 Comm: f2b/f.ADSL-sysl Not tainted 6.6.141-current-odroidxu4 #1 [ 280.597710] Hardware name: Samsung Exynos (Flattened Device Tree) [ 280.603777] unwind_backtrace from show_stack+0x10/0x14 [ 280.608973] show_stack from dump_stack_lvl+0x40/0x4c [ 280.613999] dump_stack_lvl from panic+0x118/0x32c [ 280.618764] panic from out_of_memory+0x2d0/0x344 [ 280.623442] out_of_memory from __alloc_pages+0xcd4/0x122c [ 280.628904] __alloc_pages from __get_free_pages+0x14/0x24 [ 280.634361] __get_free_pages from pgd_alloc+0x14/0x138 [ 280.639560] pgd_alloc from mm_init+0xfc/0x274 [ 280.643981] mm_init from alloc_bprm+0x88/0x2b0 [ 280.648486] alloc_bprm from do_execveat_common+0x6c/0x1d8 [ 280.653946] do_execveat_common from sys_execve+0x30/0x38 [ 280.659317] sys_execve from ret_fast_syscall+0x0/0x54 [ 280.664436] Exception stack(0xf3115fa8 to 0xf3115ff0) [ 280.669457] 5fa0: 00000001 00000000 b1200a50 b0f86f00 beb4dd28 004a9640 [ 280.677604] 5fc0: 00000001 00000000 00000000 0000000b 00000000 b0f86f00 b1200a30 b1200af0 [ 280.685749] 5fe0: b6dfc2e9 aabfe464 0020585d b6dfc308 [ 280.690784] CPU0: stopping [ 280.690804] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.6.141-current-odroidxu4 #1 [ 280.690817] Hardware name: Samsung Exynos (Flattened Device Tree) [ 280.690829] unwind_backtrace from show_stack+0x10/0x14 [ 280.690861] show_stack from dump_stack_lvl+0x40/0x4c [ 280.690880] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 280.690898] do_handle_IPI from ipi_handler+0x18/0x20 [ 280.690915] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 280.690937] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 280.690961] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 280.690982] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 280.690999] generic_handle_arch_irq from __irq_svc+0x88/0xc8 [ 280.691014] Exception stack(0xc1201f20 to 0xc1201f68) [ 280.691028] 1f20: 003d9b44 00000001 00000000 00000000 c1207900 c13379c4 c1203e54 c1203eb0 [ 280.691039] 1f40: 00000000 00000000 10c5387d 00000000 c1355390 c1201f70 c0c875e4 c0c884fc [ 280.691047] 1f60: 600f0013 ffffffff [ 280.691053] __irq_svc from default_idle_call+0x50/0x110 [ 280.691071] default_idle_call from do_idle+0x28c/0x2f0 [ 280.691093] do_idle from cpu_startup_entry+0x28/0x2c [ 280.691113] cpu_startup_entry from rest_init+0xcc/0xd0 [ 280.691131] rest_init from arch_post_acpi_subsys_init+0x0/0x8 [ 280.691178] CPU2: stopping [ 280.691187] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 6.6.141-current-odroidxu4 #1 [ 280.691197] Hardware name: Samsung Exynos (Flattened Device Tree) [ 280.691205] unwind_backtrace from show_stack+0x10/0x14 [ 280.691227] show_stack from dump_stack_lvl+0x40/0x4c [ 280.691242] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 280.691259] do_handle_IPI from ipi_handler+0x18/0x20 [ 280.691276] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 280.691294] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 280.691314] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 280.691334] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 280.691350] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 280.691373] call_with_stack from __irq_svc+0x98/0xc8 [ 280.691390] Exception stack(0xf08bdf50 to 0xf08bdf98) [ 280.691400] df40: 0007df7c 00000001 00000002 00000000 [ 280.691412] df60: c1963c00 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 280.691422] df80: c1355390 f08bdfa0 c0c875e4 c0c884fc 600d0013 ffffffff [ 280.691429] __irq_svc from default_idle_call+0x50/0x110 [ 280.691445] default_idle_call from do_idle+0x28c/0x2f0 [ 280.691464] do_idle from cpu_startup_entry+0x28/0x2c [ 280.691483] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 280.691504] secondary_start_kernel from 0x40101340 [ 280.691525] CPU1: stopping [ 280.691534] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.6.141-current-odroidxu4 #1 [ 280.691544] Hardware name: Samsung Exynos (Flattened Device Tree) [ 280.691552] unwind_backtrace from show_stack+0x10/0x14 [ 280.691573] show_stack from dump_stack_lvl+0x40/0x4c [ 280.691588] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 280.691604] do_handle_IPI from ipi_handler+0x18/0x20 [ 280.691621] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 280.691639] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 280.691659] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 280.691679] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 280.691695] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 280.691714] call_with_stack from __irq_svc+0x98/0xc8 [ 280.691731] Exception stack(0xf08b9f50 to 0xf08b9f98) [ 280.691742] 9f40: 0007daa4 00000001 00000001 00000000 [ 280.691753] 9f60: c1963200 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 280.691763] 9f80: c1355390 f08b9fa0 c0c875e4 c0c884fc 600d0013 ffffffff [ 280.691770] __irq_svc from default_idle_call+0x50/0x110 [ 280.691787] default_idle_call from do_idle+0x28c/0x2f0 [ 280.691805] do_idle from cpu_startup_entry+0x28/0x2c [ 280.691825] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 280.691845] secondary_start_kernel from 0x40101340 [ 280.691866] CPU3: stopping [ 280.691874] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.6.141-current-odroidxu4 #1 [ 280.691884] Hardware name: Samsung Exynos (Flattened Device Tree) [ 280.691892] unwind_backtrace from show_stack+0x10/0x14 [ 280.691913] show_stack from dump_stack_lvl+0x40/0x4c [ 280.691928] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 280.691945] do_handle_IPI from ipi_handler+0x18/0x20 [ 280.691962] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 280.691980] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 280.692000] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 280.692020] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 280.692037] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 280.692055] call_with_stack from __irq_svc+0x98/0xc8 [ 280.692072] Exception stack(0xf08c1f50 to 0xf08c1f98) [ 280.692082] 1f40: 0007401c 00000001 00000003 00000000 [ 280.692094] 1f60: c1964600 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 280.692104] 1f80: c1355390 f08c1fa0 c0c875e4 c0c884fc 600d0013 ffffffff [ 280.692111] __irq_svc from default_idle_call+0x50/0x110 [ 280.692127] default_idle_call from do_idle+0x28c/0x2f0 [ 280.692146] do_idle from cpu_startup_entry+0x28/0x2c [ 280.692165] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 280.692185] secondary_start_kernel from 0x40101340 [ 280.692206] CPU6: stopping [ 280.692212] CPU: 6 PID: 438 Comm: btrfs-cleaner Not tainted 6.6.141-current-odroidxu4 #1 [ 280.692218] Hardware name: Samsung Exynos (Flattened Device Tree) [ 280.692222] unwind_backtrace from show_stack+0x10/0x14 [ 280.692234] show_stack from dump_stack_lvl+0x40/0x4c [ 280.692241] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 280.692249] do_handle_IPI from ipi_handler+0x18/0x20 [ 280.692257] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 280.692267] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 280.692278] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 280.692287] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 280.692294] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 280.692304] call_with_stack from __irq_svc+0x98/0xc8 [ 280.692312] Exception stack(0xf10b1750 to 0xf10b1798) [ 280.692317] 1740: cb9d4028 2dc3d000 00319e67 0000b8b5 [ 280.692322] 1760: c1184030 00000004 f10b181c 00000084 00000001 00000004 f10b194c 00000000 [ 280.692326] 1780: c1283464 f10b17a0 c0300d68 c0c926c8 600d0013 ffffffff [ 280.692329] __irq_svc from _raw_spin_unlock_irq+0x1c/0x44 [ 280.692341] _raw_spin_unlock_irq from shrink_lruvec+0x610/0xc04 [ 280.692353] shrink_lruvec from shrink_node+0x374/0x808 [ 280.692360] shrink_node from do_try_to_free_pages+0x150/0x588 [ 280.692368] do_try_to_free_pages from try_to_free_pages+0x130/0x2d4 [ 280.692375] try_to_free_pages from __alloc_pages+0x4d8/0x122c [ 280.692384] __alloc_pages from __filemap_get_folio+0xc8/0x2dc [ 280.692396] __filemap_get_folio from pagecache_get_page+0xc/0x3c [ 280.692408] pagecache_get_page from alloc_extent_buffer+0x220/0x620 [btrfs] [ 280.692537] alloc_extent_buffer [btrfs] from read_tree_block+0x20/0x8c [btrfs] [ 280.692715] read_tree_block [btrfs] from find_parent_nodes+0x300/0x2034 [btrfs] [ 280.692894] find_parent_nodes [btrfs] from btrfs_find_all_roots_safe+0x88/0x130 [btrfs] [ 280.693071] btrfs_find_all_roots_safe [btrfs] from btrfs_qgroup_trace_extent_post+0x78/0xb4 [btrfs] [ 280.693249] btrfs_qgroup_trace_extent_post [btrfs] from btrfs_qgroup_trace_leaf_items+0x118/0x14c [btrfs] [ 280.693427] btrfs_qgroup_trace_leaf_items [btrfs] from btrfs_qgroup_trace_subtree+0x1c4/0x314 [btrfs] [ 280.693605] btrfs_qgroup_trace_subtree [btrfs] from do_walk_down+0x648/0x6cc [btrfs] [ 280.693783] do_walk_down [btrfs] from walk_down_tree+0xc8/0x110 [btrfs] [ 280.693959] walk_down_tree [btrfs] from btrfs_drop_snapshot+0x1bc/0xa48 [btrfs] [ 280.694135] btrfs_drop_snapshot [btrfs] from btrfs_clean_one_deleted_snapshot+0x9c/0xe0 [btrfs] [ 280.694312] btrfs_clean_one_deleted_snapshot [btrfs] from cleaner_kthread+0x7c/0x15c [btrfs] [ 280.694490] cleaner_kthread [btrfs] from kthread+0x110/0x12c [ 280.694586] kthread from ret_from_fork+0x14/0x28 [ 280.694592] Exception stack(0xf10b1fb0 to 0xf10b1ff8) [ 280.694596] 1fa0: 00000000 00000000 00000000 00000000 [ 280.694601] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 280.694604] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 280.694609] CPU5: stopping [ 280.694614] CPU: 5 PID: 0 Comm: swapper/5 Not tainted 6.6.141-current-odroidxu4 #1 [ 280.694620] Hardware name: Samsung Exynos (Flattened Device Tree) [ 280.694623] unwind_backtrace from show_stack+0x10/0x14 [ 280.694634] show_stack from dump_stack_lvl+0x40/0x4c [ 280.694641] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 280.694649] do_handle_IPI from ipi_handler+0x18/0x20 [ 280.694657] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 280.694665] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 280.694675] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 280.694684] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 280.694691] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 280.694700] call_with_stack from __irq_svc+0x98/0xc8 [ 280.694708] Exception stack(0xf08c9f50 to 0xf08c9f98) [ 280.694713] 9f40: 000c49ac 00000001 00000005 00000000 [ 280.694718] 9f60: c1965a00 c13379c4 c1203e54 c1203eb0 00000000 00000000 00000000 00000000 [ 280.694722] 9f80: c1355390 f08c9fa0 c0c875e4 c0c884fc 60030013 ffffffff [ 280.694725] __irq_svc from default_idle_call+0x50/0x110 [ 280.694733] default_idle_call from do_idle+0x28c/0x2f0 [ 280.694744] do_idle from cpu_startup_entry+0x28/0x2c [ 280.694753] cpu_startup_entry from secondary_start_kernel+0x190/0x220 [ 280.694763] secondary_start_kernel from 0x40101340 [ 280.694786] CPU7: stopping [ 280.694791] CPU: 7 PID: 90 Comm: kswapd0 Not tainted 6.6.141-current-odroidxu4 #1 [ 280.694797] Hardware name: Samsung Exynos (Flattened Device Tree) [ 280.694800] unwind_backtrace from show_stack+0x10/0x14 [ 280.694811] show_stack from dump_stack_lvl+0x40/0x4c [ 280.694818] dump_stack_lvl from do_handle_IPI+0x1e8/0x210 [ 280.694826] do_handle_IPI from ipi_handler+0x18/0x20 [ 280.694834] ipi_handler from handle_percpu_devid_irq+0x8c/0x238 [ 280.694842] handle_percpu_devid_irq from handle_irq_desc+0x50/0x80 [ 280.694851] handle_irq_desc from gic_handle_irq+0x88/0xa8 [ 280.694861] gic_handle_irq from generic_handle_arch_irq+0x34/0x44 [ 280.694868] generic_handle_arch_irq from call_with_stack+0x18/0x20 [ 280.694877] call_with_stack from __irq_svc+0x98/0xc8 [ 280.694885] Exception stack(0xf0cb1d30 to 0xf0cb1d78) [ 280.694890] 1d20: cd500828 2dc4f000 2dc4f000 00007907 [ 280.694895] 1d40: c11840bc 00000020 f0cb1dfc 00000000 00000001 00000020 f0cb1f10 00000000 [ 280.694900] 1d60: c1283464 f0cb1d80 c0300e20 c0c926c8 600d0013 ffffffff [ 280.694903] __irq_svc from _raw_spin_unlock_irq+0x1c/0x44 [ 280.694914] _raw_spin_unlock_irq from shrink_lruvec+0x6c8/0xc04 [ 280.694924] shrink_lruvec from shrink_node+0x374/0x808 [ 280.694931] shrink_node from kswapd+0x4c8/0xae0 [ 280.694938] kswapd from kthread+0x110/0x12c [ 280.694945] kthread from ret_from_fork+0x14/0x28 [ 280.694951] Exception stack(0xf0cb1fb0 to 0xf0cb1ff8) [ 280.694956] 1fa0: 00000000 00000000 00000000 00000000 [ 280.694960] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 280.694964] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 281.758837] ---[ end Kernel panic - not syncing: Out of memory: system-wide panic_on_oom is enabled ]---
September 9Sep 9 Author Yes, on a usb stick attached to give more storage. Is there a specific problem with btrfs that causes this?
September 9Sep 9 On 9/8/2026 at 9:43 AM, Iain McFarlane said: Hello, after upgrading my xu4 to trixie I have begun seeing the oom killer wrecking havoc! I had 1Gb of swap and have upgraded it to have 6Gb of swap but it does not seem to help. I have set vm.panic_on_oom = 1 as without this it just keeps killing tasks and then says deadlocked on memory There is likely a memory leak somewhere. Quick search does not reveal how much real RAM this XU4 has so please can you mention that. I only know it is 32-bit Samsung Exynos SoC. But 6GB of swap makes no sense I assume unless you have a very dedicated use-case with a small working-set but mayb lager virtual memory needed over a long period of time, much longer than lowest speed random 4kB I/O spoeed for your storage. Still no proper option if there is a memory leak. Main question is, how is your swap space organized? I hope you know Btrfs needs dedicated command to set up swapspace on the filesystem. But maybe you used separate swap partition, that is at least what I do on all computers, also on ancient RaspberryPi1 with 512M RAM and Btrfs rootfs with option compress-force=zstd for many years. Same for NanoPi-NEO's, including a variant with just 256M RAM. I disable Armbian zram swap mostly, so please also mention how and if you use that. The kernels on mentioned systems are 6.18 based and even 7.1.8-edge-sunxi runs without issues. But 6.6 and Exynos I am not so sure how robust/stable that is under load. Also what profile are you using? Please show output of: sudo btrfs device usage /
September 9Sep 9 Just a guess. If you can try to remove the additional storage via btrfs and see if the issue persists.
September 11Sep 11 Author Thanks for the pointers guys. I took the external ssd with btrfs on it and checked it on another computer. It showed absolutely no problems but while I had it off I disabled quota support. Started the odroid xu4 without it plugged in (after some messing about with nofail in the fstab) and it seemed stable so I plugged the SSD back in and nothing happened. everything is back up and running and looks to be good. I suspect the quotas were probably causing a lot of memory usage. Thanks for allowing me to keep my sanity for another few weeks!
September 11Sep 11 1 hour ago, Iain McFarlane said: I took the external ssd with btrfs on it and checked it on another computer. It showed absolutely no problems but while I had it off I disabled quota support. OK, I assumed you were using multi-device for rootfs, but the Btrfs filesystem is just something external I see. Qgroups/quota/squata should be disabled, unless you really need it.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.