Hi All, I did upgrade of my Cubieboard 3 from 5.31 to Armbian 5.36 and everything was fine. But than by short system audit I find an old package "rpimonitor" that was used long time before and never being used now. I did "sudo apt purge rpimonitor" and "sudo apt autoremove" to remove dependencies and it starts...

Somehow my CPU load was vanished, on Idle mode it shows at least 2.0+ and at least 4+ on load. Looks like it is twice higher than it is. If I run top command, nothing shown. Only mysql time is 117+ Hours. But, Idle is 98.7...

:~$ top

top - 19:29:50 up 22:07,  3 users,  load average: 2.28, 2.35, 2.93
Tasks: 112 total,   1 running, 111 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.3 us,  0.5 sy,  0.0 ni, 98.7 id,  0.5 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :  2046812 total,    64960 free,   484692 used,  1497160 buff/cache
KiB Swap:   131068 total,   131068 free,        0 used.  1483864 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                            
10787 user      20   0    6996   1536   1016 R   1.0  0.1   0:00.19 top                                                                                                                
  713 mysql     20   0  627040 329036   6020 S   0.7 16.1 117:16.47 mysqld                                                                                                             
  777 root      20   0   51836   7972   2032 S   0.3  0.4   2:20.05 fail2ban-server                                                                                                    
  908 snmp      20   0   12044   2424    712 S   0.3  0.1   2:24.67 snmpd                                                                                                              
    1 root      20   0   22824   2944   1700 S   0.0  0.1   0:22.26 systemd                                                                                                            
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.04 kthreadd                                                                                                           
    3 root      20   0       0      0      0 S   0.0  0.0   2:56.23 ksoftirqd/0            

Also Cacti graphs looks bad now:
 

Spoiler

 

 

 

 

 

But temperature did not goes up as I should if CPU is 100% in use:

 

Spoiler

 

 

Funny, but I do not "feel" the cpu load, server is quite responsible, but seems only showing wrong data.

 

Welcome screen from Armbian is "hanging" with old information, "updates total" was last time checked before update to 5.36 and do not want to be updated.

Welcome to ARMBIAN 5.36 user-built Ubuntu 16.04.3 LTS 3.4.113-sun7i   
System load:   2.16 2.33 2.94  	Up time:       22:07 hours		Local users:   2            	
Memory usage:  24 % of 1998MB 	IP:            xxx.xxx.xxx.xxx
CPU temp:      36°C           	HDD temp:      37°C           	
Usage of /:    7% of 28G    	storage/:      75% of 459G   	Battery:       100% full      	

[ 0 security updates available, 8 updates total: apt upgrade ]
Last check: 2017-11-26 19:11

Last login: Wed Jan  3 14:09:19 2018

From the /var/log/apt/history.log:

Start-Date: 2017-11-26  20:38:00
Commandline: apt upgrade
Requested-By: user (1000)
Upgrade: linux-image-sun7i:armhf (5.31, 5.35), linux-headers-sun7i:armhf (5.31, 5.35), armbian-tools-xenial:armhf (5.30, 5.35), linux-xenial-root-cubietruck:armhf (5.31, 5.35), armbian-firmware:armhf (5.30, 5.35), sunxi-tools:armhf (1.4.2-1~armbian5.34.171017+1, 1.4.2-1~armbian5.35+1), linux-u-boot-cubietruck-default:armhf (5.31, 5.35), hostapd:armhf (2:2.6-4~armbian5.34.171017+1, 2:2.6-4~armbian5.35+1)
End-Date: 2017-11-26  20:43:39

Funny, I did not found any 5.36 package, only 5.35 and Armbian welcome screen says it is 5.36.

 

Long story short is:

Problem 1. After purge of rpimonitor I have fake (?) 100% CPU load.

Problem 2. After upgrade to newer Armbian welcome screen has some problems.

 

Any ideas how to solve it?

On 4.1.2018 at 10:43 AM, gas_85 said:

Welcome screen from Armbian is "hanging" with old information, "updates total" was last time checked before update to 5.36 and do not want to be updated.

I also had this problem after updating to 5.36. I then noticed a permission error in  /var/log/syslog. Once I corrected the permissions of the appropriate path/file the welcome-screen did not „hang“ anymore with old information.

Unfortunately I can‘t remember which path or file I had to change the permissions, but it was clear when I saw the error in syslog.

Hope that helps at least for that „hanging“-with-old-information issue.

On 4. 1. 2018 at 10:43 AM, gas_85 said:

Funny, I did not found any 5.36 package, only 5.35 and Armbian welcome screen says it is 5.36.

This is normal. We only pushed an update to packages which had some troubles - board support and left others as is.

 

On 4. 1. 2018 at 10:43 AM, gas_85 said:

Problem 1. After purge of rpimonitor I have fake (?) 100% CPU load.

Problem 2. After upgrade to newer Armbian welcome screen has some problems.

I started with a clean image made from sources and could not reproduce problems you have.  Here is my armbianmonitor -u: http://ix.io/DLr

 

4 hours ago, Bernie_O said:

I can‘t remember which path or file I had to change the permissions, but it was clear when I saw the error in syslog.

It was wrong permission for this file: /etc/cron.d/armbian-updates

Thank you both! I change permission for the group to read only (-rw-r--r-- 1 root root 83 Dec 2 11:22 /etc/cron.d/armbian-updates). No error in syslog any more, but it does not seems to unfreeze my welcome screen. I even run manually /usr/lib/armbian/apt-updates. Will wait till tomorrow and restart server to see if it could help.

 

With CPU I will  try do fresh installation of the server on a few next weeks, this one is running already over years and has a lot of things that were used before, but not needed any more, it is complicate to do clean up. It is also funny, because temperature is quite low (as was usual in idle more), server is responsible and even Webmin shows as 10% on a 3.22 load...

1 hour ago, gas_85 said:

No error in syslog any more, but it does not seems to unfreeze my welcome screen.

Now that you wrote that, I also remember, that it took quite a while for  the „freeze“ to disappear. So all you have to do is being patient ;-)

On 08.01.2018 at 11:25 AM, Bernie_O said:

Now that you wrote that, I also remember, that it took quite a while for  the „freeze“ to disappear. So all you have to do is being patient ;-)

You right, it took at least one day...

I find out what was wrong. It was my mistake with mysql and iptables.

Basically I setup wrong settings for iptables and localhost (like from 127.0.0.1 to 127.0.0.1), but this will cause situation when MySQL port 3306 will not accept some of inbound connections from e.g. 10.0.x.x that is also "local". This will cause issue when CPU load is extremely high, but you could not recognize app/service that cause it, only "TIME+" can give a hint on root mysql process.

It is hardly to catch, because issue will not accrue till reboot, and you do not need to reboot the system for iptables. This cause it last time: iptables set long time ago and then reboot was needed that shows the problems.

Today I did the same mistake on a fresh installed system. I set iptables (orange pointer) and reboot the system (green pointer) later and it starts...

SOLUTION: To solve the problem it is needed something like this in iptables:

-A INPUT -s 127.0.0.1/32 -j ACCEPT