Page: 1, 2, 3  Next

5.4.5 introduces high CPU usage
NoXPhasma
Status: Interested
Joined: 21 Nov 2019
Posts: 14
Reply Quote
I don't know why this happens, but with 5.4.0-5.1-liquorix-amd64 a lot of background programs have a much higher CPU usage and the CPU never idles. When I reboot with 5.4.0-1.1-liquorix this load is gone. I would guess this has to do with the HZ change from 1000 to 250.

Programs like conky, which usually don't consume much CPU, need suddenly severe load of CPU. I can see sporadically daemons which should idle, jumping up to the top CPU consumers.
Back to top
damentz
Status: Assistant
Joined: 09 Sep 2008
Posts: 1143
Reply Quote
Can you show the output of inxi -b?
Back to top
NoXPhasma
Status: Interested
Joined: 21 Nov 2019
Posts: 14
Reply Quote
Sure:

:: Code ::
System:    Host: NoXP-PC Kernel: 5.4.0-1.1-liquorix-amd64 x86_64 bits: 64 Desktop: Cinnamon 4.4.5
           Distro: Linux Mint 19.3 Tricia
Machine:   Type: Desktop Mobo: ASUSTeK model: PRIME X370-PRO v: Rev X.0x serial: <root required>
           UEFI [Legacy]: American Megatrends v: 1201 date: 10/18/2017
CPU:       8-Core: AMD Ryzen 7 1700 type: MT MCP speed: 3052 MHz
Graphics:  Device-1: NVIDIA GP104 [GeForce GTX 1080] driver: nvidia v: 440.44
           Display: x11 server: X.Org 1.20.4 driver: nvidia resolution: 1920x1080~60Hz, 1920x1080~60Hz
           OpenGL: renderer: GeForce GTX 1080/PCIe/SSE2 v: 4.6.0 NVIDIA 440.44
Network:   Device-1: Intel I211 Gigabit Network driver: igb
Drives:    Local Storage: total: 2.71 TiB used: 1.87 TiB (69.0%)
Info:      Processes: 483 Uptime: 12h 39m Memory: 31.36 GiB used: 8.03 GiB (25.6%) Shell: zsh inxi: 3.0.36

Back to top
damentz
Status: Assistant
Joined: 09 Sep 2008
Posts: 1143
Reply Quote
Can you measure the temperature of your CPU using sensors on 5.4.0-1.1 and 5.4.0-5.2 with inxi -s? I'm curious if time accounting issues also manifest as real CPU usage increases.

Should look similar to this:
:: Code ::
$ inxi -s
Sensors:   System Temperatures: cpu: 40.0 C mobo: N/A gpu: nvidia temp: 47 C
           Fan Speeds (RPM): N/A gpu: nvidia fan: 0%


Also, the output of inxi -tc will be useful during these CPU load spikes. For example, I'm just streaming a youtube video and my top CPU consumers look like so:

:: Code ::
$ inxi -tc
Processes: CPU top: 5
           1: cpu: 4.6% command: xorg pid: 1151
           2: cpu: 4.3% command: firefox pid: 3733
           3: cpu: 3.5% command: firefox pid: 3877
           4: cpu: 2.7% command: kwin_x11 pid: 1724
           5: cpu: 1.9% command: firefox pid: 2000037

Back to top
NoXPhasma
Status: Interested
Joined: 21 Nov 2019
Posts: 14
Reply Quote
This is in idle with 5.4.0-5.1-liquorix-amd64.
:: Code ::
$ inxi -tc
Processes: CPU top: 5
           1: cpu: 27.5% command: cinnamon pid: 2861
           2: cpu: 20.6% command: firefox-bin pid: 5939
           3: cpu: 14.4% command: conky pid: 3104
           4: cpu: 13.0% command: wire-desktop pid: 3317
           5: cpu: 11.2% command: xorg pid: 1508

CPU temperature looks normal, maybe slightly higher CPU temperature (water cooling)
:: Code ::
$ inxi -s
Sensors:   System Temperatures: cpu: 31.1 C mobo: 29.0 C gpu: nvidia temp: 41 C
           Fan Speeds (RPM): N/A gpu: nvidia fan: 23%


This is in idle with 5.4.0-1.1-liquorix-amd64:

:: Code ::

Processes: CPU top: 5
           1: cpu: 12.7% command: firefox pid: 6157
           2: cpu: 11.9% command: cinnamon pid: 2664
           3: cpu: 10.5% command: xorg pid: 1506
           4: cpu: 7.3% command: firefox-bin pid: 6263
           5: cpu: 6.4% command: wire-desktop pid: 3236


:: Code ::
Sensors:   System Temperatures: cpu: 29.6 C mobo: 28.0 C gpu: nvidia temp: 41 C
           Fan Speeds (RPM): N/A gpu: nvidia fan: 23%

Back to top
techAdmin
Status: Site Admin
Joined: 26 Sep 2003
Posts: 4128
Location: East Coast, West Coast? I know it's one of them.
Reply Quote
A desktop should not be consuming 12-28% of your cpu. There may be a process in cinnamon that is spinning out control. Firefox using many resources is normal if you aren't using script blocking since javascripts will be running frantically out of control in the background. conky also should not be using that much cpu, unless it was a coincidence and you got the process cpu data right when conky was updating itself.
Back to top
NoXPhasma
Status: Interested
Joined: 21 Nov 2019
Posts: 14
Reply Quote
The issue is only happening with 5.4.0-5.1-liquorix-amd64. Basically any process which usually doesn't consume much CPU does with the latest kernel. I don't see conky ever in the CPU usage tasks, but with this kernel it's on top. Even mysql server jumps up periodically, even it's not in usage at all.

This is the first time I see this behaviour and the inxi output does not even show the actual issue as it reports completely wrong values. While it says cinnamon would consume 10% CPU on 5.4.0-1.1-liquorix-amd64, it actually only consumes 2% in other monitors. I will show you more exact data tomorrow, with screenshots what actually is going on on my end, then you will see that something is really wrong.

It also has nothing to do with cpu power saving, as this is disabled in the motherboard and the CPU never drops below 3GHz. It's really hard to explain, but even after boot and no window is open, the CPU usage is crazy. It constantly jumps from almost no usage to severe usage.

I already looked in the kernel log and there is nothing suspicious and the only change from the Kernel config which differs, beside new modules, is the Hz change.
Back to top
damentz
Status: Assistant
Joined: 09 Sep 2008
Posts: 1143
Reply Quote
Hmm, I think what we're seeing here is increasing tick rate from 250hz to 1000hz affects applications that spin on every tick. What's more unusual is that this shouldn't increase the CPU usage of every application.

Could you run # perf stat -a sleep 5

An example of the output:
:: Code ::
$ sudo perf stat -a sleep 5

 Performance counter stats for 'system wide':

         80,014.60 msec cpu-clock                 #   15.999 CPUs utilized         
            36,004      context-switches          #    0.450 K/sec                 
             3,904      cpu-migrations            #    0.049 K/sec                 
             3,827      page-faults               #    0.048 K/sec                 
     2,852,149,067      cycles                    #    0.036 GHz                   
     1,440,339,928      instructions              #    0.51  insn per cycle         
       299,239,565      branches                  #    3.740 M/sec                 
        11,771,733      branch-misses             #    3.93% of all branches       

       5.001138358 seconds time elapsed


Another thing you can try is run # perf record -g -a sleep 5, then # perf report --percent-limit 2 and paste the output here. This will show us what symbols are spinning the fastest on your system.

And as an example of mine, within 2% of all symbols sampled, all of them are either kernel upkeep (swapper/kernel.vmlinux), or QSGRenderThread (QT rendering thread for kwin/plasma-desktop):

:: Code ::
Samples: 20K of event 'cycles', Event count (approx.): 5239122952
  Children      Self  Command          Shared Object                      Symbol
+   45.53%     0.23%  swapper          [kernel.vmlinux]                   [k] do_idle
+   45.52%     0.00%  swapper          [kernel.vmlinux]                   [k] 0xffffffff810000d4
+   45.52%     0.01%  swapper          [kernel.vmlinux]                   [k] cpu_startup_entry
+   40.95%     0.00%  swapper          [kernel.vmlinux]                   [k] cpuidle_enter
+   40.94%     0.67%  swapper          [kernel.vmlinux]                   [k] cpuidle_enter_state
+   37.64%    37.46%  swapper          [kernel.vmlinux]                   [k] intel_idle
+    5.27%     0.35%  QSGRenderThread  [kernel.vmlinux]                   [k] entry_SYSCALL_64
+    5.24%     0.00%  QSGRenderThread  [unknown]                          [k] 0x243d8020ec834853
+    5.24%     0.00%  QSGRenderThread  libnvidia-glcore.so.440.44         [.] 0x00007fa441c44af0
+    4.94%     0.01%  QSGRenderThread  libc-2.30.so                       [.] __sched_yield
+    4.91%     0.19%  QSGRenderThread  [kernel.vmlinux]                   [k] do_syscall_64
+    4.34%     0.03%  QSGRenderThread  [kernel.vmlinux]                   [k] __ia32_sys_sched_yield
+    4.02%     0.10%  QSGRenderThread  [kernel.vmlinux]                   [k] schedule
+    3.92%     0.76%  QSGRenderThread  [kernel.vmlinux]                   [k] __schedule
+    2.22%     0.01%  swapper          [kernel.vmlinux]                   [k] apic_timer_interrupt




EDIT:
:: Quote ::
It also has nothing to do with cpu power saving, as this is disabled in the motherboard and the CPU never drops below 3GHz. It's really hard to explain, but even after boot and no window is open, the CPU usage is crazy. It constantly jumps from almost no usage to severe usage.

That actually sounds like more time accounting bugs in Linux. The time accounting just over a year ago was MUCH worse than what you're reporting, and it affected both cpufreq/intel-pstate, causing both frequency governors to blast the CPU frequency to max. From my current testing, when running powersave on intel-pstate or ondemand on acpi-cpufreq, CPU frequency is set properly.

Also, remember, Liquorix uses the performance governor (both acpi-cpufreq+intel-pstate), by default. If you desire lower power consumption at the trade off of lower throughput (sometimes by a lot), you can install TLP and set the governor to ondemand/powersave for your respective cpufreq driver. For acpi-cpufreq, you want ondemand; for intel-pstate, you want powersave.

At this point, most people get a benefit from 1000hz + tickless idle, but it seems there's a few bespoke machines out there that will either report 0 or 100% cpu usage when nearly completely idle. Hopefully the perf output gives us a clue of what might be the issue.
Back to top
NoXPhasma
Status: Interested
Joined: 21 Nov 2019
Posts: 14
Reply Quote
There is no liquorix linux tools package and by that I don't have a working perf tool available.

*edit*
Building the tools in the headers source folder also fails:

:: Code ::
$ sudo make -C tools/ perf_install

make: Verzeichnis „/usr/src/linux-headers-5.4.0-5.1-liquorix-amd64/tools“ wird betreten
  DESCEND  perf
make[1]: Verzeichnis „/usr/src/linux-headers-5.4.0-5.1-liquorix-amd64/tools/perf“ wird betreten
  BUILD:   Doing 'make -j16' parallel build
Makefile.perf:8: ../scripts/utilities.mak: Datei oder Verzeichnis nicht gefunden
make[2]: *** Keine Regel, um „../scripts/utilities.mak“ zu erstellen.  Schluss.
Makefile:109: recipe for target 'install' failed
make[1]: *** [install] Error 2
make[1]: Verzeichnis „/usr/src/linux-headers-5.4.0-5.1-liquorix-amd64/tools/perf“ wird verlassen
Makefile:111: recipe for target 'perf_install' failed
make: *** [perf_install] Error 2
make: Verzeichnis „/usr/src/linux-headers-5.4.0-5.1-liquorix-amd64/tools“ wird verlassen

Back to top
damentz
Status: Assistant
Joined: 09 Sep 2008
Posts: 1143
Reply Quote
Can you not use your distribution's perf tool? There's nothing particularly special about the perf tool that comes with a kernel. All that matters is that it's relatively modern.

For example, I'm on Arch and I use the generic perf package named community/perf.
Back to top
Display posts from previous:   
Page: 1, 2, 3  Next
All times are GMT - 8 Hours