+- Moode Forum (https://moodeaudio.org/forum)
+-- Forum: moOde audio player (https://moodeaudio.org/forum/forumdisplay.php?fid=3)
+--- Forum: Support (https://moodeaudio.org/forum/forumdisplay.php?fid=7)
+--- Thread: [SOLVED] Memory usage / mpd issues as of Moode 6.6.0? (/showthread.php?tid=3656)
Memory usage / mpd issues as of Moode 6.6.0? - mafeu - 03-28-2021
Hi,
first of all a big thank you to all involved making Moode Audio Player this fine piece of open source software!
I'm running Moode for a (long) while now on a Raspberry Pi 1B using the headphone jack audio output, a USB 802.11n WLAN Adapter and a SanDisk Extreme PRO Class 10 SD Card. Being happy with the setup and Moode I didn't update for a long time, I think Version 5.3.0 was still running until I updated to 7.1.0 a few days ago.
After the update I had severe problems with the Pi going to 100% system load due to a lot of IO waits and kswapd0 using a lot of CPU time. No surprise, the system is then not usable on the command line via ssh and also the web frontend is not reachable and the browser shows the "Reconnect" button. This started to happen already during setup whenever setting and saving new settings.
Fortunately, the new settings did get saved and after a few hard reboots I had the system set up and running. However, after a couple of minutes playing i.e. radio streams the system still hangs with the above issues.
Looking into the issue I found out that I can still run Moode 6.5.2 without any of the above issues, they start to show up with Moode 6.6.0.
Here is a typical top output for the issue:
Code:
top - 16:08:55 up 48 min, 1 user, load average: 25.68, 26.79, 23.71
Tasks: 111 total, 2 running, 109 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 90.1 sy, 0.0 ni, 0.0 id, 0.1 wa, 0.0 hi, 9.7 si, 0.0 st
MiB Mem : 178.9 total, 73.6 free, 81.5 used, 23.8 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 56.8 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
33 root 20 0 0 0 0 R 79.6 0.0 27:20.10 kswapd0
1891 root 0 -20 0 0 0 D 7.3 0.0 2:38.05 kworker/0:0H+kblockd
7 root 20 0 0 0 0 S 2.5 0.0 1:01.12 ksoftirqd/0
2543 root 0 -20 0 0 0 I 1.4 0.0 0:32.57 kworker/0:2H-kblockdIn one test session I let the system sit blocked. It unblocked after several 10 minutes and I found the following in the kernel ring buffer:
Code:
[Sun Mar 28 18:25:52 2021] Out of memory: Killed process 1094 (mpd) total-vm:169696kB, anon-rss:8680kB, file-rss:216kB, shmem-rss:0kB, UID:1001 pgtables:106kB oom_score_adj:0Unfortunately I did not check the logs of mpd in this session.
Does anybody have a clue what is wrong? Did I miss some important setup?
Cheers,
Martin
Here are some more details of my system:
Code:
pi@moode:~ $ moodeutl -l
20210328 170959 worker: -- Start
20210328 171000 worker: Successfully daemonized
20210328 171002 engine-mpd: Connection to MPD failed
20210328 171004 worker: Integrity check (passed)
20210328 171004 worker: Session loaded
20210328 171004 worker: Debug logging (off)
20210328 171004 worker: Device raw: (0:Headphones|1:empty|i2s:none)
20210328 171004 worker: Device mpd: (0:On-board audio device)
20210328 171004 worker: Device ses: (0|On-board audio device||Headphone|0%)
20210328 173544 worker: ALSA Headphone volume (0%)
20210328 173544 worker: -- System
20210328 173555 worker: Host (moode)
20210328 173556 worker: moOde (6.6.0 2020-07-09)
20210328 173556 worker: RaspiOS (10.4)
20210328 173556 worker: Kernel (5.4.49+ #1323)
20210328 173556 worker: Platform (Pi-1B 256MB)
20210328 173556 worker: ARM arch (armv6l, 32-bit kernel)
20210328 173556 worker: MPD ver (0.21.24)
20210328 173556 worker: CPU gov (ondemand)
20210328 173556 worker: USB boot not available
20210328 173556 worker: File system not expanded yet
20210328 173557 worker: HDMI port on
20210328 173600 worker: File check (OK)
20210328 173600 worker: -- Network
20210328 173600 worker: eth0 exists
20210328 173600 worker: eth0 wait 0 for IP address
20210328 173605 worker: eth0 wait 1 for IP address
20210328 173608 worker: eth0 wait 2 for IP address
20210328 173612 worker: eth0 wait 3 for IP address
20210328 173615 worker: eth0 wait 4 for IP address
20210328 173618 worker: eth0 address not assigned
20210328 173618 worker: wlan0 exists
20210328 173618 worker: wifi country (DE)
20210328 173618 worker: wlan0 trying SSID (esnet)
20210328 173619 worker: IP addr (192.168.178.83)
20210328 173620 worker: Netmask (255.255.255.0)
20210328 173620 worker: Gateway (192.168.178.1)
20210328 173621 worker: Pri DNS (192.168.178.1)
20210328 173621 worker: Domain (fritz.box)
20210328 173622 worker: -- Audio
20210328 173624 worker: MPD conf updated
20210328 173625 worker: ALSA outputs unmuted
20210328 173625 worker: ALSA card number (0)
20210328 173625 worker: Audio output (On-board audio device)
20210328 173628 worker: Audio formats (U8, S16_LE)
20210328 173628 worker: ALSA mixer name (Headphone)
20210328 173628 worker: MPD volume control (software)
20210328 173629 worker: Hdwr volume controller exists
20210328 173629 worker: Max ALSA volume (100%)
20210328 173629 worker: Reset renderer active flags
20210328 173629 worker: -- MPD
20210328 173630 worker: MPD started
20210328 173633 worker: MPD connection refused
20210328 173633 worker: Configure MPD outputs
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker: MPD crossfade (off)
20210328 173637 worker: -- Feature availability
20210328 173637 worker: Source select (available)
20210328 173637 worker: Source select (source: MPD)
20210328 173637 worker: Source select (output: On-board audio device)
20210328 173638 worker: Bluetooth (available)
20210328 173638 worker: Airplay renderer (available)
20210328 173638 worker: Spotify renderer (available)
20210328 173638 worker: Spotify renderer (started)
20210328 173638 worker: Squeezelite renderer (available)
20210328 173638 worker: UPnP renderer (available)
20210328 173638 worker: DLNA server (available)
20210328 173638 worker: UPnP browser (available)
20210328 173638 worker: Audio scrobbler (available)
20210328 173638 worker: GPIO button handler (available)
20210328 173638 worker: -- Music sources
20210328 173639 worker: USB sources (none attached)
20210328 173639 worker: NAS and UPnP sources (none configured)
20210328 173639 worker: -- Miscellaneous
20210328 173639 worker: USB volume knob (Off)
20210328 173639 worker: USB auto-mounter (udisks-glue)
20210328 173640 worker: LED0 (On)
20210328 173640 worker: LED1 (sysclass does not exist)
20210328 173641 worker: Saved MPD vol level (0)
20210328 173641 worker: Preamp volume level (0)
20210328 173642 worker: MPD volume level (0) restored
20210328 173643 worker: ALSA Headphone volume (100%)
20210328 173643 worker: Auto-play (Off)
20210328 173643 worker: Maintenance interval (3 hrs)
20210328 173643 worker: Screen saver activation (Never)
20210328 173646 worker: Session permissions (OK)
20210328 173646 worker: Watchdog started
20210328 173646 worker: ReadyCode:
pi@moode:~ $ moodeutl -s
Generating info...
S Y S T E M P A R A M E T E R S
moOde release = 6.6.0 2020-07-09
RaspiOS = 10.4
Linux kernel = 5.4.49+ #1323
Platform = Pi-1B 256MB
Architecture = armv6l (32-bit)
System uptime = up 3 minutes
Timezone = Europe/Berlin
Current time = 2021-03-28 17:37:57
Host name = moode
Ethernet address = unassigned
Ethernet MAC = b8:27:eb:78:7d:22
WLAN address = 192.168.178.83
WLAN MAC = 74:da:38:df:22:d1
WLAN country = DE
SoC identifier = bcm2835
Core count = 1
Kernel timer freq = 100 Hz
SDCard freq = 50 MHz
USB boot = not available
Warranty = OK
Root size = 3.4G
Root used = 80%
Root available = 673M
Root expand = not expanded
Memory free = 86 MB
Memory used = 65 MB
SoC temperature = 54.1°C
CPU governor = ondemand
Onboard WiFi = None
Onboard BT = None
HDMI output = On
LED state = 1,1
Eth addr wait = On
Max USB current = Off
USB (UAC2) fix = Off
Pi-3B+ eth fix = Off
SSH term server = Off
PHP-FPM version = 7.3.14
NGINX version = 1.14.2
SQLite3 version = 3.27.2
Hostapd version = 2.8-devel
WiringPi version = 2.50
RPi.GPIO version = 0.7.0
A U D I O P A R A M E T E R S
Audio device = On-board audio device
Interface = SoC
Mixer name = Headphone
Hardware volume = Controller detected
Max ALSA volume = 100
Max MPD volume = 100
Volume step limit = 10
Audio source = Local
Output device = Local
Resume MPD = No
Volume knob = 0
Volume mute = Unmuted
Saved MPD vol = 0
Preamp volume = 0
ALSA version = 1.1.8-1+rpt1
SoX version = 0.1.2-3
Bluetooth controller = Off
Pairing agent = Off
Airplay receiver = Off
Spotify receiver = On
Squeezelite = Off
UPnP client = Off
DLNA server = Off
GPIO button handler = Off
UPnP browser = Off
Auto-shuffle = Off
Autoplay = Off
Rotary encoder = Off
Encoder params = 100 2 3 23 24
USB volume knob = Off
Polarity inversion = Off
Crossfeed = Off
Crossfade = Off
Parametric EQ = Off
Graphic EQ = Off
MPD httpd = Off
A P P E A R A N C E S E T T I N G S
Themes and backgrounds
----------------------
Theme = Default
Accent color = Emerald
Alpha blend = 1.0
Adaptive background = No
Background image = No
Cover backdrop = No
Cover blur = 20px
Cover scale = 1.25
Library options
----------------------
Instant play action = Add/Play
Show tagview genres = Yes
Show tagview covers = Yes
Show sample rate = No
Ellipsis limited text = No
Thumbnail columns = 6/2 (Default)
Albumview sort order = by Artist
Tagview sort order = by Artist
Compilation identifier = Various Artists
Recently added = 1 Month
Ignore articles = a,an,the
UTF8 character filter = No
Hi-res thumbs = Auto
Cover search pri = Embedded cover
Pixel ratio = 1
Coverview screen saver
----------------------
CoverView auto-display = Never
CoverView style = Gradient (Linear)
Other options
----------------------
Font size = Normal
Auto-shuffle filter = None
Extra metadata = track,disc,date,composer,encoded
Playback history = No
First use help = n,y
M P D S E T T I N G S
Version = 0.21.24
Volume control = software
ALSA device = hw:0
SoX resampling = disabled
SoX quality = very high
SoX multithreading = off
DSD over PCM (DoP) = no
Replaygain = off
Replaygain preamp = 0
Volume normalization = No
Audio buffer = 4096 (kb)
Output buffer size = 131072 (kb)
B L U E T O O T H S E T T I N G S
Bluetooth ver = 5.50
Bluealsa ver = v2.1.0
Speaker sharing = No
Resume MPD = No
PCM buffer time = 500000 (μs)
A I R P L A Y S E T T I N G S
Version = 3.3.6
Friendly name = Moode Airplay
ALSA device = hw:0
Interpolation = soxr
Output bit depth = S16
Output sample rate = 44100
Session interruption = no
Session timeout = 120 (ms)
Latency offset = 0.0 (secs)
Audio buffer = 0.2 (secs)
Resume MPD = No
S P O T I F Y S E T T I N G S
Friendly name = Moode Spotify
ALSA device = plughw:0
Bit rate = 160
Initial volume = 0
Volume curve = Logarithmic
Volume normalization = No
Normalization pregain = 0
Autoplay = No
Resume MPD = No
S Q U E E Z E L I T E S E T T I N G S
Version = 1.8.7-1052 "DSD/SRC enabled"
Friendly name = Moode
ALSA device = hw:0
ALSA params = 80:4::1
Output buffers = 40000:100000
Task priority = 45
Codec list = flac,pcm,mp3,ogg,aac,alac,dsd
Other options = -W -D 500 -R E -S /var/local
Resume MPD = No
L O C A L D I S P L A Y S E T T I N G S
Local UI display = Off
Mouse cursor = On
Screen blank = 600 Secs
Wake display on play = Off
Brightness = 255
Pixel aspect ratio = Default
Rotate screen = 0 Deg
M O O D E S T A R T U P L O G
20210328 170959 worker: -- Start
20210328 171000 worker: Successfully daemonized
20210328 171002 engine-mpd: Connection to MPD failed
20210328 171004 worker: Integrity check (passed)
20210328 171004 worker: Session loaded
20210328 171004 worker: Debug logging (off)
20210328 171004 worker: Device raw: (0:Headphones|1:empty|i2s:none)
20210328 171004 worker: Device mpd: (0:On-board audio device)
20210328 171004 worker: Device ses: (0|On-board audio device||Headphone|0%)
20210328 173544 worker: ALSA Headphone volume (0%)
20210328 173544 worker: -- System
20210328 173555 worker: Host (moode)
20210328 173556 worker: moOde (6.6.0 2020-07-09)
20210328 173556 worker: RaspiOS (10.4)
20210328 173556 worker: Kernel (5.4.49+ #1323)
20210328 173556 worker: Platform (Pi-1B 256MB)
20210328 173556 worker: ARM arch (armv6l, 32-bit kernel)
20210328 173556 worker: MPD ver (0.21.24)
20210328 173556 worker: CPU gov (ondemand)
20210328 173556 worker: USB boot not available
20210328 173556 worker: File system not expanded yet
20210328 173557 worker: HDMI port on
20210328 173600 worker: File check (OK)
20210328 173600 worker: -- Network
20210328 173600 worker: eth0 exists
20210328 173600 worker: eth0 wait 0 for IP address
20210328 173605 worker: eth0 wait 1 for IP address
20210328 173608 worker: eth0 wait 2 for IP address
20210328 173612 worker: eth0 wait 3 for IP address
20210328 173615 worker: eth0 wait 4 for IP address
20210328 173618 worker: eth0 address not assigned
20210328 173618 worker: wlan0 exists
20210328 173618 worker: wifi country (DE)
20210328 173618 worker: wlan0 trying SSID (esnet)
20210328 173619 worker: IP addr (192.168.178.83)
20210328 173620 worker: Netmask (255.255.255.0)
20210328 173620 worker: Gateway (192.168.178.1)
20210328 173621 worker: Pri DNS (192.168.178.1)
20210328 173621 worker: Domain (fritz.box)
20210328 173622 worker: -- Audio
20210328 173624 worker: MPD conf updated
20210328 173625 worker: ALSA outputs unmuted
20210328 173625 worker: ALSA card number (0)
20210328 173625 worker: Audio output (On-board audio device)
20210328 173628 worker: Audio formats (U8, S16_LE)
20210328 173628 worker: ALSA mixer name (Headphone)
20210328 173628 worker: MPD volume control (software)
20210328 173629 worker: Hdwr volume controller exists
20210328 173629 worker: Max ALSA volume (100%)
20210328 173629 worker: Reset renderer active flags
20210328 173629 worker: -- MPD
20210328 173630 worker: MPD started
20210328 173633 worker: MPD connection refused
20210328 173633 worker: Configure MPD outputs
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker:
20210328 173637 worker: MPD crossfade (off)
20210328 173637 worker: -- Feature availability
20210328 173637 worker: Source select (available)
20210328 173637 worker: Source select (source: MPD)
20210328 173637 worker: Source select (output: On-board audio device)
20210328 173638 worker: Bluetooth (available)
20210328 173638 worker: Airplay renderer (available)
20210328 173638 worker: Spotify renderer (available)
20210328 173638 worker: Spotify renderer (started)
20210328 173638 worker: Squeezelite renderer (available)
20210328 173638 worker: UPnP renderer (available)
20210328 173638 worker: DLNA server (available)
20210328 173638 worker: UPnP browser (available)
20210328 173638 worker: Audio scrobbler (available)
20210328 173638 worker: GPIO button handler (available)
20210328 173638 worker: -- Music sources
20210328 173639 worker: USB sources (none attached)
20210328 173639 worker: NAS and UPnP sources (none configured)
20210328 173639 worker: -- Miscellaneous
20210328 173639 worker: USB volume knob (Off)
20210328 173639 worker: USB auto-mounter (udisks-glue)
20210328 173640 worker: LED0 (On)
20210328 173640 worker: LED1 (sysclass does not exist)
20210328 173641 worker: Saved MPD vol level (0)
20210328 173641 worker: Preamp volume level (0)
20210328 173642 worker: MPD volume level (0) restored
20210328 173643 worker: ALSA Headphone volume (100%)
20210328 173643 worker: Auto-play (Off)
20210328 173643 worker: Maintenance interval (3 hrs)
20210328 173643 worker: Screen saver activation (Never)
20210328 173646 worker: Session permissions (OK)
20210328 173646 worker: Watchdog started
20210328 173646 worker: ReadyRE: Memory usage / mpd issues as of Moode 6.6.0? - TheOldPresbyope - 03-28-2021
@mafeu
I'm not sure Tim claims the 256KB RPi 1B is still a supported device. As you see in your logs and your top output, there's just not enough memory.
My personal opinion is that it's time to upgrade to a new RPi model. However, if you really, really want to continue using the RPi 1B you could try enabling swapping to a file. The resulting performance will be poor but perhaps it will be good enough for your purposes.
Regards,
Kent
RE: Memory usage / mpd issues as of Moode 6.6.0? - Tim Curtis - 03-28-2021
Quick test with 7.1.0 on a 1B / 256MB / Ethertnet and no issues on my end.
It's not obvious to me what might be happening with your config.
RE: Memory usage / mpd issues as of Moode 6.6.0? - mafeu - 03-28-2021
Hi Kent, Tim,
thanks for your quick replies. If you say that there is no visible misconfiguration (and straight from the ISO-image I only changed Timezone and WLAN configuration) my suspicion is either of these
- SD Card health, as kblockd, kswapd0 and high IO wait load seem to point to problems here. But why are the same cards working perfectly well with versions below 6.6.0 then?
- mpd, but maybe mpd is only collateral damage as it is the best choice to kill for oom-killer but not the root of the problem
- switch from kernel 4.x to 5.x, this is as far as I can tell the biggest change between versions 6.5.2 and 6.6.0. But why does nobody else seem to have this problem? It's running fine on Tim's 1B and I guess there are more setups like mine out there...
Well, maybe Kent's proposal is the best way forward. A 3B+ looks good, and I would get Bluetooth support, which would actually be quite useful to me.
Thanks for looking into the issue!
Cheers,
Martin
RE: Memory usage / mpd issues as of Moode 6.6.0? - TheOldPresbyope - 03-28-2021
I can repro the OP's top output on an RPi ZeroW by setting gpu_mem=320 in /boot/config.txt.
This gives me about the same total/free/used mem numbers and it also causes kswapd0 badness here.
So, a thought. AFAIK the default gpu_mem value for a 256MB RPi is 64MB while the minimum allowable is 16MB (consult the RPi docs).
The OP could try setting gpu_mem=16 in /boot/config.txt. Prolly have to do this on another host.
I just tried to approximate this on an RPi ZeroW by setting gpu_mem=272 (which is 256+16).
Code:
pi@ZeroW710:~ $ top -b -n 1
top - 17:34:20 up 3 min, 1 user, load average: 1.13, 1.06, 0.48
Tasks: 99 total, 1 running, 98 sleeping, 0 stopped, 0 zombie
%Cpu(s): 20.8 us, 20.8 sy, 0.0 ni, 54.2 id, 0.0 wa, 0.0 hi, 4.2 si, 0.0 st
MiB Mem : 225.9 total, 86.8 free, 78.9 used, 60.1 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 90.7 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1935 pi 20 0 10520 2992 2576 R 20.0 1.3 0:00.10 top
1382 mpd 20 0 173768 29180 19200 S 10.0 12.6 0:08.39 mpd
1 root 20 0 15260 5968 4452 S 0.0 2.6 0:09.03 systemd
...Playing radio stations without issue. No idea whether this would suffice for OPs purposes.
Regards,
Kent
Regards,
Kent
RE: Memory usage / mpd issues as of Moode 6.6.0? - mafeu - 03-29-2021
Hi Kent,
thanks a lot! Looks like reducing GPU memory to a minimum is doing the trick. 6.6.0 was running smoothly streaming radio for several hours while staying responsive at the web front end.
I'm back to 7.1.0 now already streaming for half an hour and no problems so far as well.
Cheers,
Martin