[SOLVED] RPi out of memory

[Sunfish]

Using RPi 3B+ with allo kali and allo piano 2.1 with a 4TB USB HD, running MoOde 8.3.3.
The problem discribed has occurred already before using the first 64bit version of MoOde. To avoid the problem I simply used the legacy version.
Last week I gave the 64bit version another try. Fresh install and after the library was scanned everything worked fine until I added some music to the HD. Now I clicked "Update library" under the M in the upper right corner.
Apparently the update succeeded, but after some time (minutes) the music stopped and MoOde stopped responding. Also SSH was not working.
After several reboots the same problem occurred over and over again.
During boottime I was able to open a SSH session. Here it showed that at first the memory was 0.28/0.98 MB, music was playing, but the memory was filling up until 0.90/0.98 MB. Because I was not able to do anything in SSH, no output of moodeutl -l is/was possible.
By means of journalctl I obtained the output of the previous bootlog. This output is way to long to fit into the max of 65535 bytes of this post therefore the lines showing the problem which appears 17 times in the logfile:

Code:

Jun 19 10:16:55.328673 moode kernel: [  18422]  1000 18422     1254       15    36864        0             0 df
Jun 19 10:16:55.328811 moode kernel: [  18964]    33 18964   224183   164164  1433600        0             0 php-fpm7.4
Jun 19 10:16:55.328967 moode kernel: [  18994]    33 18994    51349     2306   131072        0             0 php-fpm7.4
Jun 19 10:16:55.329098 moode kernel: [  18995]    33 18995    51349     2027   131072        0             0 php-fpm7.4
Jun 19 10:16:55.329231 moode kernel: [  18996]    33 18996    51267     1695   122880        0             0 php-fpm7.4
Jun 19 10:16:55.329354 moode kernel: [  18997]    33 18997    51353     2057   131072        0             0 php-fpm7.4
Jun 19 10:16:55.329516 moode kernel: [  19007]    33 19007    51353     2080   131072        0             0 php-fpm7.4
Jun 19 10:16:55.329799 moode kernel: [  19042]     0 19042     1234       16    45056        0             0 sleep
Jun 19 10:16:55.329944 moode kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=php-fpm7.4,pid=18964,uid=33
Jun 19 10:16:55.330078 moode kernel: Out of memory: Killed process 18964 (php-fpm7.4) total-vm:896732kB, anon-rss:652848kB, file-rss:1504kB, shmem-rss:2288kB, UID:33 pgt>
lines 474-527/527 (END)

any suggestions on the problem?
regards

[Sunfish]

ETA: I just found out that the mentioned problem only occurrs when opening the MoOde UI.
The memory fills up in about 10 seconds after opening the UI.

[Tim Curtis]

Out of memory situations are typically due to a memory leak bug in some component or some sort of resource size bug.

1. Does your configuration use any resources that contain a huge number of items for example a playlist or the music collection itself?
2. Are you using any options that process the MPD database for example Auto-shuffle random play?

The main challenge with troubleshooting an OOM issue is trying to reproduce it.

[Sunfish]

The library contains 9,590 albums with 134,472 songs, a total of 3.82 TB.
The HD holds folders for classical, pop and compilation albums
The pop section (the biggest one) is subdivided into letter of alphabet, artist, albums, cd1. Each folder also has an artist.jpg, also each album has one cover.jpg.
no playlist.

[Nutul]

Could it be that 10K albums eat up the memory when generating the html of the library...?
Can we think of something to see if this is the issue? Maybe a NOTIFY on screen every 500 or 1000 albums, to see if it hangs somewhere... for debug, of course.

[Tim Curtis]

Try doubling the following parameters in /etc/php/7.4/fpm/php.ini

Code:

post_max_size from 75M to 150M
max_input_vars from 10000 to 20000

Also in the Resource allocation section of MPD Config

Double the Output buffer size: (128 -> 256)
Disable the Input cache

[Sunfish]

Made the suggested changes.
The variable max_input_vars was already set at 32768. Maybe this is the default on 64B, as I did not made any changes before.
Unfortunately the minute I opened the UI, memory was filled up quickly and the system halted. I left the system hanging for about 15 minutes, but no recovery was noted.

[Tim Curtis]

Right, the input vars parameter is updated during the image build to 32K

The only other change I can think of to try and see if the issue is related to the size of the collection is to copy something like half your tracks to another storage device, plug that one in and then see if the OOM condition occurs with a smaller Library.

[Sunfish]

I will try to make a HD with about 1.5GB music later this week. Will update you on this.
In the meantime I found out that .mpdignore, which is working properly in the legacy version of MoOde, is omitted in the 64bit version. Maybe this is related. The section with .mpdignore contains files with unusual long entries in performer, artist etc. 
Will try to sort that out, time permitting.
regards

[the_bertrum]

I'm using an mpdignore file with 64bit and it's fine. My mpdignore just contains a single wildcard to exclude everything in the directory though. Is yours more complex?