[SOLVED] RPi out of memory

[Sunfish]

I did several tests with libraries up to 3.6TB. 
1st test was with known good files, result was that the library was build properly and MoOde was working as expected. No extreme usage of memory noted.
2nd test was introducing an .mpdignore file in an already indexed folder. Again everything was working as expected. The memory usage on the other hand was jumping to about 85% of the available 0.98Gb, jumping back to less than 25% quickly.
Taking into account my first observation on files with long filenames, metadata etc, my conclusion is that the combination of long metadata fields in the audiofiles (FLAC) is causing mpdignore to choke. Recovery after this event has happened is, as far as I can tell, not possible because as soon as the MoOde Ui is opened the same event is triggered again. So the only question remaining which file is causing the problem this particular case, 
"php-fpm7.4" "mpdignore", or both.



For now I removed the files causing the problems to a different location so I can enjoy listening to MoOde 64bits.

How were you able to determine which files were triggering the issue?

I had a directory named "Hidden" in which I put a mpdignore-file containing an "*", all files in (sub)directories should be omitted.  Accidentally I noticed that the "Hidden" directory was visible in MoOde, therefore guessing it was causing problems because in the legacy version of Moode it works well. I removed that "Hidden" directory entirely, which 'solved' the problem, now mpdignore is behaving as it should.
The subject files are classical music. Checking the files revealed that filenames and/or artist and/or title in several cases have more than 140 characters. I did count the length of some of the entries. The files are correct Flac format but the naming/tagging is not yet in accordance the way I tag my music. (TBD)

[Sunfish]

I confess I overlooked your early remark about files with long metadata field entries but in any case it was made in the context of .mpdignore.

It seems to me we haven’t unraveled this combination of large number of files/directories, use of .mpdignore, and files with large metadata field entries.

Have you tried using a (presumably smaller) library containing only the files you suspect and without any .mpdignore file?

Note that using a .mpdignore file requires mpd to do pattern matching on all file and directory names in and below the directory containing it. I don’t know my way around the relevant mpd code and don’t know how efficient it is. 

Regards,
Kent

I have not tried this with the suspected files. It is quite time consuming. One of these days I will try it.
Regards
Wim

[Sunfish]

I confess I overlooked your early remark about files with long metadata field entries but in any case it was made in the context of .mpdignore.

It seems to me we haven’t unraveled this combination of large number of files/directories, use of .mpdignore, and files with large metadata field entries.

Have you tried using a (presumably smaller) library containing only the files you suspect and without any .mpdignore file?

Note that using a .mpdignore file requires mpd to do pattern matching on all file and directory names in and below the directory containing it. I don’t know my way around the relevant mpd code and don’t know how efficient it is. 

Regards,
Kent

I have not tried this with the suspected files. It is quite time consuming. One of these days I will try it.
Regards
Wim

So I copied the suspected files to a drive. At first without .mpdignore. All worked well, also after reboot and several times "update library".
The I inserted the .mpdignore into the top of the directory. Again all was working properly, all files were invisible, no jump in usage of memory. In fact the entire USB drive was invisible in the MoOde folderstructure. Therefore I made an seperate directory without mpdignore, after "update library" those files on the USB drive became visible. Again without any problems.
So the idea of too long filenames, metatags etc. is overboard.
Maybe back to the idea that the number of files/folders is causing problems.
Regards

[Tim Curtis]

Strange issue.

Did you check for hidden files, other than the .mpdignore file?

The reason I ask is that MacOS creates those annoying and unwanted "resource fork" hidden files whenever doing anything to an attached storage device. Then have to deleted via a script otherwise IIRC MPD will try to index them. I think Windows also creates some hidden files?

[Sunfish]

Strange issue.

Did you check for hidden files, other than the .mpdignore file?

The reason I ask is that MacOS creates those annoying and unwanted "resource fork" hidden files whenever doing anything to  an attached storage device. Then have to deleted via a script otherwise IIRC MPD will try to index them. I think Windows also creates some hidden files?

No hidden files present. No Mac in sight. I did remove the windows trashcan and the annoying System Volume Information folders.
But to add to this issue, and I'm totally frustrated now, is that my system crashed again. After the latest test with the alledged bad files, I swapped back in the original HD without those 'bad' files and no mpdignore, on which I did check the filesystem, and booted up the system. At first it worked fine but upon opening the UI after 10 or 15 seconds the system is hanging. After several hard reboots, killing the power is the only option left, this happens over and over again. I'm not able to get it out of this loop.
Checking memory usage, by means of Mobaxterm, showed memory was filled to the brim again.
At the moment playing the legacy version, which is behaving like it should, no hickups.

[Sunfish]

FYI.
As a test I replaced the RPi 3B+ with a RPi 4B 2Gb memory. MoOde is working without any problem. Updating the library is not causing extreme memory usage or freezing of the system.
At my place another 3Pi 3B+ is running Pihole with a brided AP. After upgrading the OS from 32bit to Rapberry Pi OS 64bit the system proved not stable. At least once a day the system froze. Changing back to Raspberry Pi OS 32bit solved this problem.

Without really being solved this issue, for me, is closed.
regards
Wim

[DRONE7]

Do check that your RPi is updated to latest Firmware.
May have some bearing on 32/64 bit stability ?

[Sunfish]

Do check that your RPi is updated to latest Firmware.
May have some bearing on 32/64 bit stability ?

Thanks for your advise, makes sense.
However in case of MoOde the version used is 8.3.3, so the kernel must be up to date.
Not being a linux expert, my understanding is that the kernel is included in the Moode image as it is based on the Raspberry Pi OS.
Regards
Wim

[TheOldPresbyope]

FYI.
As a test I replaced the RPi 3B+ with a RPi 4B 2Gb memory. MoOde is working without any problem. Updating the library is not causing extreme memory usage or freezing of the system.
At my place another 3Pi 3B+ is running Pihole with a brided AP. After upgrading the OS from 32bit to Rapberry Pi OS 64bit the system proved not stable. At least once a day the system froze. Changing back to Raspberry Pi OS 32bit solved this problem.

Without really being solved this issue, for me, is closed.
regards
Wim

So are you saying that simply running 64bit Raspberry Pi OS on your RPi 3B+ (eg, not even installing and running 64bit moOde) causes out-of-memory or instability problems? That makes this an issue for the Raspberry Pi Forum.

Regards,
Kent

[Sunfish]

So are you saying that simply running 64bit Raspberry Pi OS on your RPi 3B+ (eg, not even installing and running 64bit moOde) causes out-of-memory or instability problems? That makes this an issue for the Raspberry Pi Forum.

Regards,
Kent

Looks like a valid conclusion.
Both problems I encountered with 64b OS in combination with RPi 3B+ (2 differend boards) started after installing the OS.
I will open a thread on the Raspberry Pi Forum.