[PROBLEM] library update in progress

[arick]

Dear Tim,
after initial building the library sometimes I need to update the library or selected folder. I can do that only one time or twice.  After that I see popup “library update in progress” and can’t update library anymore, have to regenerate it.  It takes about 2 days because the library includes 30000 albums.
This situation began in 6.5.2 and continued in 6.6.0 . What can I do?
Thank you
Arkady

[Tim Curtis]

I'm surprised it works at all given the number of albums because moOde is not designed for collections of that size.

Starting with moOde 6.5.0 the MPD database update and thumbnail generation processes were combined into a new "Library update" process which runs both tasks in parallel and is much easier to use than the old method that required running each task separately.

The new method only allows one instance of Library update be running until it completes. This is to ensure the integrity of MPD database update, thumbnail generation and the internal messages that control the process and parts of the UI.

[arick]

As I understood, the new library update process resumes automatically after reboot. I propose the most time needs to thumbnail cache, I don’t need it. Can I stop library update manually via SSH? Can I configure that don’t need the thumbnails generation? The new library update process never ends, It stayed active for several weeks

[Tim Curtis]

Nope, there is no auto-resume of the process after reboot and there are no SSH commands to control the process.

For your unique usage scenario and collection size you would need to make your own fork of the moOde project and learn how to modify the source code to meet your own particular needs.

[arick]

Nope, there is no auto-resume of the process after reboot 

But after rebooting library update still in progress. How can it be?

[TheOldPresbyope]

... The new library update process never ends, It stayed active for several weeks

Something is out of whack here.

What do your moOde and MPD logs say? 

Let's work the numbers based on data from one of my players.

moOde 6.6.0 on an RPi4B, Hi-band (5GHz) WiFi connection to AP, then Ethernet connection to OpenMediaVault NAS running on a modest SBC (Odroid HC1) and SSD disk storage.

Here's what happens when I update the library

Code:

20200714 101255 worker: Job update_library
20200714 101255 thmcache: Start
20200714 101255 thmcache: Scanning: NAS, SDCARD, UPNP, MUSIC
20200714 101635 thmcache: Done: 635 album dirs processed
20200714 102504 worker: Job update_library done

That's 12+ minutes to update 6980 tracks in 635 album directories. Note that only the thmcache process writes to the moOde log. MPD is configured to write to its own log.

Let's assume we're talking the same number of tracks per album on average since they are the basis of MPD's scan. A straightline extrapolation to 30 000 albums would project the update will take about 10 hours. This is nowhere near several days or weeks.

On your player does thmcache report to the moOde log that it is done? Have you tried tracking progress in the MPD log? 

From the command line you can execute the following

Code:

tail -f /var/log/mpd/log

You should see a progression of one or more lines per track, similar to these, until the update process is done

Code:

Jul 14 10:24 : update: added NAS/OMV-Music/Classical/London Symphony Orchestra, Andre Previn/G. Gershwin- Rhapsody In Blue -  An American In Paris - Concerto in F/04 London Symphony Orchestra, Andre Previn - George Gershwin - Piano Concerto In F - II. Andante con moto.flac
Jul 14 10:24 : update: added NAS/OMV-Music/Classical/London Symphony Orchestra, Andre Previn/G. Gershwin- Rhapsody In Blue -  An American In Paris - Concerto in F/02 London Symphony Orchestra, Andre Previn - George Gershwin - An American In Paris.flac
Jul 14 10:25 : update: added NAS/OMV-Music/Classical/London Symphony Orchestra, Andre Previn/G. Gershwin- Rhapsody In Blue -  An American In Paris - Concerto in F/01 London Symphony Orchestra, Andre Previn - George Gershwin - Rhapsody In Blue.flac
^C

In this case the listing stopped because the update is complete but you'll have to CTRL-C to end the "tail -f" process whether the update is complete or not.

What model RPi are you using?


Regards,
Kent

[arick]

Dear Kent,
I’m using RPI 3b+ -> WiFi 2.4n -> router -> Ethernet - > music library on PC. The library is generated about 2 days, updated about 15 hours, but after update job has done according to the logs, I CAN’T run it again for several weeks. I have to regenerate the library. I see the system error ‘library update in progress’ and can’t stop it. The rebooting can’t help. I’ve studied the logs for many times. It is developer problem.
Regards,
Arkady

[Tim Curtis]

Of course it may be some sort of bug in the code, out-of-memory condition or some other resource issue due to the size of the collection but unless you can provide a way for us to reproduce it there is no way to troubleshoot.

My tests with the Library update process always result in Moode log entries similar to what Kent posted below.

Code:

20200714 101255 worker: Job update_library
20200714 101255 thmcache: Start
20200714 101255 thmcache: Scanning: NAS, SDCARD, UPNP, MUSIC
20200714 101635 thmcache: Done: 635 album dirs processed
20200714 102504 worker: Job update_library done

[arick]

I have the same log

20200713 010030 worker: Job update_library
20200713 010031 thmcache: Start
20200713 010031 thmcache: Scanning: NAS, SDCARD, UPNP
some messages about incorrect ID3v2 tags
20200713 175750 thmcache: Done: 28917 album dirs processed
20200713 232509 worker: Job update_library done

But I can't start update again

[Tim Curtis]

Right, but I'm not able to reproduce the "can't start update again".

Until I'm able to do that it's not possible to troubleshoot.