Folder Update

[Nuvera]

Hi Tim,
Just upgraded to 8.1.2

When I update the database from a Single Folder the whole music collection is now updated. Is this the way the update works now ?

Thanks for a great program.

[Tim Curtis]

How do you know the whole database is updated instead of just the particular folder?

[Nuvera]

How do you know the whole database is updated instead of just the particular folder?

It takes longer and other folders that have new music are added to the database.

[Tim Curtis]

Post the moOde log and I'll have a look.

Code:

moodeutl -l

[Nuvera]

20220731 162245 worker: --
20220731 162245 worker: -- Start moOde 8 series
20220731 162245 worker: --
20220731 162245 worker: Successfully daemonized
20220731 162247 worker: File check (OK)
20220731 162248 worker: Session vacuumed
20220731 162248 worker: Session loaded
20220731 162248 worker: Debug logging (OFF)
20220731 162248 worker: --
20220731 162248 worker: -- Audio debug
20220731 162248 worker: --
20220731 162248 worker: ALSA cards: (0:IQaudIODAC | 1:empty | 2:empty | 3:empty
20220731 162248 worker: MPD config: (0:IQaudIO Pi-DAC PRO | mixerDigital) | card:0)
20220731 162248 worker: ALSA mixer actual (Digital)
20220731 162248 worker: ALSA Digital volume set to (0%)
20220731 162248 worker: --
20220731 162248 worker: -- System
20220731 162248 worker: --
20220731 162249 worker: Host (moode-Pi4)
20220731 162249 worker: moOde (8.1.2 2022-07-08)
20220731 162249 worker: RaspiOS (11.3)
20220731 162249 worker: Kernel (5.15.32-v8+ #1538)
20220731 162249 worker: Platform (Pi-4B 1.2 2GB)
20220731 162249 worker: ARM arch (aarch64, 64-bit)
20220731 162249 worker: MPD ver (0.23.5)
20220731 162249 worker: CPU gov (ondemand)
20220731 162249 worker: USB boot (enabled)
20220731 162249 worker: File sys (expanded)
20220731 162249 worker: HDMI port (Off)
20220731 162250 worker: Sys LED0 (On)
20220731 162250 worker: Sys LED1 (On)
20220731 162250 worker: --
20220731 162250 worker: -- Network
20220731 162250 worker: --
20220731 162250 worker: eth0 adapter exists
20220731 162250 worker: eth0 check for address (No)
20220731 162250 worker: IP addr (192.168.0.24)
20220731 162250 worker: Netmask (255.255.255.0)
20220731 162250 worker: Gateway (192.168.0.1)
20220731 162250 worker: Pri DNS (192.168.0.1)
20220731 162250 worker: Domain (Home)
20220731 162250 worker: wlan0 adapter exists
20220731 162250 worker: wlan0 country (GB)
20220731 162250 worker: wlan0 connected SSID is (Overboard2)
20220731 162250 worker: IP addr (192.168.0.22)
20220731 162250 worker: Netmask (255.255.255.0)
20220731 162250 worker: Gateway (192.168.0.1)
20220731 162250 worker: Pri DNS (192.168.0.1)
20220731 162250 worker: Domain (Home)
20220731 162250 worker: wlan0 power save disabled
20220731 162250 worker: --
20220731 162250 worker: -- Audio config
20220731 162250 worker: --
20220731 162250 worker: MPD conf updated
20220731 162250 worker: ALSA card number (0)
20220731 162250 worker: MPD audio output (IQaudIO Pi-DAC PRO)
20220731 162250 worker: Audio formats (S16_LE, S24_LE, S32_LE)
20220731 162250 worker: ALSA mixer name (Digital)
20220731 162250 worker: MPD mixer type (software)
20220731 162250 worker: Hdwr volume controller exists
20220731 162250 worker: Max ALSA volume (100%)
20220731 162250 worker: ALSA output mode (Default: plughw)
20220731 162250 worker: ALSA loopback (Off)
20220731 162250 worker: Reset renderer active flags
20220731 162251 worker: CamillaDSP (off)
20220731 162251 worker: --
20220731 162251 worker: -- MPD startup
20220731 162251 worker: --
20220731 162252 worker: MPD started
20220731 162252 worker: MPD accepting connections
20220731 162252 worker: MPD output 1 ALSA Default (On)
20220731 162252 worker: MPD output 2 ALSA Bluetooth (Off)
20220731 162252 worker: MPD output 3 HTTP Server (Off)
20220731 162252 worker: MPD crossfade (11 secs)
20220731 162252 worker: MPD ignore CUE files (no)
20220731 162252 worker: --
20220731 162252 worker: -- Music sources
20220731 162252 worker: --
20220731 162252 worker: USB source (InfoSafe)
20220731 162252 worker: NAS sources (none configured)
20220731 162252 worker: --
20220731 162252 worker: -- Feature availability
20220731 162252 worker: --
20220731 162252 worker: Source select (available)
20220731 162252 worker: Source select (source: MPD)
20220731 162252 worker: Source select (output: IQaudIO Pi-DAC PRO)
20220731 162300 worker: Bluetooth pairing agent (started)
20220731 162300 worker: Bluetooth (available: started)
20220731 162300 worker: Airplay renderer (available)
20220731 162300 worker: Spotify renderer (available)
20220731 162300 worker: Squeezelite (available)
20220731 162300 worker: RoonBridge renderer (not installed)
20220731 162300 worker: Multiroom sender (available)
20220731 162300 worker: Multiroom receiver (available)
20220731 162300 worker: UPnP renderer (available)
20220731 162300 worker: DLNA server (available)
20220731 162300 worker: GPIO button handler (available)
20220731 162300 worker: Stream recorder (n/a)
20220731 162300 worker: --
20220731 162300 worker: -- Other
20220731 162300 worker: --
20220731 162300 worker: USB volume knob (Off)
20220731 162300 worker: USB auto-mounter (udisks-glue)
20220731 162300 worker: Saved MPD vol level (0)
20220731 162300 worker: Preamp volume level (0)
20220731 162300 worker: MPD volume level (22) restored
20220731 162300 worker: ALSA Digital volume (100%)
20220731 162300 worker: Auto-play (Off)
20220731 162300 worker: CoverView toggle (-off)
20220731 162300 worker: Maintenance interval (120 minutes)
20220731 162300 worker: Screen saver activation (600)
20220731 162300 worker: Session permissions (OK)
20220731 162300 worker: Watchdog started
20220731 162300 worker: Ready
20220731 163300 worker: Job update_library
20220731 163300 mpdindex: Start
20220731 163300 thumb-gen: Start
20220731 163300 thumb-gen: Priority: Embedded cover
20220731 163300 thumb-gen: Res,Qual: Auto
20220731 163300 thumb-gen: Px ratio: 1
20220731 163300 thumb-gen: Th width: 100
20220731 163300 thumb-gen: Thm qual: 75
20220731 163300 thumb-gen: Scanning: NAS, SDCARD, InfoSafe
20220731 163647 thumb-gen: Done: 5565 folders scanned, 1 thumbs created, 5564 already in cache.
20220731 163855 mpdindex: Done: indexed 11303 artists, 5288 albums, 108884 songs
20220731 163855 worker: Job update_library done

[Tim Curtis]

I did a quick test using a USB stick (format is FAT32) where I added a single file to an existing folder and then in Folder view I updated just that folder. MPD correctly only added that new file to it's database and no others.

Code:

pi@moode:~ $ sudo truncate /var/log/mpd/log --size 0
pi@moode:~ $ sudo mv ./HEJ07-Black\ Crow.flac /media/VFAT128/Test_mixed/Adrian/
pi@moode:~ $ tail -f /var/log/mpd/log
Jul 31 17:56 : update: added USB/VFAT128/Test_mixed/Adrian/HEJ07-Black Crow.flac
^C
pi@moode:~ $

It's not obvious to me what might be happening in your case. What format is the file system on your storage device?

[TheOldPresbyope]

@Tim Curtis

FYI, I tried a slightly different test.

Test artifact: moOde 8.1.2 with SMB NAS share "OMV-Music"

1. update library ( m > Update library)

Code:

20220731 194844 worker: Job update_library
20220731 194844 mpdindex: Start
20220731 194844 thumb-gen: Start
20220731 194844 thumb-gen: Priority: Embedded cover
20220731 194844 thumb-gen: Res,Qual: Auto
20220731 194844 thumb-gen: Px ratio: 1
20220731 194844 thumb-gen: Th width: 100
20220731 194844 thumb-gen: Thm qual: 75
20220731 194844 thumb-gen: Scanning: NAS, SDCARD
20220731 194905 thumb-gen: Done: 526 folders scanned, 0 thumbs created, 526 already in cache.
20220731 194927 mpdindex: Done: indexed 726 artists, 518 albums, 5871 songs
20220731 194927 worker: Job update_library done

2. Add a new album to the NAS: OMV-Music/rips/A Change of Pace/Prepare the Masses/ containing 10 tracks (no judgment, please---this came as a test from elsewhere)

3. In the Library/Folder view, drill down to the rips folder and "Update this folder"

Like you, I see only the 10 new tracks in this folder being added in the mpd log, implying that only the chosen folder is being scanned, but additionally

Code:

20220731 210524 worker: Job update_library
20220731 210524 mpdindex: Start
20220731 210525 thumb-gen: Start
20220731 210525 thumb-gen: Priority: Embedded cover
20220731 210525 thumb-gen: Res,Qual: Auto
20220731 210525 thumb-gen: Px ratio: 1
20220731 210525 thumb-gen: Th width: 100
20220731 210525 thumb-gen: Thm qual: 75
20220731 210525 thumb-gen: Scanning: NAS, SDCARD
20220731 210549 thumb-gen: Done: 527 folders scanned, 1 thumbs created, 526 already in cache.
20220731 210614 mpdindex: Done: indexed 734 artists, 526 albums, 5881 songs
20220731 210614 worker: Job update_library done

thumb-gen seems to have scanned all sources. (It added just the one new album thumb as expected.) Scanning the OP's 5500+ folders and 100+K tracks could be "expensive."

The mpdindex counts in my second update might seem wacky but actually make sense. The additional album contained 10 tracks and it turns out their metadata collectively comprise 8 album names and 8 artists. (Like I said, it came from elsewhere for testing.) Seems like the operation itself wouldn't be "expensive", even for a large library, since it's just asking mpd for its current stats.

Or not?

Regards,
Kent

[Tim Curtis]

Here's the code. Basically same as doing mpc stats which returns results instantly.

daemon/worker.php

Code:

// Check for library update complete
function chkLibraryUpdate() {
    //workerLog('chkLibraryUpdate');
    $sock = openMpdSock('localhost', 6600);
    $status = getMpdStatus($sock);
    $stats = getMpdStats($sock);
    closeMpdSock($sock);

    if (!isset($status['updating_db'])) {
        sendEngCmd('libupd_done');
        $GLOBALS['check_library_update'] = '0';
        workerLog('mpdindex: Done: indexed ' . $stats['artists'] . ' artists, ' . $stats['albums'] . ' albums, ' .  $stats['songs'] . ' songs');
        workerLog('worker: Job update_library done');
    }
}

inc/mpd.php

Code:

function getMpdStats($sock) {
    sendMpdCmd($sock, 'stats');
    $resp = readMpdResp($sock);
    return parseDelimFile($resp, ': ');
}

[Nuvera]

The USB drive is formatted NTFS. I did some more checks this morning and the Folder update is still updating the whole drive.
This also happens if I update just the SD card.

Its not a big issue as the update only takes 3 to 4 min.

[TheOldPresbyope]

The USB drive is formatted NTFS. I did some more checks this morning and the Folder update is still updating the whole drive.
This also happens if I update just the SD card.

Its not a big issue as the update only takes 3 to 4 min.

The penny dropped as I drank the second cup of coffee. This time I added new albums in two different parts of the NAS directory structure, asked moOde/MPD to update only one of the affected folders, and found the new tracks from the other folder were updated as well.

Regards,
Kent