[SOLVED] mpd error malformed utf-8 characters, possibly incorrectly encoded

[chaos333]

Cool. I'll add them to upcoming 8.1.0

thanks.

but still the mpd erro malform problem happen when i choose chinese album to play and not solve yet.
i though it was solved after clear cache. but
even after i reconnect the first Nas and read the chinese alburms, it happen again.


dont know how to proceed..

[the_bertrum]

Cool. I'll add them to upcoming 8.1.0

thanks.

but still the mpd erro malform problem happen when i choose chinese album to play and not solve yet.
i though it was solved after clear cache. but
even after i reconnect the first Nas and read the chinese alburms, it happen again.


dont know how to proceed..

Are there Chinese characters in the filename or path to the file?

[Tim Curtis]

Cool. I'll add them to upcoming 8.1.0

thanks.

but still the mpd erro malform problem happen when i choose chinese album to play and not solve yet.
i though it was solved after clear cache. but
even after i reconnect the first Nas and read the chinese alburms, it happen again.


dont know how to proceed..

That error which is referenced in your first post https://moodeaudio.org/forum/showthread....7#pid41037 looks like it might be coming from the front-end Javascript routine that tries to parse the JSON encoded metadata being sent by MPD.

Zip up one of the files and PM a download link to @TheOldPresbyope and myself and we will take a look at the Browser console with Debug logging turned on (System config).

The code block below from playerlib.js is what may be generating the error.
https://developer.mozilla.org/en-US/docs...JSON/parse
https://developer.mozilla.org/en-US/docs...yntaxError

Code:

// Always have valid json
try {
    MPD.json = JSON.parse(data);
   //console.log(MPD.json);
}
    catch (e) {
    MPD.json['error'] = e;
}

[chaos333]

Cool. I'll add them to upcoming 8.1.0

thanks.

but still the mpd erro malform problem happen when i choose chinese album to play and not solve yet.
i though it was solved after clear cache. but
even after i reconnect the first Nas and read the chinese alburms, it happen again.


dont know how to proceed..

That error which is referenced in your first post https://moodeaudio.org/forum/showthread....7#pid41037 looks like it might be coming from the front-end Javascript routine that tries to parse the JSON encoded metadata being sent by MPD.

Zip up one of the files and PM a download link to @TheOldPresbyope and myself and we will take a look at the Browser console with Debug logging turned on (System config).

The code block below from playerlib.js is what may be generating the error.
https://developer.mozilla.org/en-US/docs...JSON/parse
https://developer.mozilla.org/en-US/docs...yntaxError

Code:

// Always have valid json
try {
   MPD.json = JSON.parse(data);
   //console.log(MPD.json);
}
   catch (e) {
   MPD.json['error'] = e;
}

hi Tim,
thanks.
but how i can proceed to zip? sorry i know nothing of the linux scrip and i can only using ssh to log into the moode..
please if yo have time, show some detail step for a newbie like me to do.
thanks.

[chaos333]

Cool. I'll add them to upcoming 8.1.0

thanks.

but still the mpd erro malform problem happen when i choose chinese album to play and not solve yet.
i though it was solved after clear cache. but
even after i reconnect the first Nas and read the chinese alburms, it happen again.


dont know how to proceed..

Are there Chinese characters in the filename or path to the file?

sorry to say, but both...

path to the files + alburm name and song name..

[TheOldPresbyope]

Look up thread for my post about tests with synthetic files using either Greek or Chinese characters in metadata and in file names. Result: I didnt have a problem..

I'm still dubious about the specific metadata in the OP's WAV files. Looking forward to seeing the latest.

Regards,
Kent

[chaos333]

lucky you are kent.

still no solution yet.
but seems it will happen when i play chinese character albums.

and some times not. i dont know why.

[TheOldPresbyope]

@chaos333

Let's review, shall we?

As I understand it, there are three aspects to your issue

1. the "mpd error malformed UTF-8 characters..." message appears for your files when the UTF8 character filter preference is enabled in moOde 8.0.2
   
2. the artist name and album name for your files are represented by strings of question marks.
   
3. the presentation of information for your files in the playback panel differs in the WebUI vs the local display
   

In my testing here on a fresh install of moOde 8.0.2 using the WAV file contained in the chaos333.zip file you provided

1. I was unable to repro the appearance of the mpd error message
   
2. I found that the metadata in the file is represented in a mixture of RIFF INFO fields and ID3 frames. The strings of question marks are embedded as IPRD and IART values in the RIFF INFO chunk, NOT in the ID3 frames embedded in the same chunk. (I'm skipping over some details.)
   
3. the difference in presentations went away when I installed the Chinese font set that Tim suggested.
   

This is why I say I'm suspicious of the metadata contained in your files. 

In my limited testing, Kid3 does not touch the RIFF fields and so can't fix the contradiction between RIFF INFO and ID3 tags. This even though it indicates it knows there is an RIFF INFO chunk present. It does appear to succeed in reorganizing things though. I don't know if this reorganization is helpful, benign, or harmful.

No tool I commonly use to manipulate metadata in media files has shown itself able to edit the mixture of RIFF INFO metadata and ID3 metadata in your WAV file (exiftool looked promising but on examination I find it treats RIFF files as read-only). OTOH I'm not comfortable working with WAV files so perhaps I'm missing something simple. I suppose one could delete the RIFF INFO chunk and then create a new one containing only pre-defined RIFF fields. One might even be able to accomplish this using only ffmpeg but that's outside my experience.

I outlined an alternative approach: first use ffmpeg to convert the WAV file to FLAC (which gets rid of the inconvenient RIFF approach to metadata) and then touch up the metadata in the VORBIS_COMMENT block that results. It seems to me this could be scripted in several different ways but I suggest you ask on other, more general forums such as https://hydrogenaud.io/ or elsewhere.

Regards,
Kent

[Sehnsucht]

@chaos333

Let's review, shall we?

As I understand it, there are three aspects to your issue

1. the "mpd error malformed UTF-8 characters..." message appears for your files when the UTF8 character filter preference is enabled in moOde 8.0.2
   
2. the artist name and album name for your files are represented by strings of question marks.
   
3. the presentation of information for your files in the playback panel differs in the WebUI vs the local display
   

In my testing here on a fresh install of moOde 8.0.2 using the WAV file contained in the chaos333.zip file you provided

1. I was unable to repro the appearance of the mpd error message
   
2. I found that the metadata in the file is represented in a mixture of RIFF INFO fields and ID3 frames. The strings of question marks are embedded as IPRD and IART values in the RIFF INFO chunk, NOT in the ID3 frames embedded in the same chunk. (I'm skipping over some details.)
   
3. the difference in presentations went away when I installed the Chinese font set that Tim suggested.
   

This is why I say I'm suspicious of the metadata contained in your files. 

In my limited testing, Kid3 does not touch the RIFF fields and so can't fix the contradiction between RIFF INFO and ID3 tags. This even though it indicates it knows there is an RIFF INFO chunk present. It does appear to succeed in reorganizing things though. I don't know if this reorganization is helpful, benign, or harmful.

No tool I commonly use to manipulate metadata in media files has shown itself able to edit the mixture of RIFF INFO metadata and ID3 metadata in your WAV file (exiftool looked promising but on examination I find it treats RIFF files as read-only). OTOH I'm not comfortable working with WAV files so perhaps I'm missing something simple. I suppose one could delete the RIFF INFO chunk and then create a new one containing only pre-defined RIFF fields. One might even be able to accomplish this using only ffmpeg but that's outside my experience.

I outlined an alternative approach: first use ffmpeg to convert the WAV file to FLAC (which gets rid of the inconvenient RIFF approach to metadata) and then touch up the metadata in the VORBIS_COMMENT block that results. It seems to me this could be scripted in several different ways but I suggest you ask on other, more general forums such as https://hydrogenaud.io/ or elsewhere.

Regards,
Kent

I had loads of problems with dodgy tags in flac files depending on the player.  In the end I decided to clean them up by deleting all tags except the ones I care about.  In Linux you could:

Code:

metaflac --show-tag DATE --show-tag TITLE --show-tag GENRE --show-tag ARTIST --show-tag ALBUM --show-tag TRACKNUMBER --show-tag DISCNUMBER "$1" | metaflac --remove-all-tags --import-tags-from=- "$1"

Or write the tags to an intermediate file, editing them and then writing them back.

[Tim Curtis]

lucky you are kent.

still no solution yet.
but seems it will happen when i play chinese character albums.

and some times not. i dont know why.

Its most likely because the standard JSON format that moOde uses for data transfer between client and server does not support non-UTF8 characters that are present in "Many Chinese songs and song directories.." as described below by developer @lazybat

Code:

// Many Chinese songs and song directories have characters that are not UTF8 causing json_encode to fail which leaves the libcache json file empty. Replacing the non-UTF8 chars in the array before json_encode solves this problem (@lazybat).

He contributed some code a few years ago that replaces a pattern of non-UTF8 characters by ? to prevent an empty Library.

You could try to edit your file metadata to be JSON compliant or just live with the ?. You could also try contacting @lazybat on Github https://github.com/lazybat