[SOLVED] mpd error malformed utf-8 characters, possibly incorrectly encoded

[chaos333]

hi all,
did any body using moode meet this problem: 
   
mpd error malformed utf-8 characters, possibly incorrectly encoded. 

it happen that i have a lot of the Chinese Albums with which are using chinese characters as album name.
when i install moode8.0.2 and it display as ???. i know this might because there is no utf-8 support.

so i change setting during playing view at 
m-preferences-library (advcnced)-utf8 character filter-- and change from 'no' to 'yes'--update..
then this happen.

and i realize this issue mpd error malformed utf-8 characters, possibly incorrectly encoded happening every 4 seconds as warning sign.
so i re-set from 'yes' to No at the same path.
m-preferences-library (advcnced)-utf8 character filter-- and change from 'yes' to 'no'--update..
   
but problem remain and then it never solved.

please if somebody know how to solve, advice me how to do with it.

even though it can not solve the display of Chinese characters albums names but at lease turn this warning off?

i know re-writing the image is a one-stop solution. and if still no way to solve, i think i have to re-write all system..

thanks.

[Tim Curtis]

Try Menu, Configure, Library, CLEAR Library tag cache

[chaos333]

Try Menu, Configure, Library, CLEAR Library tag cache

hi tim, thanks for reply,

cleared-reboot--problem remains..

then i tried another time, cleared, rebooted, problem remains.

same showing MPD error Malformed UTF-8 characters, possibly incorrectly encoded (5)

is there any other setting that i need to carry out?

[Tim Curtis]

The only setting for UTF-8 encoding is the one in Preferences that you already tried. It was code submitted by a Chinese developer that causes certain UTF-8 character sequences to be replaced by "?" to prevent the Library cache from being corrupted.

[chaos333]

The only setting for UTF-8 encoding is the one in Preferences that you already tried. It was code submitted by a Chinese developer that causes certain UTF-8 character sequences to be replaced by "?" to prevent the Library cache from being corrupted.

thanks Tim,
seems i will have to re-build reburn the whole images then.

but is there a way to make the album view showing the correct chinese characters instead of the "???"
it is really a very helpful improvement if it solve for me.

or do i need to do anything to make my music file to be shown correctly but NOT the "???"

[Tim Curtis]

I don't know but below is the UTF -8 replacement function that was contributed by developer @lazybat some years back.

Code:

// UTF8 replace (@lazybat)
function utf8rep($some_string) {
    // Reject overly long 2 byte sequences, as well as characters above U+10000 and replace with ? (@lazybat)
    $some_string = preg_replace('/[\x00-\x08\x10\x0B\x0C\x0E-\x19\x7F]'.
        '|[\x00-\x7F][\x80-\xBF]+'.
        '|([\xC0\xC1]|[\xF0-\xFF])[\x80-\xBF]*'.
        '|[\xC2-\xDF]((?![\x80-\xBF])|[\x80-\xBF]{2,})'.
        '|[\xE0-\xEF](([\x80-\xBF](?![\x80-\xBF]))|(?![\x80-\xBF]{2})|[\x80-\xBF]{3,})/S',
        '--', $some_string );

    // Reject overly long 3 byte sequences and UTF-16 surrogates and replace with ?
    $some_string = preg_replace('/\xE0[\x80-\x9F][\x80-\xBF]'.
        '|\xED[\xA0-\xBF][\x80-\xBF]/S','--', $some_string );

    return $some_string;
}

I think there are some Chinese character encodings that are incompatible with the JSON data format thats used to encode the Library cache. Someone with expertise in Chinese software coding could prolly explain whats going on.

[TheOldPresbyope]

@chaos333

So @Tim Curtis said

Someone with expertise in Chinese software coding could prolly explain whats going on.

Full disclosure: I am not that person!

However, I'm always interested in learning more about metadata issues. 

As a starter, moOde does support UTF-8 by default.

For example, I just created two test FLAC files-

The first uses common Greek surnames for album title Δημήτρης, artist Ἰάκωβος, and track name Ἑλένη [I chose Greek because I know the alphabet and more-or-less can sound out words].  The names show up fine in moOde 8.0.2 in Library and Playback views. 

The second file uses, instead, popular Chinese surnames 李, 王, and 张 (copied and pasted from a website discussing names). These names also show up fine in moOde 8.0.2.

Neither file triggers display of ? marks (this is the accepted way to signal 'I can't display this character') nor causes error messages to pop up. It didn't matter if the UTF-8 character filter setting was off or on. All this is as I would expect since the files are properly encoded.

Note that each popular music file format such as FLAC, MP3/ID3, etc., has its own conventions for representing and encoding metadata. I'm working only with FLAC files here.

Granted, my naive exercise is not a definitive test of moOde's ability to deal with the entirety of Unicode and UTF-8 encoding but it does makes me wonder about your files. Firstly, are they properly encoded and, secondly, what is it about them which is causing a problem with moOde.

Could you zip up one or more files you know contain Chinese characters, upload the result to a file-sharing site such as Dropbox, Google Drive, or whatever, and send me the link in a private message on this forum? Tim may wish to be Cc'ed as well.

Regards,
Kent

PS - if you haven't already flashed a new copy of moOde, you could try regenerating the library instead of simply updating it. It took a full regeneration for me to get rid of residual effects of one, particularly ill-conceived experiment last fall.

[chaos333]

Hi prespyope,
thanks.
i will update to google driver tonight.

[chaos333]

hi, dear all,
here are the music files with the problems on both my hdmi monitor and webbrowsers.

https://drive.google.com/drive/folders/1...sp=sharing


the web browser google chromw shows only the name but no alburm information only "???"
   


and the hdmi-monitor is even worth and even the name of it is also ???


   

thank you.

i really dont know what happen so if i can get helped. that is really very thanks.

[OldNick]

hi, dear all,
here are the music files with the problems on both my hdmi monitor and webbrowsers.

https://drive.google.com/drive/folders/1...sp=sharing


the web browser google chromw shows only the name but no alburm information only "???"



and the hdmi-monitor is even worth and even the name of it is also ???




thank you.

i really dont know what happen so if i can get helped. that is really very thanks.

Interestingly enough, I had a DSD file last night that displayed ? for the sample rate.  I will investigate later today (time willing) and report back here if I can find anything useful...