[PROBLEM] [PROBLEM] 9.x: MPD crashing ad-hoc / stops playing

[kurt1970]

So, if you can't reproduce it - despite numerous log files from my side - it's not an issue???
Strange reaction...

[TheOldPresbyope]

So, if you can't reproduce it - despite numerous log files from my side - it's not an issue???
Strange reaction...

Seemed pretty honest to me and to be clear the issue lies in MPD and its stream decoder not in moOde itself.

Regards,
Kent

[kurt1970]

Don't entirely agree. moOde perfectly can decide when to restart the MPD process based on behavior in the logs, as all info is there.
This would indeed still result in hick-ups, which is absolutely normal due to the delay in restarting the MPD service, but at least music would keep playing.

In addtion, it's not related to that 1 radio station I mentioned.

It was also me bringing on the bug about the radio station update failing ad-hoc. Initially, the answer was also "can't reproduce here". At the end it was indeed a hard to trace bug, which is now fixed.

I'm not spending my precious time in posting things, copying logs, if it would be BS. I'm just trying to help making moOde an ever better product.

[philrandal]

Don't entirely agree. moOde perfectly can decide when to restart the MPD process based on behavior in the logs, as all info is there.
This would indeed still result in hick-ups, which is absolutely normal due to the delay in restarting the MPD service, but at least music would keep playing.

In addtion, it's not related to that 1 radio station I mentioned.

It was also me bringing on the bug about the radio station update failing ad-hoc. Initially, the answer was also "can't reproduce here". At the end it was indeed a hard to trace bug, which is now fixed.

I'm not spending my precious time in posting things, copying logs, if it would be BS. I'm just trying to help making moOde an ever better product.

An aggressive stance isn't going to get you any help.

Try a fresh full install of Moode, as already suggested (not just replacing one php file).

Cheers,

Phil

[kurt1970]

@philrandal, not aggressive at all. I've 3 moOde instances. 2 on live systems 1 on test. Next to that, the patch was proposed by @Tim Curtis. The other instances are fresh installs... So, no, this is not the solution.
Also, the issue is there since a while. It's just now, based on Tim's instructions, I went deeper in the issue and the logs.

"I'm just trying to help making moOde an ever better product." expresses my right intentions. I brought on a hard to trace bug, which is now fixed. Due to the timezone diff, I stayed up at night to try things out - happy to do that. This is proof of what I'm writing.

I'm not native English speaking, so my apologies if the way I express myself is misunderstood.

Look-up my system, and you'll understand I'm quite serious about audio and my appreciation for moOde.

Not looking forward to further escalate this, as it doesn't contribute to anything. Only thing I ask @Tim Curtis is to further investigate, as it's an issue.

[Tim Curtis]

So, if you can't reproduce it - despite numerous log files from my side - it's not an issue???
Strange reaction...

I never said it's not an issue for you, I said I can't reproduce it after considerable time testing and therefore I'm not able to troubleshoot it any further. A repro is essential for 99% of issues otherwise it's just a wild goose chase and a waste of everyones time.

Based on what I've observed the problem lies with the broadcaster and how they are encoding their streams, not with MPD or moode.

Also, the Radio stream monitor is not a solution. It's just a "hope for the best" workaround for certain types of external problems for example network or broadcaster issues.

[kurt1970]

@Tim Curtis , being myself in SW Dev I understand the challenge to fix an issue without a reproducer...
In this case it's a hi-res station in Chile. Latency between Chile (if it's hosted there?) and Belgium may be significant. Combined with a 96kHz/24bit stream, I understand that this is not always that evident. At home, latency and bandwidith is not an issue, but I've the same with a basic AAC stream of our National Public radio station hosted in Belgium. So, the issue is not related to 1 specific station and/or HD streams.

[the_bertrum]

I've a particular interest in the xrun condition, having been affected by it for long time and believing myself to be the only person how saw it.  Since it didn't happen much to other people, I put it down to my own location (either my ISP, the wires to my house, or my home network) and developed a little script that would recover things for me when it happened (posted above).  Then other people came forward who also saw this issue, and Tim included the MPD restart stream monitor option based on a daemon that another user had developed to restart their stream when this happened.  I use the built in monitor now and it works very well.

There are a number of thing that come out of this discussion as I see it:

1. The xrun is not a problem in moOde, other users can run the same station and never see it, some stations will xrun predictably at specific points, people who see xruns will often have some stations that never see them, and so on.  The evidence points to it being a stream plus location issue, i.e either the stream is poor somehow, or the delivery to moOde is poor somehow, or both.  Diagnosing that is next to impossible and isn't a moOde issue anyhow.
   
2. The xrun condition affects other players on the same stream at the same location (There is evidence in this thread of a Naim system that didn't even try to recover).  Further evidence that this isn't a moOde problem.
   
3. Certain streams in this discussion have an abnormally frequent rate of the xrun condition at a particular location.
   
4. The stream monitor works.  When it is enabled, it restarts the stream effectively.
   
5. The stream monitor did have an issue where it might clash with the monitor that tried to detect MPD crashes, a situation Tim has fixed.
   
6. During the investigations, the system of Kurt1970 has managed to get in a state where the stream monitor no longer works.
   

So what to do?  First, a fresh install has been proposed.  That would ensure that there are no quirks left over from the diagnostic tinkering, and is expected to result in a system with a stream monitor that will restart successfully (note that Tim needed a specific set of monitor settings in his testing).
Secondly, assuming that the situation after the fresh installation still isn't satisfactory, replacing the MPD restart with a simple stop/start might help, it would avoid any conflict with the MPD monitor certainly.  This can be tested by turning off the stream monitor, and using the script I posted for a proof of concept.

So to summarise, the xrun is an issue, the xrun is not a moOde issue, moOde does have a mechanism to cope with xruns (unlike other players), the xrun recovery is no longer working for Kurt1970.  A fresh install should fix the xrun recovery, and there is an alternative to try if for some reason it still isn't happy.

[kurt1970]

Hi Robert,

Thanks for your comprehensive summary! I'm at work at the moment of writing, but will come back on it later today.

What I see now, is that your post about the script completely missed my attention. Apologies for this.

Keep you posted.
Best,
k

[kurt1970]

Hi @the_bertrum , @Tim Curtis

Having some more time to reply.

Thanks again for your input.

The discussion is going more and more into the direction of it-is-well-or-not-a-moode-or-mpd issue, which, imo, this is not the right approach. moOde users are looking for an audiophile experience. Period. At least this is what the website says: “Audiophile streamer for the wonderful Raspberry Pi family”. So most of the users don’t care whether it’s mpd or a toaster making music play, cause they’ve chosen “moOde”.

Myself being in SW dev (Oracle), I fully understand things are sometimes out of your span of control, and it has to be investigated on a case per case basis.
In this case I totally agree the issue is mpd related (evidence in the “mpd” logs; socket remains open), but… moOde is a layer on top of it, and is able to anticipate on this by stopping/starting the mpd service when this situation is detected. And yes, this may cause an interruption in the stream, but at least it keeps playing.

Before answering your questions. The patch Tim suggested made the monitor didn’t kick in  anymore, didn’t retry anymore, causing a final “drop”. More specific, the connection didn’t drop, but kept on playing silence forever. This is also in my logs. I did a clean install, and get back to the origininal situation, where the monitor kicks in, but sometimes conflicts with a restarting mpd, with a crash and audio drop as result. This is basically the reason of my initial post.

There are a number of thing that come out of this discussion as I see it:
1. The xrun is not a problem in moOde, other users can run the same station and never see it, some stations will xrun predictably at specific points, people who see xruns will often have some stations that never see them, and so on.  The evidence points to it being a stream plus location issue, i.e either the stream is poor somehow, or the delivery to moOde is poor somehow, or both.  Diagnosing that is next to impossible and isn't a moOde issue anyhow.
Kurt: See my point of it's a moOde issue or not. Obviously location, but basically latency and bandwith are key. In my case I've the issue with a flac 96kHz/24bot station in Chile as well as with a ~96kb/s AAC stream here in Brussels at 30 km away.

2. The xrun condition affects other players on the same stream at the same location (There is evidence in this thread of a Naim system that didn't even try to recover).  Further evidence that this isn't a moOde problem.
Kurt: Yes, Naim is horrible on that (I've still 2 muso's). An internet hick-up is enough to permantly drop the stream.

3. Certain streams in this discussion have an abnormally frequent rate of the xrun condition at a particular location.
Kurt: Correct. Although, according to Belgian standards, I've a decent bandwidth (300 mbit/s). But... this doesn't say anything about the quality/stability.

4. The stream monitor works.  When it is enabled, it restarts the stream effectively.
Kurt: Correct. Point is that some restarts cause a crash of mpd, resulting in a permant audio drop, and the reason of opening this thread.

5. The stream monitor did have an issue where it might clash with the monitor that tried to detect MPD crashes, a situation Tim has fixed.
Kurt: Yes, @Tim Curtis identified a conflicting situation and proposed a patch. This patch actually caused the monitor not to intervene anymore, and mpd doing x-runs forever. So, no, this patch didn't work, hence my request to Tim to further investigate.

6. During the investigations, the system of Kurt1970 has managed to get in a state where the stream monitor no longer works.
Kurt: Correct. See 5. Hence, I did a fresh install.

@the_bertrum Now, I'll test your script. Give me some time, I'll come back on that.