Some musings on the PWA "add to homescreen" feature on Android devices.

Some musings on the PWA "add to homescreen" feature on Android devices. - Printable Version

+- Moode Forum (https://moodeaudio.org/forum)
+-- Forum: Community (https://moodeaudio.org/forum/forumdisplay.php?fid=18)
+--- Forum: General Discussion (https://moodeaudio.org/forum/forumdisplay.php?fid=21)
+--- Thread: Some musings on the PWA "add to homescreen" feature on Android devices. (/showthread.php?tid=7169)

Pages: 1 2

Some musings on the PWA "add to homescreen" feature on Android devices. - the_bertrum - 12-05-2024

TL;DR:

New PWA installations of the moOde on Android stopped working some time ago.
Correctly configuring SSL on moOde and the Android phone, and re-applying the manifest in header.php restores the basic functions of PWA.
The PWA would need a bit more work to make it as good as before even so.

Verbose version:
Some time ago I noticed that the "Progressive Web App" feature that allows the moOde UI to be launched in a full screen native app style interface on mobile devices had stopped working for new installations on Android devices. Existing apps on the home screen would continue to operate as normal, but a new "add to homescreen" now always just opens a new browser instance with address bars and tabs and all the other clutter that PWA was supposed to avoid. I decided to investigate, here's a dump of what I discovered.

Doing some research led me to suspect that the culprit was that PWAs need to be served over https. So I set about enabling the (experimental) https mode in one of my moOde players. The first hurdle was using the automatic mode, the certificate generated there just would not install on my phone, so I couldn't get any browser to trust the https connection to moOde. OK, so let's set ourselves up as a CA and authorise our own certificate, plenty of advice on how to do that on the internet, I mostly followed a guide on askssl.com. Quite a bit of other fiddling needed, but that's not the subject of this post, I might write that up later... Upshot is that I got a secure connection to my moOde installation from a browser on my phone in the end.

The "add to homescreen" when using ssl in Chromium based applications now changed to "open app", which did just that, opened the moOde interface in an app like window. Everything worked well enough in terms of the moOde interface, but there was no way to add a launch icon to the homescreen. Swapping to any other app would close the web app with no way back but to open the web browser again and use the "open app" command again.

A nose around in the moOde code (header.php) revealed that the link to the manifest json file is commented out, there are instructions that seems to be for IOS, and I suppose those are what was allowing the partial behaviour, but with the manifest missing I don't think the Android experience would be as good as it could be. I don't know when that change was made, or why, but it is simple to uncomment and reboot. Voila!, "open app" became "Install app" and that allows an app like experience to be installed.

The App experience would need a bit of finessing to make it really useable again I think; the top bar with the clock, notifications and system icons vanished behind a solid black bar for one thing. Very different to how it looks on a PWA installed before the Chromium upgrades that broke it in the first place. I'm sure playing with the manifest and so on could fix all that, but at this point I stopped poking around and reverted back to http and undid my hacks. I still happen to have old PWA links to all my players, so I still use those day to day. It was an interesting little excursion though and the things I found may be of use to others who might ant to go further down the path.

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - Tim Curtis - 12-05-2024

Regarding "The first hurdle was using the automatic mode, the certificate generated there just would not install on my phone". What errors did you get?

The Automatic mode cert is a self signed cert thats generated by /var/www/util/gen-cert.sh

Code:
#!/bin/bash

#

# SPDX-License-Identifier: GPL-3.0-or-later

# Copyright 2014 The moOde audio player project / Tim Curtis

#

# Template

OPENSSL_CFG_FILE=/tmp/moode-selfsigned.conf

cat >> $OPENSSL_CFG_FILE <<EOF

[ req ]

default_bits            = 2048

encrypt_key             = no

default_md              = sha256

string_mask             = nombstr

prompt                  = no

distinguished_name      = req_dn

req_extensions          = req_ext

[ req_dn ]

commonName              = $HOSTNAME.local

[ req_ext ]

basicConstraints        = critical, CA:FALSE

keyUsage                = digitalSignature, keyEncipherment, nonRepudiation

extendedKeyUsage        = clientAuth, serverAuth

subjectAltName          = @req_sans

[ req_sans ]

DNS.1                   = $HOSTNAME.local

DNS.2                   = $HOSTNAME

IP.1                    = 172.24.1.1

EOF

# Create cert

SSL_CSR_FILE=/tmp/moode.csr

SSL_CRT_FILE=/etc/ssl/certs/moode.crt

SSL_KEY_FILE=/etc/ssl/private/moode.key

openssl req -new -config $OPENSSL_CFG_FILE -out $SSL_CSR_FILE -keyout $SSL_KEY_FILE

openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -extensions req_ext

# TEST: Add to chromium-browser trust store

#sudo apt -y install libnss3-tools

#CERT_NICKNAME=moOde self-signed cert

#certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n $CERT_NICKNAME -i $SSL_CRT_FILE

# TEST: Add to RaspiOS/Debian trust store (needed?)

#sudo cp $SSL_CRT_FILE /usr/local/share/ca-certificates/

#sudo update-ca-certificates

Some googling and it looks like there are a lot of reports about broken PWA on Android/Chrome...
Not good.

As far as the site.manifest file goes, here is a line from the release note from 4.2.0 way back in 2018

Code:
################################################################################

#

# 2018-09-27 moOde 4.3.0 (Stretch)

#

################################################################################

Bug fixes

- FIX: Webmanifest ref breaks Add to Home on IOS

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - TheOldPresbyope - 12-05-2024

@the_bertrum

Nice detective work Cool

I wish I'd seen that mdn doc previously. it would have helped me understand what I was experiencing on various devices here.

Regards,
Kent

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - the_bertrum - 12-06-2024

(12-05-2024, 07:35 PM)Tim Curtis Wrote: Regarding "The first hurdle was using the automatic mode, the certificate generated there just would not install on my phone". What errors did you get?

The Automatic mode cert is a self signed cert thats generated by /var/www/util/gen-cert.sh

Code:
#!/bin/bash # # SPDX-License-Identifier: GPL-3.0-or-later # Copyright 2014 The moOde audio player project / Tim Curtis # # Template OPENSSL_CFG_FILE=/tmp/moode-selfsigned.conf cat >> $OPENSSL_CFG_FILE <<EOF [ req ] default_bits = 2048 encrypt_key = no default_md = sha256 string_mask = nombstr prompt = no distinguished_name = req_dn req_extensions = req_ext [ req_dn ] commonName = $HOSTNAME.local [ req_ext ] basicConstraints = critical, CA:FALSE keyUsage = digitalSignature, keyEncipherment, nonRepudiation extendedKeyUsage = clientAuth, serverAuth subjectAltName = @req_sans [ req_sans ] DNS.1 = $HOSTNAME.local DNS.2 = $HOSTNAME IP.1 = 172.24.1.1 EOF # Create cert SSL_CSR_FILE=/tmp/moode.csr SSL_CRT_FILE=/etc/ssl/certs/moode.crt SSL_KEY_FILE=/etc/ssl/private/moode.key openssl req -new -config $OPENSSL_CFG_FILE -out $SSL_CSR_FILE -keyout $SSL_KEY_FILE openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -extensions req_ext # TEST: Add to chromium-browser trust store #sudo apt -y install libnss3-tools #CERT_NICKNAME=moOde self-signed cert #certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n $CERT_NICKNAME -i $SSL_CRT_FILE # TEST: Add to RaspiOS/Debian trust store (needed?) #sudo cp $SSL_CRT_FILE /usr/local/share/ca-certificates/ #sudo update-ca-certificates

Some googling and it looks like there are a lot of reports about broken PWA on Android/Chrome...
Not good.

As far as the site.manifest file goes, here is a line from the release note from 4.2.0 way back in 2018

Code:
################################################################################ # # 2018-09-27 moOde 4.3.0 (Stretch) # ################################################################################ Bug fixes - FIX: Webmanifest ref breaks Add to Home on IOS

The certificate generated by the automatic method would not import into Android, the certificate import routine returns "private key required".

It definitely worked after 2018, so maybe the manifest isn't strictly needed after all. Either way it seems https is a must.

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - the_bertrum - 12-06-2024

(12-05-2024, 08:08 PM)TheOldPresbyope Wrote: @the_bertrum

Nice detective work

I wish I'd seen that mdn doc previously. it would have helped me understand what I was experiencing on various devices here.

Regards,
Kent

Thanks Kent, that mdn doc made so much clear that was not clear before Big Grin

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - Tim Curtis - 12-06-2024

(12-06-2024, 08:02 AM)the_bertrum Wrote:
(12-05-2024, 07:35 PM)Tim Curtis Wrote: Regarding "The first hurdle was using the automatic mode, the certificate generated there just would not install on my phone". What errors did you get?

The Automatic mode cert is a self signed cert thats generated by /var/www/util/gen-cert.sh

Code:
#!/bin/bash # # SPDX-License-Identifier: GPL-3.0-or-later # Copyright 2014 The moOde audio player project / Tim Curtis # # Template OPENSSL_CFG_FILE=/tmp/moode-selfsigned.conf cat >> $OPENSSL_CFG_FILE <<EOF [ req ] default_bits = 2048 encrypt_key = no default_md = sha256 string_mask = nombstr prompt = no distinguished_name = req_dn req_extensions = req_ext [ req_dn ] commonName = $HOSTNAME.local [ req_ext ] basicConstraints = critical, CA:FALSE keyUsage = digitalSignature, keyEncipherment, nonRepudiation extendedKeyUsage = clientAuth, serverAuth subjectAltName = @req_sans [ req_sans ] DNS.1 = $HOSTNAME.local DNS.2 = $HOSTNAME IP.1 = 172.24.1.1 EOF # Create cert SSL_CSR_FILE=/tmp/moode.csr SSL_CRT_FILE=/etc/ssl/certs/moode.crt SSL_KEY_FILE=/etc/ssl/private/moode.key openssl req -new -config $OPENSSL_CFG_FILE -out $SSL_CSR_FILE -keyout $SSL_KEY_FILE openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -extensions req_ext # TEST: Add to chromium-browser trust store #sudo apt -y install libnss3-tools #CERT_NICKNAME=moOde self-signed cert #certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n $CERT_NICKNAME -i $SSL_CRT_FILE # TEST: Add to RaspiOS/Debian trust store (needed?) #sudo cp $SSL_CRT_FILE /usr/local/share/ca-certificates/ #sudo update-ca-certificates

Some googling and it looks like there are a lot of reports about broken PWA on Android/Chrome...
Not good.

As far as the site.manifest file goes, here is a line from the release note from 4.2.0 way back in 2018

Code:
################################################################################ # # 2018-09-27 moOde 4.3.0 (Stretch) # ################################################################################ Bug fixes - FIX: Webmanifest ref breaks Add to Home on IOS

The certificate generated by the automatic method would not import into Android, the certificate import routine returns "private key required".

It definitely worked after 2018, so maybe the manifest isn't strictly needed after all. Either way it seems https is a must.

Does it just generate the error message and then done or are you prompted to import the private key file?

A message and then done would suggest Android needs the private key embedded in the cert. I think it's just a --keyout param in the openssl command that generates the cert.

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - the_bertrum - 12-06-2024

(12-06-2024, 10:22 AM)Tim Curtis Wrote:
(12-06-2024, 08:02 AM)the_bertrum Wrote:
(12-05-2024, 07:35 PM)Tim Curtis Wrote: Regarding "The first hurdle was using the automatic mode, the certificate generated there just would not install on my phone". What errors did you get?

The Automatic mode cert is a self signed cert thats generated by /var/www/util/gen-cert.sh

Code:
#!/bin/bash # # SPDX-License-Identifier: GPL-3.0-or-later # Copyright 2014 The moOde audio player project / Tim Curtis # # Template OPENSSL_CFG_FILE=/tmp/moode-selfsigned.conf cat >> $OPENSSL_CFG_FILE <<EOF [ req ] default_bits = 2048 encrypt_key = no default_md = sha256 string_mask = nombstr prompt = no distinguished_name = req_dn req_extensions = req_ext [ req_dn ] commonName = $HOSTNAME.local [ req_ext ] basicConstraints = critical, CA:FALSE keyUsage = digitalSignature, keyEncipherment, nonRepudiation extendedKeyUsage = clientAuth, serverAuth subjectAltName = @req_sans [ req_sans ] DNS.1 = $HOSTNAME.local DNS.2 = $HOSTNAME IP.1 = 172.24.1.1 EOF # Create cert SSL_CSR_FILE=/tmp/moode.csr SSL_CRT_FILE=/etc/ssl/certs/moode.crt SSL_KEY_FILE=/etc/ssl/private/moode.key openssl req -new -config $OPENSSL_CFG_FILE -out $SSL_CSR_FILE -keyout $SSL_KEY_FILE openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -extensions req_ext # TEST: Add to chromium-browser trust store #sudo apt -y install libnss3-tools #CERT_NICKNAME=moOde self-signed cert #certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n $CERT_NICKNAME -i $SSL_CRT_FILE # TEST: Add to RaspiOS/Debian trust store (needed?) #sudo cp $SSL_CRT_FILE /usr/local/share/ca-certificates/ #sudo update-ca-certificates

Some googling and it looks like there are a lot of reports about broken PWA on Android/Chrome...
Not good.

As far as the site.manifest file goes, here is a line from the release note from 4.2.0 way back in 2018

Code:
################################################################################ # # 2018-09-27 moOde 4.3.0 (Stretch) # ################################################################################ Bug fixes - FIX: Webmanifest ref breaks Add to Home on IOS

The certificate generated by the automatic method would not import into Android, the certificate import routine returns "private key required".

It definitely worked after 2018, so maybe the manifest isn't strictly needed after all. Either way it seems https is a must.

Does it just generate the error message and then done or are you prompted to import the private key file?

A message and then done would suggest Android needs the private key embedded in the cert. I think it's just a --keyout param in the openssl command that generates the cert.

There's no option to import a key, it just dumps you back out again.

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - Tim Curtis - 12-06-2024

If you want to experiment modify gen-cert.sh and add the -keyout param to embed the private key in the cert (not tested). Then see if the generated cert works in Android import.

Change:
openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -extensions req_ext

To:
openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -keyout $SSL_CRT_FILE -extensions req_ext

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - the_bertrum - 12-06-2024

(12-06-2024, 11:05 AM)Tim Curtis Wrote: If you want to experiment modify gen-cert.sh and add the -keyout param to embed the private key in the cert (not tested). Then see if the generated cert works in Android import.

Change:
openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -extensions req_ext

To:
openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -keyout $SSL_CRT_FILE -extensions req_ext

Alas, same error when using a cert generated for this code. Private key required.

Personally, I think if I were to go down the https route on my own players, I'd set up my own CA and sign a cert for all my players using that. It isn't a great deal more work than getting self signed certs into various stores and having them trusted (still can't get my linux mint installation to accept one either).

RE: Some musings on the PWA "add to homescreen" feature on Android devices. - Tim Curtis - 12-06-2024

(12-06-2024, 04:14 PM)the_bertrum Wrote:
(12-06-2024, 11:05 AM)Tim Curtis Wrote: If you want to experiment modify gen-cert.sh and add the -keyout param to embed the private key in the cert (not tested). Then see if the generated cert works in Android import.

Change:
openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -extensions req_ext

To:
openssl req -x509 -days 3650 -config $OPENSSL_CFG_FILE -in $SSL_CSR_FILE -key $SSL_KEY_FILE -out $SSL_CRT_FILE -keyout $SSL_CRT_FILE -extensions req_ext

Alas, same error when using a cert generated for this code. Private key required.

Personally, I think if I were to go down the https route on my own players, I'd set up my own CA and sign a cert for all my players using that. It isn't a great deal more work than getting self signed certs into various stores and having them trusted (still can't get my linux mint installation to accept one either).

ITs prolly some sort of cert format issue. I'll look into it more deeply when time permits, or if anyone else wants to dig into it don't hesitate :-)