A developer tool to build a Moode image

[badbat75]

Hi all,
I want to share with you the tool I'm developing to accelerate the build of a moode image in order to speedup tests when a change in the build is included.
Here you can download the tool.
https://github.com/badbat75/rpi_moode_build
This is under development even if I released a version 1.0 dedicated to build a stable r4.0.
It is divided in 2 scripts.

• The first buildmoode.sh dedicated to create the empty image and mount everything in a chroot environment. It will also inject the second script that is the core of the build phase, derivated from the official recipe.
  I created this script in order to have full control in the developer toolchain I want to use to build the image, for this reason it is possible choose
  - the raspbian image version
  - the moode version (I don't know if it is really possible specify an different version from the tested one)
  - specify the compiler flags (-O3 in gcc and g++)
  - parallelize the build passing the -j [number of cores] to the Makefiles
  - decide to cache the building phase objects in order to speedup the following builds (ccache)
  - if you want the squashfs enabled or not
  - mount the image and use the shell to run commands as if you were in a moode environment
  - zip the image
  Before launching the build process it injects the second script in the /home/pi of the target image and insert on top a list of environment variable and runs it in the chrooted environment.
  
• The second runinchroot.sh is a bash script transposition of the official recipe.
  

So if you need to change something in the moode build you need to change the runinchroot.sh if you want to change something in the toolchain you have to change the parameters when you launch the script.
In the github repository you can see the README.md to config your build.
From the second build you can speed up the process to 4 times, on a RPi3 I got the entire build in 5 minutes. (depends on the updates)
At the end of the build you can find the logs of the two scripts in different files, to debug what happened during the build.

For example:

Code:

ZIP_FORMAT=XZ ./buildmoode.sh runinchroot.sh

You choose to use XZ to zip the image.

Code:

ENABLE_CCACHE=0 ENABLE_SQUASHFS=0 ./buildmoode.sh runinchroot.sh

You disable to cache the build, disable the squashfs filesystem.

Code:

DEV_MODE=1 ./buildmoode.sh

It reuse the current raspbian image and launch a bash in the chrooted environment. From here you can build manually what you want.

[Tim Curtis]

Very cool. Are you seeing significantly faster Build times with cache and compiler parallels?

-Tim

[badbat75]

MoodeBuild v1.1
Changelog:

• Work with Moode v4.1
  
• Added AtptX codec for Bluetooth (not included in the official build)
  
• Change bluealsa systemctl script to meet the new bluealsa version parameters (currently bluetooth is not working with official recipe)
  
• -O3 optimization during compiling (performance boost to be evaluated)
  
• Added resource folder to share content from host to chrooted environment (useful in future to add patch or files to be included in the build)
  
• Added possibility to choose compression algorythm with ZIP_FORMAT={ZIP|XZ}
  
• Added timestamps and build duration
  

To get it:
https://github.com/badbat75/rpi_moode_bu...e/v1.1.zip

Time to rebuild on an RPi3 with cache hit at 100% image extraction included (no OS updates available currently): 25mins (previous value was wrong)

[TheOldPresbyope]

MoodeBuild v1.1
Changelog:

• Work with Moode v4.1
  
• Added AtptX codec for Bluetooth (not included in the official build)
  
• Change bluealsa systemctl script to meet the new bluealsa version parameters (currently bluetooth is not working with official recipe)
  
• -O3 optimization during compiling (performance boost to be evaluated)
  
• Added resource folder to share content from host to chrooted environment (useful in future to add patch or files to be included in the build)
  
• Added possibility to choose compression algorythm with ZIP_FORMAT={ZIP|XZ}
  
• Added timestamps and build duration
  

To get it:
https://github.com/badbat75/rpi_moode_bu...e/v1.1.zip

Time to rebuild on an RPi3 with cache hit at 100% image extraction included (no OS updates available currently): 40mins

@badbat75

Thanks. I haven't had time to kick the tires and try the lights but I hope to soon.

Long ago I was a proponent of parallel-izing but IIRC Tim ran into a problem running parallel make jobs on every component and stopped doing it. I don't remember which software component was the culprit. I take it parallel-make jobs are working fine for you.

Regards,
Kent

[badbat75]

Hi guys,
I released a completely new build tool that offers more functionalities and boost up more the build of moode.

Changelog:

Code:

Complete refactoring of the script
There are a new command line implementation and a lot of new functions:

[*]Mount and unmount an image
[*]List the available build scripts in bbs directory
[*]New configuration parametres available.
[*]Dedicated configuration file in conf directory
[*]Possibility to add external resources inf resources directory (patches, files to add)
[*]For further details see the README.md

Added build for moode-4.2 (main differences from official build)
[*]Everything built with -O3 optimizations
[*]Included APT-X and AAC codecs in the alsa bluetooth stack
[*]Latency reduction via BUFFERSIZE parameter in /etc/bluealsaaplay.conf

You can find more information here:
https://github.com/badbat75/bbbuild/releases

And download it from here:
https://github.com/badbat75/bbbuild/archive/2.0.zip

Command line usage:
https://github.com/badbat75/bbbuild

You can build your moode running:

Code:

ZIP_FORMAT=XZ ./bbbuild moode-r42

[Tim Curtis]

Nice! I'll check it out.

If you wouldn't mind, create a Thread in the Test Team section to discuss APT-X and AAC codecs for Bluetooth.

-Tim

[badbat75]

Nice! I'll check it out.

If you wouldn't mind, create a Thread in the Test Team section to discuss APT-X and AAC codecs for Bluetooth.

-Tim

Oh yes, I will.

[FizzyTea]

Hello @badbat75 I have used your tool to build moode 4.2.
I entered interactive mode to add files ssh, wpa_supplicant.conf, moodecfg.txt before flashing using etcher and booting the image.
All seems good initially.
However when I attempt to edit /var/www/js/scripts-panels.js I am told (by nano)

Code:

File '/var/www/js/scripts-panels.js' is unwritable

Permissions look normal compared to permissions on moode 4.2 built with mosbuild. Root is the owner and has read-write permissions.
Doing some tests I try to create some blank files. Using 'sudo touch test' I am told

Code:

touch: cannot touch 'test': Read-only file system

For /var/www/js & /var/www this is the case.
I am however able to create a blank file (using sudo of course) in /var/

Any thoughts would be appreciated. Thanks.

[TheOldPresbyope]

Nice! I'll check it out.

If you wouldn't mind, create a Thread in the Test Team section to discuss APT-X and AAC codecs for Bluetooth.

-Tim

So I see openaptx is used to gain aptx capability. Quoting from the openaptx github README.md

This project is for research purposes only. Without a proper license private and commercial usage might be a case of a patent infringement. If you are looking for a library, which can be installed and used legally (commercial, private and educational usage), go to the Qualcomm® aptX™ homepage and contact Qualcomm customer service.

I'm just saying....

Regards,
Kent

[badbat75]

Hello @badbat75 I have used your tool to build moode 4.2.
I entered interactive mode to add files ssh, wpa_supplicant.conf, moodecfg.txt before flashing using etcher and booting the image.
All seems good initially.
However when I attempt to edit /var/www/js/scripts-panels.js I am told (by nano)

Code:

File '/var/www/js/scripts-panels.js' is unwritable

Permissions look normal compared to permissions on moode 4.2 built with mosbuild. Root is the owner and has read-write permissions.
Doing some tests I try to create some blank files. Using 'sudo touch test' I am told

Code:

touch: cannot touch 'test': Read-only file system

For /var/www/js & /var/www this is the case.
I am however able to create a blank file (using sudo of course) in /var/

Any thoughts would be appreciated. Thanks.

By default, /var/www is mounted on squashfs that is a readonly filesystem. If you want to disable during the build add ENABLE_SQUASHFS=0 before bbbuild command:

Code:

ENABLE_SQUASHFS=0 ZIP_FORMAT=XZ ./bbbuild moode-r42